Remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement

Abstract

The invention relates to the technical field of remote sensing images, in particular to a remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement, based on an MAA-SFENet model architecture, a multi-head adaptive attention (MAA) and attention driven feature association (ADFC) module is integrated, the MAA focuses on capturing fine local changes, and the ADFC module focuses on capturing fine local changes. And meanwhile, the global context is fused, and the ADFC focuses on processing large-scale high-dimensional data, so that the sensitivity to a spatial variation mode is enhanced. Besides, in order to further improve the adaptability to different environments, a spatial adaptive feature modulation (SAFM) module is introduced, and spatial information representation is dynamically adjusted, so that the detection precision under a complex background is improved. Experimental results on two SCD data sets show that the MAA-SFENet is superior to other advanced methods in qualitative and quantitative evaluation. The ablation experiment further verifies the important contribution of the three modules to performance improvement. Experimental results show that the MAA-SFENet has excellent performance and reliability in the remote sensing image SCD.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image technology, and in particular to a remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement. Background Art

[0002] Change detection (CD) is a key image interpretation task that aims to identify areas in remote sensing imagery that have changed over different observation periods. CD is typically categorized into binary CD (BCD) and semantic CD (SCD). While BCD has garnered significant research and application attention, its primary focus on identifying the location of change limits its ability to comprehensively analyze and understand relevant phenomena. In contrast, SCD addresses this limitation by analyzing land cover (LC) classes. Specifically, SCD reveals the types of LCs affected by change, enabling researchers to more deeply explore the processes, causes, and potential impacts of change. This comprehensive perspective is of great significance for applications such as urban dynamics research, environmental monitoring, resource surveys, and other related research topics.

[0003] Previous research has shown that modeling dependencies between two-temporal images is a key step in capturing semantic change. The proposed method uses change vector analysis to initially identify regions of change and then classifies these regions into LCs. In contrast, the work of [ 15 ] and [ 16 ] uses the Bayesian rule to iteratively estimate the change in LC types, comprehensively considering the joint prior probabilities of semantic categories before and after the change.

[0004] In recent years, with the outstanding performance of deep learning in automatic feature extraction from remote sensing imagery, deep learning-based SCD methods have garnered increasing attention. Deep learning-driven SCD methods can generally be categorized into three types: single-branch, dual-branch, and multi-task. Single-branch methods feed the difference generated by fusing two temporal images into a CNN to extract deep features of the changing objects. These methods equate the SCD task with direct semantic segmentation (SS). However, since the number of change categories is equal to the square of the number of LC categories, this leads to a severe class imbalance. Dual-branch methods classify the two temporal images separately and then determine the change category by comparing the classification results. However, the reliability of these methods relies on the accuracy of the classification process, and classification errors can accumulate, especially near object boundaries. In contrast, multi-task learning (MTL) methods combine SS and BCD, allowing LC classification and BCD to be performed simultaneously. This approach enhances collaborative learning between related tasks by sharing features, thereby improving feature representation and the accuracy of SCD. Recent research has demonstrated that multi-task methods offer significant advantages in SCD. For example, Yang et al. designed an asymmetric three-branch Siamese network to handle SS and CD tasks separately. Daudt et al. designed a three-branch CNN architecture. However, these methods still have limitations. These include insufficient ability to capture detailed features, which can lead to missed or misdetected small objects, and inaccurate boundary detection. Furthermore, in complex SCD scenarios, accurately locating the changed area and correctly identifying the change type remain significant challenges.

[0005] To address the above challenges, the present invention proposes a change detection method based on the SCD network (MAA-SFENet) that integrates multi-head adaptive attention and spatial feature enhancement. The multi-head adaptive attention (MAA) mechanism can dynamically focus on key areas in the input image, more accurately capture local changes and efficiently fuse global information. In addition, an attention-driven feature correlation (ADFC) module is designed to further enhance the representation ability of deep features and strengthen multi-scale feature correlation modeling, thereby improving the model's sensitivity to detail changes. In order to alleviate the interference of the unchanged background area on the changing area and further improve the detection accuracy, the spatial adaptive feature modulation (SAFM) module is introduced to realize the dynamic selection of representative features. Summary of the Invention

[0006] The purpose of the present invention is to provide a remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement. By combining multi-head adaptive attention, spatial adaptive feature modulation, attention-driven feature association and multi-task learning, it not only optimizes the extraction of detail changes, but also improves the accuracy and robustness of change detection in complex backgrounds.

[0007] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0008] A method for semantic change detection in remote sensing imagery based on multi-head adaptive attention and spatial feature enhancement is proposed. Based on the MAA-SFENet model architecture, this model adopts a Multi-Transformation Layer (MTL)-based SCD structure and consists of three main branches: two SS branches and a CD branch. Each branch is responsible for a different feature extraction and analysis task, working together to complete the SCD task. The two SS branches utilize an encoder integrated with the MAA mechanism to capture fine local features while fusing global information to generate high-quality semantic feature representations. Before entering the differential module, the semantic features are optimized using the SAFM module to enhance the model's ability to identify changing regions in complex backgrounds. The CD branch focuses on extracting change information from the optimized SS branches and performs a differential operation to initially locate and describe changing regions. During the decoding phase, the ADFC module deeply fuses the features of the three branches. This fusion enhances the expression of feature correlations while suppressing redundant information. Through this series of designs, MAA-SFENet achieves accurate detection of changing regions in complex backgrounds, significantly improving the robustness and adaptability of detection. The method includes the following steps:

[0009] S1: Construct a multi-task network architecture consisting of two semantic segmentation branches and a single change detection branch. The input dual-temporal remote sensing imagery is feature-encoded using a weight-shared Transformer encoder, and features are extracted using a multi-head adaptive attention module. The module generates an adaptive attention weight matrix based on dynamic spatial sampling points to capture local detail changes and fuse them with global context.

[0010] S2: The encoded bi-temporal features are processed through a spatially adaptive feature modulation module. Through a multi-scale feature decomposition and dynamic reorganization strategy, the input features are divided into multiple groups of sub-features and convolution and pooling operations of different scales are performed on them. After upsampling and splicing, modulation weights are generated. A nonlinear activation function is combined to enhance the spatial features and suppress complex background interference.

[0011] S3: In the decoding stage, an attention-driven feature association module is constructed to enhance the bi-temporal and difference features across scales through the coordinate attention mechanism. A hierarchical feature fusion strategy is used to model the correlation between spatial and channel dimensions to generate semantically consistent change-sensitive features.

[0012] S4: Through a multi-task joint optimization strategy, the bi-temporal semantic map output by the semantic segmentation branch is deeply fused with the binary change map output by the change detection branch. The semantic co-alignment loss function is used to constrain the semantic consistency of the unchanged regions, and the network parameters are jointly trained by combining the cross entropy loss and the binary change loss.

[0013] S5: Perform morphological optimization and conditional random field post-processing on the initial semantic change map output by the network to generate the final detection result containing change type and location information.

[0014] Furthermore, the multi-head adaptive attention mechanism (MAA) in step S1 enhances feature representation by setting multiple sparse spatial locations around each feature point and adaptively searching for target-related information; specifically, the MAA mechanism includes:

[0015] MAA consists of two input sequences f1 and f2, and the spatial information is embedded into the input feature f1 by point-by-point addition using sine-cosine position encoding. For each feature point, starting from the latest tracking result (the latest scale and position of the target in the feature map), M spatial positions are dynamically set according to the target scale. The interest information is searched at each position, and the offset with the corresponding feature point is output. The offset is added to the unit grid point G generated according to the dimension (W, H) of the input feature f1 to obtain the sampling position coordinates. At the same time, the position-encoded features are sent to another branch to generate the attention weight W a The feature f2 is adjusted through the fully connected layer and then sampled according to the calculated sampling coordinates. Finally, the features of interest are aggregated with their corresponding attention weights to generate adaptive features. The principle of the entire MAA module is defined as follows:

[0016]

[0017] Where p is the sensing point, W n 、W off and W a is a fully connected layer, W a is the attention weight, δ is the softmax function, and PE is the sine-cosine position encoding.

[0018] Furthermore, the Spatial Adaptive Feature Modulation (SAFM) module in step S2 effectively extracts and enhances the representation of local and global features through multi-scale decomposition and dynamic modulation mechanisms, thereby highlighting the significance of the changed area and suppressing background interference. Through spatial weighting and adaptive feature aggregation, SAFM improves the model's sensitivity to subtle changes and the discriminative power of feature representation, including:

[0019] SAFM divides the input features into four different sets of components and then feeds them into the Multi-Scale Feature Generation Unit (MFGU). The input feature X is evenly divided into four sets of sub-features {X0, X1, X2, X3} along the channel dimension. In the MFGU, the first component is processed using a 3×3 depthwise convolution, while the remaining components are sampled separately using a pooling operation.

[0020] For the input feature X, this process can be expressed as:

[0021] [X0,X1,X2,X3]=Split(X), (2)

[0022]

[0023] Where Split(·) corresponds to the channel splitting operation, DW-Conv3×3(·) is a 3×3 depth convolution, ↑ p (·) represents the fast implementation of upsampling features at a specific level to the original resolution p by the closest interpolation,↓ p / 2 i Indicates pooling the input features to p / 2 i Using this design, different features can be obtained from different representation subspaces. These features are then aggregated by concatenating these extracted short-term or long-term features on the channel dimension and performing a 1×1 convolution. This can be expressed as:

[0024]

[0025] Where Concat(·) represents the concatenation operation, and Conv1×1(·) represents the 1×1 convolution. After that, it is normalized by GELU nonlinearity to estimate the attention map, and the input X is adaptively adjusted by element product according to the estimated attention. This process can be written as:

[0026]

[0027] in represents the GELU function, is the element-wise product.

[0028] Furthermore, the attention-driven feature correlation module (ADFC) in step S3 adopts advanced feature enhancement and correlation calculation methods to improve the representation and modeling of multi-scale feature correlations, dynamically capture the multi-scale dependencies between query and supporting features, and thus more accurately describe the changing patterns in complex scenes. Specifically,

[0029] ADFC first uses the attention-based feature enhancement (AFE) module to enhance the query feature F through multi-scale space and channelq And supports feature F s , to enhance the representation ability of global information and local details.

[0030]

[0031] Then, an improved correlation calculation method is used to integrate features at different levels to generate a multi-layer correlation matrix with strong semantic alignment capability.

[0032]

[0033] Where C is the correlation matrix, where each element C ij represents the similarity between the i-th position of the query feature and the j-th position of the support feature. F' represents the expansion of the feature map into a planar vector. This module adopts a hierarchical feature fusion strategy, significantly improving the detection ability of changing regions by modeling the correlation of features across space and scale.

[0034] AFE enhances the integration of contextual feature information, better balancing the model's ability to process global and local details. Based on a coordinate attention mechanism, it enhances the ability to capture long-range spatial dependencies through accurate positional information. For input X, it encodes two channels along the horizontal and vertical axes. AFE introduces a third 3×3 convolutional path to capture multi-scale feature information. X is divided into g sub-features:

[0035] X=[X0,X i ,...,X g-1 ](10)

[0036] where X i ∈R c / / g×H×W , g is the number of sub-features.

[0037] The AFE module captures dependencies between all channels and minimizes computational complexity by modeling cross-channel information interactions. In the first two branches, encoding is applied to two spatial dimensions to promote channel interactions. In the third branch, a 3×3 convolution kernel is used to capture more features. Cross-spatial learning integrates features learned by the parallel subnetworks, adjusts weights, and generates the final feature map. The design of the parallel subnetworks enhances cross-dimensional interactions and establishes dependencies between them.

[0038] Furthermore, the multi-task joint optimization strategy in step S4 specifically includes:

[0039] For model training, three different loss functions are used: semantic segmentation loss L ss， Binary change loss L cd and semantic co-alignment loss L sca .

[0040] Ls is the cross entropy loss between the bi-temporal semantic segmentation map and the true value, expressed as:

[0041]

[0042] Where M is the total number of categories, i is the index of each category, and y i is the probability of the true category corresponding to category i, p i is the probability that the predicted category is classified as i.

[0043] L cd is the binary cross entropy loss between the predicted binary change map and the true change map, as follows:

[0044] L cd =-ylog(p)-(1-y)log(1-p) (12)

[0045] Among them, y represents the true value change map, and p represents the predicted binary change map.

[0046] In the SCD task, it is usually assumed that unchanged regions will maintain the same semantics, while changed regions will exhibit different semantic categories. Based on this, the SCA loss function is introduced. This function calculates the logical association between the predicted bi-temporal semantic maps, which can be expressed as:

[0047]

[0048] Where p1 and p2 are the predicted bi-temporal semantic segmentation maps, y is the true value change map, and α is the balance factor, which is generally set to 0.22. In summary, the total loss can be expressed as:

[0049] L=L ss +L cd +L sca (14)

[0050] Beneficial effects of the present invention:

[0051] The MAA-SFENet of the present invention significantly improves the ability to capture subtle changes by introducing the multi-head adaptive attention (MAA) mechanism. MAA dynamically generates multiple sparse spatial positions for each feature point and adaptively adjusts their correlation. This enables the model to focus on local details more accurately, improving the sampling efficiency and accuracy of key change areas in remote sensing images. Compared with the traditional attention mechanism with fixed parameters, MAA can dynamically optimize the attention weights according to the features of the input image and its context. This flexibility enables the model to effectively suppress background noise that is not related to the change, thereby enhancing the sensitivity to the boundary of the change. Experimental data show that the introduction of MAA has significantly improved the model in multiple evaluation indicators. For example, the mIoU and F1 values of the JL1 and SECOND datasets both show obvious performance gains, indicating the effectiveness of the MAA mechanism in detail capture and information fusion.

[0052] The spatial adaptive feature modulation (SAFM) module of the MAA-SFENet of the present invention further enhances the change detection capability of the model in complex backgrounds by dynamically adjusting the spatial representation of features. The SAFM module adopts a multi-scale feature generation method to divide the input features into multiple components, and enhances or suppresses feature expression through a dynamic modulation mechanism. This strategy allows the model to select the most representative features when dealing with environments with strong background interference, thereby improving the detection rate of changed areas. By weighting and combining features of different scales, SAFM enables the model to achieve a better balance between global and local information, highlighting the significance of key change areas and suppressing redundant information. Experimental results show that SAFM has achieved remarkable results in improving indicators such as mIoU and SeK, proving its ability to improve the accuracy of change detection in complex backgrounds.

[0053] The attention-driven feature correlation (ADFC) module of the present invention enhances the model's understanding of change patterns by improving feature representation and multi-scale correlation modeling. It aims to solve the common representation inconsistency problem in multi-scale feature fusion. ADFC first processes the input features using an attention-based feature enhancement (AFE) strategy to capture the relationship between global information and local details. Subsequently, by constructing a cross-layer correlation matrix, ADFC can effectively integrate features of different resolutions, thereby emphasizing important information related to the changing area. This process not only improves the consistency between deep features, but also enables the model to respond more flexibly to changes in complex scenes. This multi-scale feature correlation capability enables MAA-SFENet to surpass existing state-of-the-art methods in multiple evaluation indicators, especially in application scenarios that require refined change identification.

[0054] The overall architecture of MAA-SFENet of the present invention adopts a design based on multi-task learning (MTL), which integrates the two tasks of semantic segmentation (SS) and change detection (CD) to improve the overall performance. By jointly training the two tasks, the model can fully utilize the semantic association between the dual-phase images, making the feature extraction process more efficient. Within the framework of multi-task learning, the SS task provides richer contextual information for the CD task, and the feedback of the CD task can further optimize the feature learning of the SS task. This synergistic effect between tasks greatly improves the model's ability to recognize changed areas and reduces the impact of category imbalance. The experimental results clearly show that after adopting the MTL framework, MAA-SFENet exhibits higher detection accuracy and robustness, especially in complex environments and background conditions, and shows extremely strong adaptability.

[0055] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 Schematic diagram of the MAA-SFENet model architecture; the encoder integrates a Transformer architecture with multi-head adaptive attention and weight sharing. Semantic and variation information is enhanced through the SAFM module and ADFC module.

[0058] Figure 2 This is a schematic diagram of the MAA module architecture;

[0059] Figure 3 It is a schematic diagram of the SAFM module structure;

[0060] Figure 4 It is a structural diagram of the ADFC module; the AFE module is used to enhance the characteristics;

[0061] Figure 5 Schematic diagram of the AFE module architecture;

[0062] Figure 6 Schematic diagram of the qualitative results of the ablation experiment; (a) shows the scene of farmland being converted into buildings and roads, (b) shows the scene of a bare surface being converted into farmland, and (c) shows the scene of a building being converted into farmland. Yellow boxes indicate areas with significant differences.

[0063] Figure 7 Schematic diagram of qualitative comparison of different methods on the JL1 dataset; (a)-(c) show the detection performance of different models in the scene of farmland conversion to construction. Yellow boxes indicate areas with significant differences;

[0064] Figure 8 Schematic diagram of qualitative comparison results on the SECOND dataset; (a) shows the scene of a bare surface transformed into a building, (b) shows the scene of a bare surface transformed into a water body, and (c) shows the scene of a bare surface transformed into a building and low vegetation. Yellow and purple boxes indicate areas with significant differences.

[0065] Figure 9 Schematic diagram of the impact of the differential module and semantic segmentation on the results; yellow and dotted boxes indicate areas with significant differences. DETAILED DESCRIPTION

[0066] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0067] Example 1

[0068] This embodiment describes a remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement, and proposes a MAA-SFENet model architecture. The model adopts an MTL-based SCD structure and includes three main branches: two SS branches and one CD branch. Each branch is responsible for different feature extraction and analysis tasks, and works together to complete the SCD task. The two SS branches use an encoder integrated with the MAA mechanism to capture fine local features and fuse global information to generate high-quality semantic feature representations. Before entering the differential module, the semantic features are optimized by the SAFM module to enhance the model's ability to recognize changing areas under complex backgrounds. The CD branch focuses on extracting change information from the optimized SS branch and performs preliminary positioning and description of the changing area through differential operations. In the decoding stage, the ADFC module is used to deeply fuse the features of the three branches. This fusion enhances the expression of feature correlation while suppressing redundant information. Through this series of designs, MAA-SFENet achieves accurate detection of changing areas under complex backgrounds and significantly improves the robustness and adaptability of detection. It includes the following steps:

[0069] S1: Construct a multi-task network architecture consisting of two semantic segmentation branches and a single change detection branch. The input dual-temporal remote sensing imagery is feature-encoded using a weight-shared Transformer encoder, and features are extracted using a multi-head adaptive attention module. The module generates an adaptive attention weight matrix based on dynamic spatial sampling points to capture local detail changes and fuse them with global context.

[0070] S2: The encoded bi-temporal features are processed through a spatially adaptive feature modulation module. Through a multi-scale feature decomposition and dynamic reorganization strategy, the input features are divided into multiple groups of sub-features and convolution and pooling operations of different scales are performed on them. After upsampling and splicing, modulation weights are generated. A nonlinear activation function is combined to enhance the spatial features and suppress complex background interference.

[0071] S3: In the decoding stage, an attention-driven feature association module is constructed to enhance the bi-temporal and difference features across scales through the coordinate attention mechanism. A hierarchical feature fusion strategy is used to model the correlation between spatial and channel dimensions to generate semantically consistent change-sensitive features.

[0072] S4: Through a multi-task joint optimization strategy, the bi-temporal semantic map output by the semantic segmentation branch is deeply fused with the binary change map output by the change detection branch. The semantic co-alignment loss function is used to constrain the semantic consistency of the unchanged regions, and the network parameters are jointly trained by combining the cross entropy loss and the binary change loss.

[0073] S5: Perform morphological optimization and conditional random field post-processing on the initial semantic change map output by the network to generate the final detection result containing change type and location information.

[0074] Example 2

[0075] Overall network architecture

[0076] The overall structure of MAA-SFENet is as follows Figure 1As shown in the figure. The model adopts an MTL-based SCD structure and contains three main branches: two SS branches and one CD branch. Each branch is responsible for different feature extraction and analysis tasks, and works together to complete the SCD task. The two SS branches use the encoder integrated with the MAA mechanism to capture fine local features, while fusing global information to generate high-quality semantic feature representations. Before entering the differential module, the semantic features are optimized by the SAFM module to enhance the model's ability to recognize changing areas under complex backgrounds. The CD branch focuses on extracting change information from the optimized SS branch and performs preliminary positioning and description of the changing area through differential operations. In the decoding stage, the ADFC module is used to deeply fuse the features of the three branches. This fusion enhances the expression of feature correlation while suppressing redundant information. Through this series of designs, MAA-SFENet achieves accurate detection of changing areas under complex backgrounds, significantly improving the robustness and adaptability of detection.

[0077] MAA Module

[0078] In the original Transformer structure, an attention mechanism is required to learn the correlation between any two elements in the input features. This leads to high computational complexity and is easily interfered by semantically similar information. A multi-head adaptive attention (MAA) mechanism is used to dynamically adjust the weight of each attention head. Specifically, MAA enhances feature representation by setting multiple sparse spatial locations around each feature point and adaptively searching for target-related information. Unlike the original version that uses fixed parameters, the spatial position around each feature point in the adaptive attention mechanism is dynamically calculated in real time based on the target scale. This design enables the model to better focus on the target area, thereby effectively reducing the impact of background interference.

[0079] like Figure 2 As shown. MAA requires two input sequences, f1 and f2. Spatial information is embedded into the input feature f1 by point-by-point addition using sine-cosine position encoding. For each feature point, starting from the latest tracking result (the latest scale and position of the target in the feature map), M spatial positions are dynamically set according to the target scale. Interest information is searched at each position, and the offset with the corresponding feature point is output. The offset is added to the unit grid point G generated according to the dimension (W, H) of the input feature f1 to obtain the sampling position coordinates. At the same time, the position-encoded features are sent to another branch to generate the attention weight W a The feature f2 is adjusted through the fully connected layer and then sampled according to the calculated sampling coordinates. Finally, the features of interest are aggregated with their corresponding attention weights to generate adaptive features. The principle of the entire MAA module can be defined as follows:

[0080]

[0081] Where p is the sensing point, W n 、W off and W a is a fully connected layer, W a is the attention weight, δ is the softmax function, and PE is the sine-cosine position encoding.

[0082] SAFM module

[0083] To address challenges in the SCD task, such as complex background interference, difficulty capturing details in the target area, and insufficient multi-scale feature extraction, the SAFM module was introduced. This module effectively extracts and enhances local and global feature representations through multi-scale decomposition and dynamic modulation, thereby highlighting the saliency of changing regions and suppressing background interference. Through spatial weighting and adaptive feature aggregation, SAFM improves the model's sensitivity to subtle changes and the discriminative power of feature representation. As a result, the algorithm achieves high detection accuracy, robustness, and adaptability to complex backgrounds, low-contrast areas, and diverse scenarios.

[0084] Table 1 The impact of different grouping numbers on the results in MFGU

[0085]

[0086] like Figure 3 As shown in Figure 1, SAFM divides the input features into four different component sets, which are then fed into the Multi-Scale Feature Generation Unit (MFGU). The component sets are grouped according to Table 1. In the MFGU, the first component is processed using a 3×3 depthwise convolution, while the remaining components are individually sampled using a pooling operation. For the input feature X, this process can be expressed as:

[0087] [X0,X1,X2,X3]=Split(X), (2)

[0088]

[0089] Where Split(·) corresponds to the channel splitting operation, DW-Conv3×3(·) is a 3×3 depth convolution, ↑ p (·) represents the fast implementation of upsampling features at a specific level to the original resolution p by the closest interpolation,↓ p / 2 i Indicates pooling the input features to p / 2 i Using this design, different features can be obtained from different representation subspaces. These features are then aggregated by concatenating these extracted short-term or long-term features on the channel dimension and performing a 1×1 convolution. This can be expressed as:

[0090]

[0091] Where Concat(·) represents the concatenation operation, and Conv1×1(·) represents the 1×1 convolution. After that, it is normalized by GELU nonlinearity to estimate the attention map, and the input X is adaptively adjusted by element product according to the estimated attention. This process can be written as:

[0092]

[0093] in represents the GELU function, is the element-wise product.

[0094] ADFC module

[0095] In order to better capture feature correlation, the ADFC module is designed. It uses advanced feature enhancement and correlation calculation methods to improve the representation and modeling of multi-scale feature correlation, dynamically capture the multi-scale dependency between query and support features, and thus more accurately describe the changing patterns in complex scenes. Specifically, ADFC first uses the attention-based feature enhancement (AFE) module to enhance the query feature F through multi-scale space and channel q And supports feature F s , to enhance the representation ability of global information and local details.

[0096]

[0097] Then, an improved correlation calculation method is used to integrate features at different levels to generate a multi-layer correlation matrix with strong semantic alignment capability.

[0098]

[0099] Where C is the correlation matrix, where each element C ij Represents the similarity between the i-th position of the query feature and the j-th position of the support feature. F' represents the expansion of the feature map into a plane vector. This module adopts a hierarchical feature fusion strategy to significantly improve the detection ability of the change area by modeling the correlation of cross-space and cross-scale features. The design of ADFC enables the network to better adapt to CD tasks in various situations. The module structure is as follows Figure 4 shown.

[0100] AFE enhances the integration of contextual feature information, better balancing the model's ability to process global and local details. Based on a coordinate attention mechanism, it enhances the ability to capture long-range spatial dependencies through accurate positional information. For input X, it encodes two channels along the horizontal and vertical axes. AFE introduces a third 3×3 convolutional path to capture multi-scale feature information. X is divided into g sub-features:

[0101] X=[X0,X i ,...,X g-1 ] (10)

[0102] where X i ∈R c / / g×H×W , g is the number of sub-features.

[0103] like Figure 5 As shown in the figure, the AFE module captures dependencies between all channels and minimizes computational complexity by modeling cross-channel information interactions. In the first two branches, encoding is applied to two spatial dimensions to promote channel interactions. In the third branch, a 3×3 convolution kernel is used to capture more features. Cross-spatial learning integrates the features learned by the parallel subnetworks, adjusts the weights, and generates the final feature map. The design of the parallel subnetworks enhances cross-dimensional interactions and establishes dependencies between them.

[0104] Loss Function

[0105] For model training, three different loss functions are used: semantic segmentation loss Lss, binary change loss Lcd, and semantic co-alignment loss Lsca.

[0106] L s is the cross entropy loss between the bi-temporal semantic segmentation map and the true value, expressed as:

[0107]

[0108] Where M is the total number of categories, i is the index of each category, and y i is the probability of the true category corresponding to category i, p i is the probability that the predicted category is classified as i.

[0109] L c is the binary cross entropy loss between the predicted binary change map and the true change map, as follows:

[0110] L cd =-ylog(p)-(1-y)log(1-p) (12)

[0111] Among them, y represents the true value change map, and p represents the predicted binary change map.

[0112] In the SCD task, it is usually assumed that unchanged regions will maintain the same semantics, while changed regions will exhibit different semantic categories. Based on this, the SCA loss function is introduced. This function calculates the logical association between the predicted bi-temporal semantic maps, which can be expressed as:

[0113]

[0114] Where p1 and p2 are the predicted bi-temporal semantic segmentation maps, y is the true value change map, and α is the balance factor, which is generally set to 0.22. In summary, the total loss can be expressed as:

[0115] L=L ss +L cd +L sca (14)

[0116] Table 2 Effect of different α values on the results

[0117]

[0118] Dataset description and experimental setup

[0119] In this example, the experimental settings are introduced from three aspects: dataset, evaluation metrics, and experimental parameter settings.

[0120] Dataset

[0121] 1) SECOND dataset: Contains 4662 pairs of aerial images, including 2968 pairs for training and 1694 pairs for testing. Each image is 512×512 in size, contains RGB channels, and has a spatial resolution of 0.5 to 3 meters. The labels are marked with 7 categories: non-vegetated ground (abbreviated as NVG surface), trees, low vegetation, water, buildings, playgrounds, and non-changing classes. It is worth noting that 61.2% of the pixels correspond to the non-changing class, 29.9% of the pixels correspond to the NVG surface, and the remaining categories account for less than 5%. Due to the extreme class imbalance, high generalization ability of the model is required.

[0122] 2) JL1 dataset. The images come from the Jilin-1 remote sensing satellite and include 6,000 images, 4,000 of which are used for training and 2,000 for testing. Each image is 256 × 256 pixels, composed of RGB channels, and has a spatial resolution of 0.75 m. Compared to the annotation method used in the SECOND dataset, the JL1 dataset provides nine categories of "form-to" labels for bitemporal images, indicating the type of change between two time points. Before training, these nine-category labels are converted to the same six categories as in the SECOND dataset.

[0123] Evaluation indicators

[0124] The quantitative evaluation metrics of SCD include overall accuracy (OA), mean inter-linking (mIoU), separation kappa (SeK) coefficient, and F1 score (F1).

[0125] Experimental setup

[0126] Experiments were conducted on a server equipped with an NVIDIA GTX 4090 GPU, 24 GB of VRAM, and Windows 11. To maintain fairness, all models were trained for 50 epochs using publicly available code. Training was performed using the AdamW optimizer. The learning rate was dynamically adjusted using a linear decay schedule, starting with an initial learning rate of 1e-4. No other techniques were used during training besides those mentioned above.

[0127] Experimental results

[0128] In this example, a series of experiments were conducted to validate the practicality of our proposed method (AA-SFENet) and its key components. We then compared our proposed method with the state-of-the-art SCD method. Finally, we discussed the results and related issues of the network.

[0129] Ablation experiments

[0130] Table 3 Comparison of quantitative results of ablation experiments

[0131]

[0132]

[0133] 1) The role of the MAA module: The MAA module accurately captures local changes by highlighting the key areas and boundary areas between changes and invariances. Its adaptive attention mechanism dynamically assigns higher importance to key areas while suppressing irrelevant background noise, thereby increasing the model's sensitivity to subtle changes. This targeted attention not only improves feature recognition, but also allows for a more robust representation of small-scale changes. As shown in Table 3, the MAA module improves mIoU, SeK, OA, and F1 by 1.02%, 2.11%, 0.71%, and 0.43%, respectively. The accuracy rates have all improved, demonstrating the effectiveness of the MAA module in extracting local features and fusing global information. Figure 6 As shown in the figure, by introducing the MAA module, the structure and boundaries of the building are further refined, the segmentation and recognition results are better, and the results are closer to the ground truth changes. It shows that it is more sensitive to detail changes and can capture more accurate boundary information, especially in challenging scenarios where small changes need to be detected.

[0134] 2) The role of the ADFC module: The ADFC module improves the model's ability to detect changes by enhancing the correlation between multi-scale features and modeling features. This method uses spatial and channel attention to integrate information of different resolutions to effectively identify small, low-contrast changes. This cross-scale fusion not only improves the consistency and robustness of feature representation, but also helps to detect subtle changes in semantic segmentation and change detection tasks. As shown in Table 3, after adding the ADFC module, OA increased by 0.72%, mIoU increased by 1.29%, and F1 score increased by 0.60%, which fully demonstrates its effectiveness in improving the overall performance of the network model. Figure 6 As shown in Figure 3, after adding the ADFC module, the problem of missed detection of Nvg and buildings is solved, and the detected change areas can more accurately reflect the actual changes in the scene.

[0135] 3) The role of the SAFM module: The SAFM module dynamically adjusts spatial information through a complex enhancement strategy that preserves fine spatial details while integrating a broader global context, allowing the network to selectively enhance or suppress the features of each pixel. This dynamic adjustment is particularly effective in complex backgrounds. As shown in Table 3, the module's ability to fine-tune feature maps significantly improves mIoU by 1.00% and SeK by 1.96%, fully demonstrating the effectiveness of the module. In addition, as Figure 6 As shown in (a), the addition of SAFM significantly improves land cover prediction, accurately identifies road changes, and the detected changes are closer to the actual changes. Figure 6 In (b) and (c), the addition of the SAFM module can significantly improve the ability of change detection in complex backgrounds.

[0136] Comparative experiment

[0137] In order to comprehensively evaluate the performance of the proposed AA-SFENet, we further compared it with the SOTA methods on the SECOND and JL1 datasets. The compared methods include:

[0138] HRSCD-str.4: Based on HRSCD-str.3, this method transfers semantic features from the semantic prediction branch to the CD branch through skip connections, thereby achieving information sharing.

[0139] Bi-SRNet: An SCD architecture that considers semantic temporal features, improves single-temporal semantic representation and cross-temporal semantic correlation through two semantic reasoning blocks, and uses cosine loss to enhance the semantic consistency of CD results.

[0140] SSCD-1: As the basic model of Bi-SRNet, SSCD-1 significantly improves the semantic detection capability of the dual branch due to its semantic consistency loss.

[0141] DEFO-MTLSCD: This method uses a decoder-centric multi-task learning architecture to enhance the decoding function of CD by leveraging the feature differences between the two SS decoding branches through the decoder's cross-subtask feature interaction module and feature aggregation module.

[0142] CdSC: It uses cross-difference semantic consistency to mine the deep differences between bi-temporal features. At the same time, it further uses deep semantic representation to enhance the local correlation of difference information, significantly improving the model's discriminative ability within the changing area.

[0143] Comparative experiment on JL1

[0144] Table 4 Comparative experimental results on the JL1 dataset

[0145]

[0146]

[0147] Table 4 shows the quantitative results of the comparison methods and the proposed method. The proposed method achieves the highest performance on the JL1 dataset. In particular, it improves mIoU and SeK by 2.24% and 4.75% respectively compared to the second-best method CdSC. Figure 7 In (a), compared with other methods, MAA-SFENet uses MAA to enhance the extraction of local information and accurately identify the change information in the yellow wireframe area.

[0148] Figure 7 (b) illustrates the scene where bare ground transforms into a building. Most of the compared methods are affected by lighting conditions and lead to misclassification, while even the relatively effective CdSC method leads to under-segmentation. MAA-SFENet utilizes a spatial feature modulation mechanism to suppress background interference and accurately detect this change. Figure 7 As shown in (c), DEFO-MTLSCD incorrectly classifies buildings as ground due to the spectral similarity between building tops and non-vegetated ground. SSCD-1 also makes a similar, smaller error. Under the supervision of MAA and SAFM, MAA-SFENet accurately extracts the overall variation of buildings. Furthermore, within building areas, MAA-SFENet captures finer details than other methods.

[0149] Comparative experiment on SECOND

[0150] Table 5 Comparative experimental results on the SECOND dataset

[0151]

[0152] Table 5 shows the quantitative results obtained by the comparison methods and MAA-SFENet. MAA-SFENet still achieves the best results on the SECOND dataset, and compared with the second-best CdSC method, MAA-SFENet improves the comprehensive indicators of mIoU and SeK by 1.07% and 1.20% respectively.

[0153] like Figure 8 Even in complex scenes, MAA-SFENet can effectively distinguish between changing and unchanged areas. Figure 8 As shown in the yellow boxes in (a) and 8(c), due to the lighting conditions, the low vegetation and bare soil categories show similar textures and colors, making it difficult for most SCD networks to distinguish between low vegetation and bare soil. In addition, Figure 8 (a) shows an example where regions that change in the bitemporal semantic map are misclassified as the same category. MAA-SFENet effectively addresses these issues. Figure 8 In the purple box of (c), only MAA-SFENet accurately identifies the vegetation cover type compared with other methods. Figure 8 The yellow box in (b) shows a change from farmland to water. Accurately identifying this type of change is challenging due to the limited number of samples for categories like water; however, the proposed method performs well under these conditions. Compared to other methods, MAA-SFENet demonstrates smaller missed detection areas, closer edges to the ground truth, and overall superior performance.

[0154] Computational complexity

[0155] Table 6 Comparison of computational complexity of different models

[0156]

[0157] As shown in Table 6, to thoroughly evaluate MAA-SFENet, detailed comparisons were conducted with various methods using four key metrics. Compared to the complex algorithms used for comparison, MAA-SFENet has lower computational complexity. The experimental results clearly demonstrate that MAA-SFENet achieves the highest accuracy on both datasets. In summary, MAA-SFENet effectively balances computational efficiency and accuracy. However, it is worth noting that the current implementation requires significantly more hardware resources than other methods. Future research will focus on techniques such as knowledge distillation and the development of lightweight modules to further improve model efficiency.

[0158] The impact of differential modules and semantic segmentation on the results

[0159] A SCD method that integrates MAA and spatial feature enhancement is proposed, which significantly improves the detection accuracy and robustness in complex scenes. Compared with existing methods, the proposed method has obvious advantages in local information modeling and global information integration of feature extraction. However, in multi-task SCD, the impact of SS branch and CD branch on CD results needs further study. Figure 9 As shown in the figure, although the buildings within the yellow boxes are accurately identified in the SS map, they are over-segmented. Furthermore, the vegetation within the dotted boxes is not correctly identified on the CD map, which affects the reliability of the final detection results. Future work could improve the overall performance of SCD by optimizing the coordination of SS branches and introducing more effective differential strategies.

[0160] This paper proposes a SCD model named MAA-SFENet, which integrates MAA and spatial feature enhancement and shows superior performance in feature extraction and representation. In order to more richly capture the correlation between features, an ADFC module is designed. In addition, SAFM is incorporated to strengthen the spatial semantic representation and improve sensitivity to changes. In order to overcome the limitations of transformer in local information extraction, a MAA mechanism is designed. This mechanism is able to detect finer local changes while effectively integrating global context. Experimental results show that MAA-SFENet outperforms the SOTA method in all indicators on two public datasets. However, it is worth noting that the existing SCD structure has high hardware requirements. Future research will focus on the lightweight structural interaction between SS and CD branches to improve the robustness of the SCD structure.

[0161] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement, characterized by: The following steps are involved: S1: Construct a multi-task network architecture consisting of two semantic segmentation branches and a single change detection branch. The input dual-temporal remote sensing imagery is feature-encoded using a weight-shared Transformer encoder, and features are extracted using a multi-head adaptive attention module. The module generates an adaptive attention weight matrix based on dynamic spatial sampling points to capture local detail changes and fuse them with global context. S2: The encoded bi-temporal features are processed through a spatially adaptive feature modulation module. Through a multi-scale feature decomposition and dynamic reorganization strategy, the input features are divided into multiple groups of sub-features and convolution and pooling operations of different scales are performed on them. After upsampling and splicing, modulation weights are generated. A nonlinear activation function is combined to enhance the spatial features and suppress complex background interference. S3: In the decoding stage, an attention-driven feature association module is constructed to enhance the bi-temporal and difference features across scales through the coordinate attention mechanism. A hierarchical feature fusion strategy is used to model the correlation between spatial and channel dimensions to generate semantically consistent change-sensitive features. S4: Through a multi-task joint optimization strategy, the bi-temporal semantic map output by the semantic segmentation branch is deeply fused with the binary change map output by the change detection branch. The semantic co-alignment loss function is used to constrain the semantic consistency of the unchanged regions, and the network parameters are jointly trained by combining the cross entropy loss and the binary change loss. S5: Perform morphological optimization and conditional random field post-processing on the initial semantic change map output by the network to generate the final detection result containing change type and location information.

2. The remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement according to claim 1, characterized in that: The multi-head adaptive attention module MAA in step S1 enhances feature representation by setting multiple sparse spatial locations around each feature point and adaptively searching for target-related information, specifically including: MAA consists of two input sequences f1 and f2. Spatial information is embedded into the input feature f1 by point-by-point addition using sine-cosine position encoding. For each feature point, M spatial positions are dynamically set according to the target scale starting from the latest tracking result. Interest information is searched at each position, and the offset with the corresponding feature point is output. The offset is added to the unit grid point G generated according to the dimension (W, H) of the input feature f1 to obtain the sampling position coordinates. At the same time, the position-encoded features are sent to another branch to generate the attention weight W. a ; Feature f2 is adjusted through the fully connected layer and then sampled according to the calculated sampling coordinates; finally, the features of interest and their corresponding attention weights are aggregated to generate adaptive features; the principle of the entire MAA module is defined as follows: Where p is the sensing point, W n 、W off and W a is a fully connected layer, W a is the attention weight, δ is the softmax function, and PE is the sine-cosine position encoding.

3. The remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement according to claim 1, characterized in that: The spatial adaptive feature modulation module SAFM in step S2 effectively extracts and enhances the representation of local and global features through multi-scale decomposition and dynamic modulation mechanism, thereby highlighting the significance of the changed area and suppressing background interference; Through spatial weighting and adaptive feature aggregation, SAFM improves the model's sensitivity to subtle changes and the discriminative power of feature representation, including: SAFM divides the input features into four different sets of components and then feeds them into the multi-scale feature generation unit MFGU; the input feature X is evenly divided into four groups of sub-features {X0, X1, X2, X3} along the channel dimension. In MFGU, the first component is processed using a 3×3 depth convolution, while the remaining components are sampled separately using a pooling operation; For the input feature X, this process can be expressed as: [X0,X1,X2,X3]=Split(X), (2) Where Split(·) corresponds to the channel splitting operation, DW-Conv3×3(·) is a 3×3 depth convolution, ↑ p (·) represents the fast implementation of upsampling features at a specific level to the original resolution p by the closest interpolation,↓ p / 2 i Indicates pooling the input features to p / 2 i Using this design, different features can be obtained from different representation subspaces; These extracted short-term or long-term features are then aggregated by concatenating them on the channel dimension and performing a 1×1 convolution; this can be expressed as: Where Concat(·) represents the concatenation operation, Conv1×1(·) represents the 1×1 convolution; in obtaining the aggregate representation After that, it is normalized by GELU nonlinearity to estimate the attention map, and the input X is adaptively adjusted by element-wise product according to the estimated attention; this process is expressed as: in represents the GELU function, is the element-wise product.

4. The remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement according to claim 1, characterized in that: The attention-driven feature correlation module (ADFC) in step S3 uses advanced feature enhancement and correlation calculation methods to improve the representation and modeling of multi-scale feature correlations, dynamically capture the multi-scale dependencies between query and supporting features, and thus more accurately describe the changing patterns in complex scenes. Specifically, ADFC first uses the attention-based feature enhancement module AFE to enhance the query feature F through multi-scale space and channel q And supports feature F s , to enhance the representation of global information and local details; Then, an improved correlation calculation method is used to integrate features at different levels to generate a multi-layer correlation matrix with strong semantic alignment capabilities. Where C is the correlation matrix, where each element C ij represents the similarity between the i-th position of the query feature and the j-th position of the support feature; F' represents the expansion of the feature map into a plane vector; this module adopts a hierarchical feature fusion strategy to significantly improve the detection ability of the changed area by modeling the correlation of cross-space and cross-scale features; AFE enhances the fusion of contextual feature information and better balances the model's ability to process global and local details. It is based on the coordinate attention mechanism and enhances the ability to capture spatial long-range dependencies through accurate position information. For input X, it encodes two channels along the horizontal and vertical axes. AFE introduces a third 3×3 convolution path to capture multi-scale feature information. X is divided into g sub-features: X=[X0,X i ,...,X g-1 ] (10) where X i ∈R c / / g×H×W , g is the number of sub-features; The AFE module captures the dependencies between all channels and minimizes computational complexity by modeling cross-channel information interactions; in the first two branches, encoding is applied to two spatial dimensions to promote channel interactions; in the third branch, 3×3 convolution kernels are used to capture more features; cross-space learning integrates the features learned by the parallel sub-networks, adjusts the weights and generates the final feature map; the design of the parallel sub-networks enhances cross-dimensional interactions and establishes dependencies between them.

5. The remote sensing image semantic change detection method based on multi-head adaptive attention and spatial feature enhancement according to claim 1, characterized in that: The multi-task joint optimization strategy in step S4 specifically includes: For model training, three different loss functions are used: semantic segmentation loss L ss， Binary change loss L cd and semantic co-alignment loss L sca ; Ls is the cross entropy loss between the bi-temporal semantic segmentation map and the true value, expressed as: Where M is the total number of categories, i is the index of each category, and y i is the probability of the true category corresponding to category i, p i is the probability that the predicted category is classified as i; L cd is the binary cross entropy loss between the predicted binary change map and the true change map, as follows: L cd =-ylog(p)-(1-y)log(1-p) (12) Where y represents the true value change map, and p represents the predicted binary change map; In the SCD task, it is usually assumed that unchanged regions will maintain the same semantics, while changed regions will show different semantic categories. Based on this, the SCA loss function is introduced. This function calculates the logical association between the predicted bi-temporal semantic maps, which can be expressed as: Where p1 and p2 are the predicted bi-temporal semantic segmentation maps, y is the true value change map, and α is the balance factor, which is generally set to 0.22; L=L ss +L cd +L sca (14) The total loss is shown in Equation 14.

Images