Multi-priori guided remote sensing change detection deep learning adaptive Token merging method and system

Abstract

The invention relates to a multi-priori guided remote sensing change detection deep learning adaptive Token merging method and system, and belongs to the technical field of artificial intelligence and remote sensing image processing. The method comprises the following steps: extracting a Token feature sequence of a dual-temporal remote sensing image; the method comprises the following steps that: firstly, a key region Token is input into a multi-priori guided adaptive Token merging module (ATM-CD), and the module generates semantic priori, change priori, structure boundary priori and Token importance priori in parallel and fuses the priori into a merging guide index, so that constrained Token merging is executed, only redundant background Token is compressed, and key region Token is protected; the combined Token sequence is input into a space and channel collaborative attention enhancement module (SCSA-CEFF), and feature enhancement is carried out through cooperation of multi-semantic space attention and progressive channel self-attention; and finally decoding and outputting a change detection result. According to the method, key information such as a change area and a structure boundary is effectively protected while significant weight reduction of the Transform model is realized, the precision and efficiency of change detection are improved, and the method has practicability.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and remote sensing image processing technology, and relates to a deep learning adaptive token merging method and system for remote sensing change detection guided by multiple priors. Background Technology

[0002] With the rapid development of high-resolution remote sensing technology, remote sensing images have been widely used in fields such as land monitoring, urban expansion analysis, and disaster assessment. Semantic change detection, as a key task in remote sensing image understanding, aims to automatically identify and locate semantic changes in land cover from images of different time phases. In recent years, models based on the Transformer architecture have shown significant advantages in remote sensing change detection tasks due to their powerful global context modeling capabilities, and have gradually become the mainstream technical approach.

[0003] However, applying the Transformer to high-resolution remote sensing images requires segmenting the image into a large number of patches and converting them into token sequences for processing. With high-resolution remote sensing images, the sheer number of tokens causes the computational complexity and memory consumption of the self-attention mechanism to increase quadratically, severely limiting the model's efficiency in practical deployments.

[0004] To alleviate this problem, researchers have proposed lightweight model techniques such as Token Merging (ToMe). These techniques typically merge redundant tokens based on feature similarity or spatial proximity to reduce sequence length. However, such methods are mostly task-agnostic general strategies. In the specific task of remote sensing change detection, images contain critical regions that are essential to the task, such as areas with semantic changes and structural boundaries between different land features. General merging strategies struggle to distinguish these critical regions from homogeneous background areas, easily leading to the blurring or loss of key information during the merging process, thus impairing the accuracy of change detection.

[0005] On the other hand, existing remote sensing change detection methods still have room for improvement in feature enhancement mechanisms. For example, one type of method, represented by Chinese patent CN119169449B, enhances semantic understanding capabilities by introducing multimodal information such as natural language descriptions; another type of method, such as Chinese patent CN120336574B, enhances feature representation capabilities through multi-level feature extraction and matching. However, these feature enhancement modules are often independent of, or even contradictory to, the model's lightweighting process: lightweighting may compromise feature integrity, while complex feature enhancement modules may exacerbate the computational burden. How to design a module that can work collaboratively with the lightweighting process and maintain or even enhance feature discriminative power after token intelligent compression remains a challenge that has not yet been fully resolved.

[0006] Therefore, there is an urgent need for an adaptive token merging method for remote sensing image semantic change detection tasks that can perceive and protect key task information, so as to ensure the accuracy and reliability of change detection while effectively reducing the weight of the Transformer model. Summary of the Invention

[0007] In view of this, the technical problem that this invention actually aims to solve is: the huge computational overhead caused by the large number of tokens generated when processing high-resolution images in existing Transformer-based remote sensing change detection models, and the problem that existing general token merging methods easily destroy key change areas and structural boundary information during compression, thereby impairing detection accuracy.

[0008] To solve the above technical problems, On the one hand, this invention provides a deep learning adaptive token merging method for remote sensing change detection guided by multiple priors, applied to a remote sensing image semantic change detection model based on the Transformer architecture. This method includes the following steps: Step S1: Input the registered dual-temporal remote sensing images into the feature encoder to extract the token feature sequences corresponding to the two times.

[0009] Step S2: Input the token feature sequence into the multi-prior-guided adaptive token merging module, i.e., ATM-CD. The ATM-CD module performs the following operations: S2.1 For each Token in the Token feature sequence, generate semantic priors, change priors, structural boundary priors, and Token importance priors in parallel; Based on the four priors mentioned above, S2.2 calculates a merged bootstrapping metric for each Token; S2.3 Performs a constrained merging operation on the token sequence based on the merging guidance index, compressing only redundant background tokens and automatically protecting tokens in changing, boundary, and important semantic regions; wherein, merging is only allowed for the token pair when the merging guidance index of the candidate merging token pair is less than the preset threshold, and the compressed token feature sequence is output.

[0010] Step S3: Input the compressed dual-temporal token feature sequence into the spatial and channel collaborative attention enhancement module, i.e., SCSA-CEFF. Through the synergistic effect of multi-semantic spatial attention and progressive channel self-attention, the discriminative power of feature differences is enhanced, and the change feature map is obtained.

[0011] Step S4: Decode the change feature map to generate the final change detection result.

[0012] Furthermore, in step S2.1, (1) The process of generating semantic priors includes: Define a set of learnable semantic prototype vectors; for each token's feature vector, calculate its similarity to each semantic prototype vector and normalize it to obtain a semantic attribution distribution; the semantic prior value is calculated based on the maximum confidence in the semantic attribution distribution, and the smaller the semantic prior value, the purer the semantics of the token.

[0013] (2) The process of generating change priors includes: for spatially aligned bi-temporal token feature sequences, calculating the distance metric of their feature vectors; normalizing the calculated distance metric; inputting the normalized distance metric into an adjustable Sigmoid function, whose output is used as the change prior. The larger the change prior value, the greater the probability that the position will change.

[0014] (3) The process of generating structural boundary priors includes: extracting edge intensity maps from the dual-temporal remote sensing images respectively; obtaining edge response values ​​from the edge intensity maps according to the image location corresponding to the Token; fusing the edge response values ​​at the corresponding locations in the dual-temporal images and normalizing them to obtain structural boundary priors. The larger the structural boundary prior value, the more likely the Token is to be located at the boundary of the ground feature.

[0015] (4) The process of generating the Token importance prior includes: calculating the degree of dependence of each Token based on the self-attention matrix in the feature encoder; calculating the channel energy of each Token feature vector as its activation intensity; fusing the degree of dependence and activation intensity and normalizing them to obtain the Token importance prior. The larger the importance prior value, the more important the Token is in the feature representation.

[0016] Furthermore, in step S2.2, the guiding indicators are merged. The calculation formula is:

[0017] in, , , , These represent semantic priors, change priors, structural boundary priors, and token importance priors, respectively. , , , These are the learnable weight coefficients.

[0018] Furthermore, in step S3, the SCSA-CEFF module includes a serially connected multi-semantic space attention submodule SMSA and an asymptotic channel self-attention submodule PCSA. The SMSA submodule is used to: decompose the input features into multiple sets of sub-features along the spatial height and width dimensions, extract multi-semantic spatial context information through convolution at different scales, and generate spatial attention maps to modulate the input features; The PCSA submodule is used to: spatially downsample the features modulated by SMSA, perform self-attention calculation to model the inter-channel relationships, generate channel attention maps, and perform channel recalibration on the features modulated by SMSA.

[0019] On the other hand, the present invention also provides a remote sensing image semantic change detection system and a computer-readable storage medium; The system includes a processor and a memory. The memory stores a computer program. When the program is executed by the processor, it implements the token merging method. The ATM-CD module and the SCSA-CEFF module work together in the system. The compressed token feature sequence output by the ATM-CD module has higher semantic consistency and structural integrity and serves as the input to the SCSA-CEFF module. The aforementioned readable storage medium stores a computer program that, when executed by a processor, implements the multi-prior-guided remote sensing change detection deep learning adaptive token merging method.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) By using controlled token merging guided by multiple priors, the number of tokens is significantly reduced (approximately 50-70%), the computational complexity is reduced (from O(n²) to O((n / 2)²)) and the memory consumption is reduced (more than 40%), while the information of the key areas of the task is effectively protected, thus achieving model lightweighting without sacrificing or even improving the change detection accuracy.

[0021] (2) It integrates change priors and structural boundary priors specifically designed for remote sensing change detection tasks, enabling the token merging process to have a high degree of task awareness and scene adaptability, especially improving the model's robustness in detecting subtle and edge changes.

[0022] (3) The collaborative design of the ATM-CD module and the SCSA-CEFF module makes the lightweight process and the feature enhancement process no longer mutually restrictive, but mutually reinforcing. ATM-CD provides compressed input to SCSA-CEFF, improving the effectiveness of the attention mechanism; SCSA-CEFF further enhances the feature representation of the protected key region.

[0023] (4) The significantly improved inference efficiency makes the method described in this invention easier to deploy on resource-constrained edge computing platforms, expanding the application scenarios of high-precision remote sensing change detection technology.

[0024] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 A schematic diagram of the fusion decision-making process for an adaptive token merging module guided by multiple priors; Figure 2 The fusion steps of the adaptive token merging module guided by multiple priors; Figure 3 A structural diagram of the attention enhancement module for spatial and channel coordination. Detailed Implementation

[0026] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0028] This invention mainly provides a lightweight change detection model that combines a deep learning adaptive token merging method for remote sensing change detection guided by multiple priors with a spatial-channel collaborative attention module (SCSA-CEFF). Figure 1-3 A schematic diagram of the key modules of the present invention is shown.

[0029] Example 1: Overall Model Architecture 1. System Overall Architecture The core of this embodiment is the ATM-SCSA-Former model, which includes, in sequence: a dual-temporal image input module, a feature encoder, a multi-prior guided adaptive token merging module (ATM-CD), a spatial and channel collaborative attention enhancement module (SCSA-CEFF), and a decoder.

[0030] 2. Specific implementation details of each module Feature Encoder: A pre-trained Vision Transformer is used as the feature encoder. The input is a registered pair of two-temporal remote sensing images, each input to a Transformer encoder with shared weights, resulting in two temporal corresponding token feature sequences, where each token corresponds to a local region of the image.

[0031] Adaptive Token Merging (ATM-CD): Two time-corresponding token feature sequences are input into the ATM-CD module. This module processes the bi-temporal features in parallel, generates four prior maps, and calculates a merging guidance index for each token based on these maps. Subsequently, a constrained merging operation is performed on the token sequences according to the merging guidance index, outputting a compressed token sequence with key information protected.

[0032] Feature Enhancement (SCSA-CEFF): The merged dual-temporal token sequences are concatenated along the channel dimension and then input into the SCSA-CEFF module. This module learns and enhances the difference information between the dual-temporal features through a spatial and channel collaborative attention mechanism, outputting enhanced variation features.

[0033] Decoding and Output: The change features are input into a lightweight decoder to gradually restore the spatial resolution. Finally, a pixel-level change detection probability map is generated through a classification head, and a binary change mask is obtained after thresholding.

[0034] Example 2: Detailed Implementation of the ATM-CD Module This embodiment details the internal structure and working principle of the ATM-CD module. Figure 1 As shown, the ATM-CD module mainly includes a multi-priority generation unit, a token merging guidance indicator construction unit, and a constrained token merging execution unit.

[0035] 2.1 Multiple Prior Generation (1) Semantic prior ( )generate: Semantic consistency features are extracted based on the Multi-Semantic Space Attention (SMSA) module to describe the semantic subspace in which the token resides, generating a semantic prior graph. .

[0036] First, define a set of learnable semantic prototype vectors. Each prototype vector Implicit representation is a typical semantic concept of remote sensing ground features, such as buildings, vegetation, and water bodies.

[0037] Then, for the feature vector of any token Calculate the cosine similarity between it and all semantic prototypes, and obtain the semantic attribution distribution by Softmax normalization. k: ,in, ,k represents the normalized affinity between the i-th token and the k-th semantic prototype, and τ is the temperature parameter.

[0038] Finally, semantic priors Defined as: ,in, This reflects the semantic attribution confidence of the token, indicating the retrieval... The maximum value in the range k = 1 to K. Confidence level. The higher, The smaller the value, the purer the semantics of the token, the more it tends to be in a homogeneous background region, and the more suitable it is to be merged.

[0039] (2) Change prior ( )generate: First, define the bi-temporal token feature sequence from bi-temporal images, which has been precisely aligned according to spatial location, as follows: and .in, This represents the eigenvector at the i-th spatial location in the t-th time phase.

[0040] Next, calculate the Euclidean distance between the token feature vectors at the corresponding locations: , as the initial measure of difference. This reflects the feature dissimilarity between the two time phases in the i-th local region.

[0041] Then, the difference values ​​at all locations within the batch or within the image. Perform min-max normalization: This is to eliminate the influence of feature scale and ensure that the difference values ​​fall within a comparable range. It is a very small positive value to prevent division by zero errors; after normalization It provides an indication of the relative intensity of change.

[0042] Finally, a nonlinear mapping is performed using an adjustable sigmoid function to generate the final prior of change. : This enhances the contrast between changed and unchanged regions and constrains the output in probabilistic form.

[0043] in, This represents the Sigmoid function, which compresses the output to the (0,1) interval. γ represents the temperature parameter, controlling the steepness of the curve. The larger the value, the steeper the function, the stronger its ability to distinguish small differences, and the more acute the response at the boundary of change. μ represents the bias parameter, usually set to 0.5, as a reference point for the decision threshold; when > hour, A value >0.5 indicates that the region is more likely to have changed. (Change prior) The closer it is to 1, the greater the likelihood that the position will change and should be protected.

[0044] (3) Structural boundary priors ( )generate: Using classic image gradient operators for edge detection such as Sobel and Canny, or a lightweight convolutional neural network, extract the time-phase input image t. Edge intensity map .in, , This represents the edge extraction operator, whose output is the pixel-level edge response.

[0045] For the image position corresponding to the i-th token Its structural boundary prior Defined as the fusion value of the edge intensities of the two phases: Among them, the fusion function It can be either the maximum value or the average value.

[0046] Then normalization is performed: .in, , These represent the minimum and maximum values ​​of all prior token structures, respectively. It is a tiny constant used for numerical stability. This indicates the sensitivity of the i-th token at the structural boundary. The larger the value, the more likely the token is to be in the real edge area and should be protected.

[0047] (4) Prior importance of tokens ( )generate: A comprehensive assessment of the importance of the token in the global context.

[0048] On the one hand, using the self-attention matrix A in the encoder, the "degree of dependence" of Token i in global context modeling is calculated: Where N is the total number of tokens. , This represents the attention weight of the i-th token on the j-th token.

[0049] On the other hand, calculate the token feature vector. Channel energy As for its activation strength: .in, Let i be the feature vector representation of Token i.

[0050] Combining the two, we obtain the prior importance of tokens: ,in The fusion function can be expressed as a weighted sum or a multiplicative fusion form, for example: .in, To balance the weights.

[0051] Then normalize the two: .in, This represents a constant term to prevent numerical instability; after normalization This indicates the relative importance of the token. The larger the value, the greater the token's contribution to the overall semantic modeling, and the more it should be protected.

[0052] 2.2 Construction of Guiding Indicators Calculate merged bootstrapping metrics for each Tokeni : .in, , , , These represent semantic priors, change priors, structural boundary priors, and token importance priors, respectively. , , , These are learnable or preset weight parameters, and their initial values ​​can all be set to 1. The larger the value, the less suitable the token is for merging.

[0053] 2.3 Basis for Merger Decision Define a merging threshold According to the guiding indicators With threshold Based on this relationship, all tokens are divided into two categories: ①Key Token: These tokens correspond to areas of change, structural boundaries, or important semantic regions, and are protected in this merge iteration, prohibiting their participation in the merge.

[0054] ② Tokens that can be merged: First, feature similarity is calculated, and a weighted merging operation is performed based on the similarity level. These tokens mainly correspond to semantically consistent homogeneous background regions and are considered redundant tokens, which are allowed to participate in the merging as candidates.

[0055] This constraint achieves the desired change region ( ) tokens are not merged, boundary area tokens ( Large (not merged), high semantic importance region token ( Large (non-merging, redundant background tokens) Automatic compression is performed when the value is small (and other prior values ​​are also small).

[0056] 2.4 Token Merging Algorithm Based on Bipartite Graph Matching For the mergeable tokens selected in step 2.3, an algorithm based on bipartite graph matching is used to perform the actual merging operation, maximizing the compression ratio while protecting critical information. The execution steps are as follows: Figure 2 As shown, the details are as follows: Step 1: Grouping. Randomly and evenly divide the set of all mergeable tokens in the current feature layer into two equal subsets, B1 and B2. Let A be the set of all key tokens. The merging operation will then be performed between B1 and B2.

[0057] Step 2: Similarity Calculation and Edge Construction. Calculate the feature similarity between each token in subset B1 and all tokens in subset B2. For each token in B1, retain only the connection between it and the most similar token in B2, forming a directed edge. This constructs a bipartite graph between B1 and B2, where the edge weights represent the similarity between token pairs.

[0058] Step 3: Edge Filtering. Sort all edges generated in Step 2 according to their similarity weight from highest to lowest. To control the aggressiveness of the merging, only the first r most similar edges (i.e., the r token pairs with the highest similarity) are retained; the rest are discarded. The parameter r determines the maximum number of merges that can occur in this iteration, and it is directly related to the target token compression ratio.

[0059] Step 4: Token Merging. For the r edges retained in Step 3, the two tokens connected by each edge will be merged. The merging operation typically uses a weighted average to generate a new token. After merging, the original token will be removed from the sequence and replaced by the new token.

[0060] Step 5: Sequence Recombination. The new token generated after merging is recombined with the tokens that were not merged, including the tokens in B1 and B2 that did not participate in the merging, as well as the key tokens in group A that have been protected throughout, to form a new and shorter token sequence, which is the output of the ATM-CD module.

[0061] Example 3: Detailed Implementation of the SCSA-CEFF Module This embodiment details the SCSA-CEFF module. For example... Figure 3 As shown, its input is the dual-temporal token feature output by the ATM-CD module and spliced ​​through channels.

[0062] (1) Multi-Semantic Space Attention (SMSA) Submodule: SMSA aims to capture spatial contextual information with different semantic orientations. First, the input features are... Global average pooling is performed along both the height (H) and width (W) dimensions to obtain two one-dimensional feature sequences. and Next, and Each feature is uniformly divided into K groups of independent sub-features:

[0063]

[0064] in, , This represents the i-th sub-feature, and the number of channels for each sub-feature is... For each set of sub-features, use different kernel sizes. Depth-separable one-dimensional convolutions can be processed independently to extract multi-scale spatial structure information:

[0065]

[0066] in, This represents the spatial structure information of the i-th sub-feature obtained after a lightweight convolution operation.

[0067] All processed sub-features are recombined and concatenated, and spatial attention maps in the height and width directions are generated by applying group normalization (GN) and sigmoid activation functions respectively. and :

[0068]

[0069] in, This represents the Sigmoid normalization function. and Let represent GNs with K groups along the H and W dimensions, respectively.

[0070] Finally, the output of SMSA is: .

[0071] (2) Progressive Channel Self-Attention (PCSA) Submodule: PCSA receives the output of SMSA. The aim is to perform efficient modeling of inter-channel relationships.

[0072] First, use 7x7 average pooling. Spatial downsampling is performed to obtain compressed features. This reduces the computational cost of subsequent self-attention.

[0073] Then, three independent 1x1 convolutional layers are used to... The projection is a query (Q), key (K), value (V) matrix. Self-attention computation is performed.

[0074] Finally, for Perform global average pooling and generate channel attention weights using the Sigmoid function, then compare them with the original... Multiplying them together yields the final output of PCSA. That is, the enhanced change characteristics .

[0075] The formula used is as follows:

[0076]

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] in, and Let represent the feature maps of the image before and after the change in the i-th layer, respectively, and Cat denotes tensor concatenation.

[0083] The SCSA-CEFF module achieves synergistic enhancement of spatial localization and channel refinement through the serial collaboration of SMSA and PCSA. Because the preceding ATM-CD module provides token sequences with more consistent semantics, less noise, and more complete structure, SCSA-CEFF can more effectively focus on real variations, thereby improving channel attention expressiveness, suppressing noise, strengthening the saliency of variation features, and ultimately obtaining a structurally clear and semantically reliable variation feature map.

[0084] Example 4: Modified Example 4.1 Deformation of the prior generating unit: (1) Semantic prior generation variation example 1: A lightweight convolutional module can be used to project the Token features and then directly calculate the distance between them and the K learnable cluster centers to generate the semantic prior Si, without having to calculate all similarities.

[0085] (2) Semantic prior generation variation example 2: semantic prototype It can be obtained through unsupervised clustering pre-training on a large number of remote sensing images, or it can be learned end-to-end as model parameters along with the task.

[0086] (3) Variation Example 2: Difference Measurement by Changing Priors It can be obtained using cosine distance, Mahalanobis distance, or by learning through a small neural network.

[0087] (4) Variation of the generation of the changed prior: Example 3: In the generation of the changed prior In this case, the temperature parameter τ can be designed as a learnable parameter, or different fixed values ​​can be used at different depths of the network, such as using a larger τ for shallow layers to enhance contrast and a smaller τ for deeper layers.

[0088] 4.2 Variations of Guiding Indicators and Merging Strategies: (1) Variation of guiding indicators: guiding indicators The construction of the algorithm is not limited to linear weighting; gating mechanisms can be used.

[0089] (2) Combined execution variant: threshold It can be configured to be dynamically adaptive. For example, based on the bootstrapping metrics of the current batch of tokens. The distribution selects tokens ranked in the bottom R% (e.g., 65%) for merging, rather than using a fixed threshold.

[0090] 4.3 Variations of the SCSA-CEFF module: (1) SMSA variant: The number of groups K is not fixed at 4, but can be dynamically adjusted according to the number of input channels N, for example, set to The set of convolution kernel sizes can also be adjusted.

[0091] (2) PCSA variant: The downsampling factor of progressive compression can be adjusted, for example, using convolution with a stride of 2 instead of average pooling for compression.

[0092] 4.4 System Variation Examples: (1) The ATM-CD module can be enabled only when the feature map resolution is low, preserving more detailed information in shallow layers. For example, only when the feature map size is the original image. and When applying token merging.

[0093] (2) For edge device deployment, the self-attention in the SCSA-CEFF module can be replaced with a more efficient linear attention variant to further reduce computational overhead.

[0094] Example 5: Experimental Verification and Results To verify the effectiveness of this invention, we conducted comparative experiments on three publicly available remote sensing change detection CD datasets: LEVIR-CD, DSIFN-CD, and CDD.

[0095] 5.1 Experimental Setup (1) Datasets: ① LEVIR-CD: A building-related CD dataset, with original image resolution of 1024×1024, cropped to 256×256 non-overlapping patches, and training / validation / test set sample sizes of 7120 / 1024 / 2048. ② DSIFN dataset: A general-purpose CD dataset containing various land cover changes, with original images of 512×512, cropped to 256×256 patches, and training / validation / test set sample sizes of 14400 / 1360 / 192. ③ CDD dataset: A general change dataset containing seasonal variations, with images cropped to 256×256 pixels, and training / validation / test set samples of 10000 / 3000 / 3000 each.

[0096] (2) Evaluation metrics: Precision, recall, F1 score, intersection-over-union ratio (IoU) and overall accuracy (OA) are used as evaluation metrics, among which F1 and IoU are the core metrics.

[0097] (3) Comparison method: Compare with classic and cutting-edge change detection methods such as FC-EF, FC-Siam-Di, FC-Siam-Conc, DTCDSCN, STANet, IFNet, SNUNet, BIT, ChangeFormer, and MDS-Net.

[0098] 5.2 Quantitative Results Analysis Tables 1, 2, and 3 present the quantitative results of the method of the present invention on the LEVIR-CD, CDD, and DSIFN-CD datasets, respectively.

[0099] Table 1 Performance comparison on the LEVIR-CD dataset (%)

[0100] Table 2 Performance comparison on the CDD dataset (%)

[0101] Table 3 Performance comparison on the DSIFN-CD dataset (%)

[0102] On three challenging datasets, the ATM-SCSA-Former model of this invention achieved state-of-the-art performance in both the F1 and IoU metrics. Particularly on the LEVIR-CD dataset, the F1 score reached 92.55%, surpassing the previous best model, MDS-Net (92.24%). On the DSIFN-CD dataset, the IoU reached 80.44%, a significant improvement over ChangeFormer (76.48%). This directly verifies that after introducing the ATM-CD module for token merging and lightweighting, the model not only did not lose accuracy but also improved its ability to detect complex and changing regions through the protection mechanism of multiple priors and the synergistic enhancement of SCSA-CEFF.

[0103] In summary, the multi-prior guided adaptive token merging method and system provided by this invention effectively solves the technical problem raised in the background art: how to protect key change information to maintain high detection accuracy while reducing the computational overhead of the Transformer model, and achieves synergistic optimization of accuracy and efficiency.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A deep learning-based adaptive token merging method for remote sensing change detection guided by multiple priors, characterized in that, The method, applied to a remote sensing image semantic change detection model based on the Transformer architecture, includes the following steps: Step S1: Input the registered dual-temporal remote sensing images into the feature encoder to obtain the token feature sequences corresponding to the two times; Step S2: Input the token feature sequence into the multi-prior-guided adaptive token merging module, i.e., ATM-CD. The ATM-CD module performs the following operations: S2.1 For each Token in the Token feature sequence, generate semantic priors, change priors, structural boundary priors, and Token importance priors in parallel; S2.2 Based on the four priors, calculate a merged bootstrapping metric for each Token; S2.3 Perform a constrained merging operation on the Token sequence according to the merging guidance index, wherein merging is only allowed for the Token pair when the merging guidance index of the candidate Token pair is less than a preset threshold, and the compressed Token feature sequence is output. Step S3: Input the compressed dual-temporal token feature sequence into the spatial and channel collaborative attention enhancement module, i.e., SCSA-CEFF, to enhance feature differences and obtain a change feature map; Step S4: Decode the change feature map to generate the final change detection result.

2. The deep learning adaptive token merging method for remote sensing change detection with multiple prior guidance as described in claim 1, characterized in that, In step S2.1, the process of generating semantic priors includes: Define a set of learnable semantic prototype vectors; for each token's feature vector, calculate its similarity to each semantic prototype vector and normalize it to obtain a semantic attribution distribution; the semantic prior value is calculated based on the maximum confidence in the semantic attribution distribution, and the smaller the semantic prior value, the purer the semantics of the token.

3. The deep learning adaptive token merging method for remote sensing change detection with multiple prior guidance as described in claim 1, characterized in that, In step S2.1, the process of generating the change prior includes: for a spatially aligned bi-temporal token feature sequence, calculating the distance metric of its feature vector; normalizing the calculated distance metric; inputting the normalized distance metric into an adjustable Sigmoid function, the output of which is used as the change prior, the larger the change prior value, the greater the probability that the position will change.

4. The deep learning adaptive token merging method for remote sensing change detection guided by multiple priors as described in claim 1, characterized in that, In step S2.1, the process of generating the structural boundary prior includes: extracting edge intensity maps from the dual-temporal remote sensing images respectively; obtaining edge response values ​​from the edge intensity maps according to the image location corresponding to the Token; fusing the edge response values ​​at the corresponding locations in the dual-temporal images and normalizing them to obtain the structural boundary prior. The larger the structural boundary prior value, the more likely the Token is to be located at the boundary of the ground feature.

5. The deep learning adaptive token merging method for remote sensing change detection with multiple prior guidance as described in claim 1, characterized in that, In step S2.1, the process of generating the Token importance prior includes: calculating the degree of dependence of each Token based on the self-attention matrix in the feature encoder; calculating the channel energy of the feature vector of each Token as its activation intensity; fusing the degree of dependence and the activation intensity and normalizing them to obtain the Token importance prior. The larger the importance prior value, the more important the Token is in the feature representation.

6. The method according to claim 1, characterized in that, In step S2.2, the merging guidance indicators The calculation formula is: in, , , , These represent semantic priors, change priors, structural boundary priors, and token importance priors, respectively. , , , These are the learnable weight coefficients.

7. The deep learning adaptive token merging method for remote sensing change detection with multiple prior guidance as described in claim 1, characterized in that, In step S3, the SCSA-CEFF module includes a serially connected multi-semantic space attention submodule SMSA and an asymptotic channel self-attention submodule PCSA. The SMSA submodule is used to: decompose the input features into multiple sets of sub-features along the spatial height and width dimensions, extract multi-semantic spatial context information through convolution at different scales, and generate a spatial attention map to modulate the input features; The PCSA submodule is used to: spatially downsample the features modulated by SMSA, perform self-attention calculation to model the inter-channel relationship, generate a channel attention map, and perform channel recalibration on the features modulated by SMSA.

8. A remote sensing image semantic change detection system, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the multi-prior-guided remote sensing change detection deep learning adaptive token merging method as described in any one of claims 1-7.

9. The system according to claim 8, characterized in that, The ATM-CD module and the SCSA-CEFF module work together. The compressed token feature sequence output by the ATM-CD module has higher semantic consistency and structural integrity, and serves as the input to the SCSA-CEFF module.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-prior-guided remote sensing change detection deep learning adaptive token merging method as described in any one of claims 1-7.

Images