Anti-noise multi-mode remote sensing image matching method based on edge perception attention fusion

Abstract

The invention belongs to the technical field of remote sensing geoscience application, and particularly relates to an anti-noise multi-mode remote sensing image matching method based on edge perception attention fusion. Comprising the following steps of data and sample set construction, feature preprocessing module design, trunk sharing and multi-scale feature extraction, Transform interactive modeling, rough matching generation and consistency screening, feature fusion module design, fine matching refinement and sub-pixel positioning, and model training and evaluation index system construction. According to the method, the effective cooperation of the structure prior and the semantic context is realized, and the matching stability and precision under the conditions of complex geometry and strong noise are improved; meanwhile, the calculation and labeling dependence is reduced through lightweight fusion and a local refinement strategy, a multi-source remote sensing image intelligent registration technology with practical value is obtained, the professional technical threshold is remarkably reduced, the intelligence and universality of remote sensing image interpretation are promoted, and the method is suitable for large-scale popularization and application. And a convenient and efficient basic capability is provided for applications such as change detection, fusion mapping and three-dimensional reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing geoscience application technology, specifically relating to a noise-resistant multimodal remote sensing image matching method based on edge-aware attention fusion. Background Technology

[0002] Multi-source remote sensing image matching is a crucial preprocessing step that integrates remote sensing data from different types of sensors (such as visible light, infrared, synthetic aperture radar (SAR), LiDAR, etc.), different time phases, and different resolutions. Its core objective is to achieve complementarity and enhancement of multi-source information through spatial alignment, thereby providing a reliable data foundation for subsequent tasks such as image fusion, change detection, target recognition, and environmental analysis. This technology has significant application value and an irreplaceable role in fields such as high-precision map production, land resource monitoring, disaster assessment, military reconnaissance, and urban planning.

[0003] The development of cross-modal remote sensing image matching technology has generally evolved from handcrafted feature methods to learning-based methods. Early research mainly relied on manually designed feature operators for feature point detection and matching, such as classic algorithms like SIFT, SAR-SIFT, and RIFT. These methods typically extract stable features from the local structure of the image, such as corner points, edges, and texture information, and then use similarity measures such as normalized cross-correlation (NCC) or mutual information (MI) to determine matching pairs. Under low noise and consistent imaging conditions, handcrafted feature methods have good interpretability and spatial geometric stability. However, when there are significant spectral differences, scale variations, or geometric distortions between images, the robustness of these methods is significantly insufficient, and they are extremely sensitive to changes in imaging conditions (such as differences in illumination, sensor type, or observation angle). Under conditions of severe noise interference or complex terrain, feature repeatability and matchability drop sharply, making it difficult to guarantee matching stability and accuracy.

[0004] With the rapid development of deep learning and computer vision technologies, learning-based cross-modal matching methods have gradually become a mainstream research direction. These methods achieve complex nonlinear mapping modeling through end-to-end feature learning mechanisms, automatically extracting robust semantic features from a large number of samples and improving feature consistency between images of different modalities. Typically, models based on convolutional neural networks (CNNs) can automatically learn local structural features in the spatial domain, while models based on the Transformer architecture utilize self-attention mechanisms to capture long-range dependencies and global contextual information, thereby improving the alignment performance of cross-domain features. For example, SwinMatcher uses a hierarchical Swin Transformer structure to achieve multi-scale feature fusion and cross-modal information interaction, significantly enhancing global contextual modeling capabilities; methods such as MAP-Net and GMN introduce graph neural networks (GNNs) and multi-head attention mechanisms, constructing structural association graphs to explicitly model the topological relationships between features, thereby improving the discriminative power and matching accuracy of semantic association features. Furthermore, some studies have incorporated frequency domain information into spatial matching frameworks to compensate for the sensitivity of traditional methods to radiometric differences. Through Fourier transform and phase correlation mechanisms, frequency domain features possess rotation, scale, and translation invariance, effectively mitigating the impact of illumination variations and radiation mismatch. For example, the MSG algorithm based on multi-scale Gaussian feature optimization achieves high-precision registration of optical-SAR images by co-optimizing edge structure features in the frequency and spatial domains, providing a new approach for cross-modal image matching.

[0005] Although methodologies have gradually transitioned from rule-based feature description to model-based data-driven learning, cross-modal remote sensing image matching still faces multiple challenges and technical bottlenecks. First, handcrafted feature methods are extremely sensitive to nonlinear radiometric differences and variations in multi-source imaging conditions, limiting their matching performance and making it difficult to maintain stability in complex imaging environments. Second, multimodal remote sensing images commonly exhibit coupling effects between noise and geometric distortion, such as speckle noise in SAR images, thermal radiation noise in infrared images, and radiometric variations in optical images. This noise masks the effective structural information of the images, weakening the recognizability of edge and texture features. Simultaneously, differences in viewpoint, scale, and perspective caused by different imaging platforms further amplify the inconsistencies in the feature space, making accurate cross-modal matching even more difficult. Existing algorithms still struggle to balance matching accuracy and robustness, especially in scenarios with extreme noise, complex terrain, and drastic changes in lighting conditions, where algorithm performance is prone to degradation.

[0006] Therefore, there is an urgent need for a noise-resistant cross-modal matching method that can integrate multi-scale features, has edge perception capabilities and attention mechanisms, so as to enhance the robustness of the model to noise and geometric distortion while ensuring feature consistency, and achieve accurate alignment and structural perception of multi-source remote sensing images. Summary of the Invention

[0007] The purpose of this invention is to address the problems of noise interference sensitivity, insufficient structural feature representation, difficulty in cross-modal feature alignment, and insufficient model robustness and generalization ability in existing cross-modal image matching. It provides a noise-resistant multimodal remote sensing image matching method based on edge-aware attention fusion. This method enhances the model's ability to recognize stable structures (such as building outlines and road textures) and effectively suppresses noise interference by fusing edge-aware feature enhancement and multi-scale attention mechanisms, comprehensively utilizing multi-level edge information and global contextual features. In the feature fusion stage, self-attention and cross-attention strategies are combined to achieve efficient interaction and semantic alignment of features from different modalities; and a learnable matching module and heatmap regression mechanism are combined to achieve sub-pixel-level accurate matching. This method significantly improves matching accuracy and robustness under conditions of high noise, large modal differences, and limited samples, providing a new approach for high-precision fusion and intelligent interpretation of multi-source remote sensing data.

[0008] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution:

[0009] This invention provides a noise-resistant multimodal remote sensing image matching method based on edge-aware attention fusion, comprising the following steps:

[0010] Step 1: Establish a training sample set;

[0011] Step 2: Feature preprocessing module design;

[0012] Step 3: Feature fusion module design;

[0013] Step 4: Design of a noise-resistant multimodal remote sensing image matching model based on edge-aware attention fusion;

[0014] Step 5: Training a noise-resistant multimodal remote sensing image matching model based on edge-aware attention fusion;

[0015] Step 6: Model performance evaluation and index analysis.

[0016] Further, step one includes the following data construction and annotation generation process: acquiring multi-source, multi-modal remote sensing image data, including but not limited to optical, SAR, and infrared images, using high spatial resolution satellite data; performing radiometric correction, geometric correction, and coordinate reference unification on each modality image sequentially to ensure spatial reference consistency, and cropping them into 1024×1024 pixel paired image patches according to a fixed scale to ensure accurate geographical correspondence between optical and non-optical images; pairing samples covering typical land cover types such as towns, roads, water bodies, cultivated land, forest land, and bare land to enhance sample diversity and representativeness; using an automatic registration algorithm to complete coarse registration, combined with manual fine-tuning to optimize the registration results, keeping the alignment error of image pairs within 1 pixel; performing data augmentation on the registered multi-modal samples, including rotation, flipping, scaling, translation, brightness and contrast perturbation, and noise injection to simulate various observation conditions and noise environments while maintaining consistent image geometry; and using PyT... The script records augmentation parameters and geometric relationships for subsequent generation of matching supervision information and consistency verification. The processed multimodal samples are randomly divided into training, validation, and test sets in a 7:2:1 ratio, and corresponding index text files are generated. All samples are normalized to ensure consistency in numerical scale across different modal inputs. A structured description file for training is generated for each sample pair based on a Python script: the data is read and indexed, image pair numbers and metadata are obtained, and geometric priors (such as registration pose / homography, keypoint pseudo-labels, or raster-level correspondences), bounding boxes, and region statistics are extracted or exported for matching supervision. Necessary noise annotations and quality markers are calculated. This information is saved as a JSON instance file and a binary index file (P file or equivalent format), containing sample numbers, modality types, dataset partition types, augmentation parameters, pairing relationships, and geometric constraint fields, serving as the input data basis for model training and evaluation.

[0017] Furthermore, in step two, based on the features of the remote sensing image, the input remote sensing image is processed through two key components: cross-scale interaction and edge feature enhancement. First, a linear projection layer is used to reduce the feature dimensionality:

[0018] ;

[0019] Where, are the parameters of the linear projection layer. After passing through the ReLU activation function, the features are further reshaped into... Spatial information processing; It is a reshaping operation; The reduction factor is used; then, four average pooling (AP) layers are used to obtain the multi-scale feature XS. And it uses deep convolutional layers to capture local details:

[0020] ;

[0021] in, A convolutional layer is defined, which contains a 1×1 kernel, followed by a... function; It is a deep convolutional layer with one 3×3 kernel and one function; It involves bilinear interpolation of upsampled features to a specific resolution; subsequently, multi-scale features are... , and The following is a fusion:

[0022] ;

[0023] in, and It is channel-wise cascading;

[0024] Finally, Reshape into labeled features; obtain the final output through a linear projection layer and a residual connection:

[0025] ;

[0026] in, It is a linear projection layer that restores the feature dimension. To address the challenges of insufficient feature generalization ability and the coupling interference of noise and geometric distortion in multi-source remote sensing images, a multi-scale edge enhancement module was designed as the core component of the feature preprocessing module, as follows:

[0027] ;

[0028] ;

[0029] in, This represents average pooling with 3×3 cores; This represents a 1×1 convolutional layer with batch normalization and a sigmoid function; The scale is Features; then, an edge enhancer is introduced. Enhance the details at each scale:

[0030] ;

[0031] in, Features for edge enhancement; the structure of the edge enhancer is represented as:

[0032] ;

[0033] ;

[0034] in, Then, these features are integrated:

[0035] ;

[0036] in, This is the feature output of the multi-scale edge enhancement module; thus, It possesses both fine-grained details and multi-scale edge information; these features can complement the overall feature set. The missing information is then concatenated, and two 3×3 convolutional layers and one 1×1 convolutional layer are applied to obtain the final result. :

[0037] ;

[0038] .

[0039] Furthermore, in step three, the feature fusion module is completed collaboratively by the grouped feature focusing unit and the multi-scale feature reconstruction unit: First, the features of adjacent stages are aligned in size and channel; the low-resolution features are upsampled and matched with the high-resolution features element-wise after 1×1 convolution to obtain coarse fused features; then, spatial attention masks are generated by single-channel spatial compression and Sigmoid, and position-sensitive weighting is applied to the coarse fused features to obtain context-aware representations; further, the weighted features are grouped along the channel dimension, and within each group, local patterns are first refined by convolution, then weighted again by cross-channel attention masks, and finally, the features are concatenated by channel to form aggregated features with stronger channel correlation, and normalization is applied by referencing the statistics (mean and standard deviation) of adjacent layers to stabilize the values ​​and inject the positional information of small targets, thereby obtaining spatial details and correlations. The system employs a comprehensive fusion representation. Based on this, a multi-scale feature reconstruction unit generates channel-level weight thresholds using global average pooling of fused base features. Batch normalization and sigmoid are applied to the original features of adjacent layers to obtain spatial importance maps. Strong and weak responses are separated by comparing positional importance with channel thresholds. Strong responses are aggregated to preserve structure and edge details, while weak responses are aggregated to supplement background and global context. Finally, features are reconstructed through multi-scale convolution and element-wise fusion, reducing interference from irrelevant and noisy components. Through the cascaded strategy of "spatial attention—group channel interaction—statistical normalization—strong-weak separation—multi-scale reconstruction," effective alignment of semantics between adjacent layers and enhanced cross-channel correlation are achieved, improving the discriminativeness and robustness of multimodal matching features. Furthermore, the optimal fusion weights can be adaptively learned during end-to-end training.

[0040] Further, in step four, the overall model structure includes a feature extraction backbone network, a feature preprocessing module, a Transformer encoder, a coarse matching module, a feature fusion module, and a fine matching refinement module. The feature extraction backbone network is a multi-scale convolutional architecture with shared weights, focusing on extracting alignable semantic and texture representations at the 1 / 8 and 1 / 2 scales to provide basic features for cross-view modeling. The feature preprocessing module focuses on multi-scale edge enhancement, combining edge-guided attention and average pooling to enhance the structural signals of building outlines, road networks, and water system boundaries at multiple resolutions, and suppressing ambiguity caused by noise and radiation differences, outputting structure-sensitive representations. The Transformer encoder consists of stacked self-attention and cross-attention, combined with two-dimensional positional encoding to achieve intra-image long-range context aggregation and cross-image semantic alignment, thereby capturing long-distance dependencies and cross-view correspondences. The coarse matching module constructs a cross-view similarity matrix based on low-resolution features and uses bidirectional consistency... The inspection and confidence screening generate broad-coverage coarse matching anchor points, providing prior knowledge for subsequent local searches. The feature fusion module consists of a grouped feature focusing unit and a multi-scale feature reconstruction unit. The grouped feature focusing unit enhances inter-layer and inter-channel correlation and suppresses redundancy and noise through scale and channel alignment, spatial attention, and grouped channel interaction. The multi-scale feature reconstruction unit performs strong and weak information separation and lightweight reconstruction on the fused representation, prioritizing the preservation of small targets and edge texture details to improve cross-modal robustness and spatial reconstruction quality. The fine matching refinement module adopts a progressive local search strategy, constructing a local cost volume or window correlation around the coarse matching position on high-resolution features. It combines interpolation sampling with learnable regression or soft-argmax to achieve sub-pixel-level localization and introduces confidence estimation and robust filtering to output high-precision correspondences. Each stage maintains information flow and scale consistency through normalization, position encoding, and skip connections, ensuring that the overall process balances efficiency, stability, and sub-pixel matching accuracy in complex geometric and radiation difference scenarios.

[0041] Further, in step five, 80% of the data is randomly extracted from the standardized multimodal sample set constructed in step one for training, and the remaining 20% ​​is used for validation. The samples maintain a spatial scale of 1024×1024, are normalized, and have a unified coordinate reference. During training, a randomly sampled controllable homography transformation is applied to the source image in any input image pair to generate a reference image, and the pixel-level true correspondence is derived online from the corresponding homography matrix, thereby eliminating manual annotation errors. At the same time, the geometric and photometric enhancement strategies consistent with those in step one are adopted to improve the model's generalization and noise resistance capabilities. Enhancements include random rotation, horizontal / vertical flipping, scaling, translation, homography parameter perturbation, brightness and contrast perturbation, motion blur, Gaussian noise, and speckle noise injection for SAR data, while ensuring effective geometric alignment. The process proceeds as follows: The enhanced multimodal image pairs are input into the network architecture designed in step four. The network sequentially passes through the feature extraction backbone network, the feature preprocessing module, and the Transformer encoder to obtain cross-view contextual representations. A broad-coverage candidate correspondence is generated in the coarse matching module, grouped feature focusing and multi-scale feature reconstruction are achieved in the feature fusion module, and sub-pixel-level localization is achieved in the fine matching refinement module through local cost volume / window correlation and learnable regression or soft-argmax, outputting the matching results. During training, the model performance is evaluated using a validation set after each epoch, and the model weights with the best validation performance are saved. A learning rate decay strategy is also introduced; when the validation set performance does not improve over several consecutive epochs, the learning rate is automatically reduced to promote convergence and improve generalization performance.

[0042] Furthermore, in step six, to verify the effectiveness and robustness of the model, a comprehensive evaluation and index analysis of the model performance are conducted. System experiments are carried out on the dataset constructed in step one, with the main quantitative indicators including: root mean square error (RMSE, in pixels or geographic units) and number of correct matches (NCM). RMSE measures sub-pixel positioning accuracy, while NCM assesses the trade-off between coverage and reliability. At the same time, computational efficiency and resource consumption are analyzed, including inference speed (runtime RT), number of parameters, and GPU memory usage. Comparative tests are conducted under different input resolutions and different modal combinations (optical-optical, optical-SAR, optical-infrared, SAR-infrared) to verify the adaptability and scalability in cross-modal, weak texture, and strong noise scenes.

[0043] Furthermore, in step six, for qualitative evaluation, the response map of multi-scale edge enhancement, the cost volume / correlation heatmap of Transformer interactive attention and coarse-fine matching are visualized to analyze the effect of structural prior on the suppression of noise and radiation inconsistency, and the fidelity improvement of small targets and edge details by grouped feature focusing and multi-scale feature reconstruction. In addition, comparative experiments are conducted with existing representative methods to comprehensively evaluate the accuracy, stability and efficiency.

[0044] The beneficial effects of this invention are:

[0045] This invention discloses a noise-resistant multimodal remote sensing image matching method based on edge-aware attention fusion. Addressing challenges such as large radiometric differences, strong noise interference, and complex geometric distortions among multi-source images, it constructs an integrated framework from preprocessing and feature fusion to coarse-to-fine matching. By introducing a multi-scale edge enhancement module in the feature preprocessing stage, it utilizes structural priors to explicitly suppress noise and radiometric inconsistencies, significantly improving the structural sensitivity of the initial representation. In the fusion stage, it employs grouped feature focusing and multi-scale feature reconstruction strategies to strengthen inter-layer and inter-channel correlations, separate and refine strong and weak features, and effectively preserve small targets and edge details. In the matching stage, it combines Transformer interaction and coarse-to-fine hierarchical optimization to achieve sub-pixel-level fine localization. Compared with traditional methods, this method exhibits higher matching accuracy and stability in cross-modal, weak-texture, and strong-noise scenarios, while balancing computational efficiency and resource controllability, reducing reliance on large-scale homogeneous labeled data and complex manual parameter tuning, and possesses good engineering usability and promotional value.

[0046] Of course, any product implementing this invention does not necessarily need to achieve all of the above advantages at the same time. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of an embodiment of the present invention;

[0049] Figure 2 This is a model framework diagram of an embodiment of the present invention;

[0050] Figure 3 This is a diagram illustrating the results of an embodiment of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] A noise-resistant multimodal remote sensing image matching method based on edge-aware attention fusion includes the following steps:

[0053] Step 1: Establish the training sample set

[0054] 1) Select multi-source, multi-modal remote sensing image data, including optical, SAR, and infrared images, prioritizing satellite data with high spatial resolution. Perform radiometric correction, geometric correction, and coordinate system unification on the images to ensure spatial reference consistency across different modalities. Subsequently, crop the images to 1024×1024 pixel paired image patches at a fixed scale to ensure accurate geographical correspondence between optical and non-optical images. Each image pair should cover typical land cover types, such as towns, roads, water bodies, farmland, forest land, and bare land, to enrich the diversity and representativeness of the sample's land cover.

[0055] 2) An automatic registration algorithm is used for coarse registration, and the matching results are further optimized by manual fine-tuning to ensure that the alignment error of the image pairs is controlled within 1 pixel. To enhance the robustness and generalization ability of the model, various data augmentation operations are performed on the registered multimodal image samples, including rotation, flipping, scaling, translation, brightness perturbation, and noise injection, to simulate different observation conditions and noise environments while maintaining the consistency of the image geometry.

[0056] 3) The processed multimodal samples are randomly divided into training, validation, and test sets in a 7:2:1 ratio, and corresponding index text files are generated. All samples are normalized to maintain the consistency of numerical scales across different modal input features. This results in a well-structured and standardized multimodal remote sensing image matching sample set, providing a high-quality data foundation for the training and evaluation of the network model.

[0057] Step 2: Feature Preprocessing Module Design

[0058] Based on the features of remote sensing images, the input remote sensing images are processed through two key components: cross-scale interaction and edge feature enhancement. First, a linear projection layer is used to reduce the feature dimensionality:

[0059]

[0060] Where, are the parameters of the linear projection layer. After passing through the ReLU activation function, the features are further reshaped into... Spatial information processing. It is a reshaping operation. This is the reduction factor.

[0061] Then, four average pooling (AP) layers are used to obtain the multi-scale feature XS. And it uses deep convolutional layers to capture local details:

[0062]

[0063] in A convolutional layer is defined, which contains a 1×1 kernel, followed by a... function. It is a deep convolutional layer with one 3×3 kernel and one function. It is a bilinear interpolation of upsampled features to a specific resolution.

[0064] Next, multi-scale features , and The following is a fusion:

[0065]

[0066] in and It is a channel-wise cascade.

[0067] Finally, The features are reshaped into tokenized data. The final output is obtained through a linear projection layer and a residual connection.

[0068]

[0069] in It is a linear projection layer that restores the feature dimension.

[0070] To address the challenges of insufficient feature generalization ability and the coupling interference of noise and geometric distortion in multi-source remote sensing imagery, a multi-scale edge enhancement module was designed as the core component of the feature preprocessing module. The details are as follows:

[0071]

[0072]

[0073] in, This represents the average pooling of 3×3 cores. This indicates that it has batch normalization and a sigmoid function. Convolutional layer. The scale is The characteristics of [the feature]. Then, an edge enhancer is introduced. Enhance the details at each scale:

[0074]

[0075] in Features for edge enhancement. The structure of an edge enhancer can be represented as:

[0076]

[0077]

[0078] in Then, these features are merged:

[0079]

[0080] in This is the feature output of the multi-scale edge enhancement module. It possesses both fine-grained details and multi-scale edge information. These features can complement existing features. The missing information is then concatenated, and two 3×3 convolutional layers and one 1×1 convolutional layer are applied to obtain the final result. :

[0081]

[0082]

[0083] Step 3: Feature Fusion Module Design

[0084] This module consists of two main parts: a grouped feature focusing unit and a multi-scale feature reconstruction unit.

[0085] The grouped feature focusing unit can effectively fuse adjacent layers with different abstract semantic information in the backbone network and extract relevant feature information from different channels. This unit enhances the correlation between features and improves the efficiency of information expression. , This represents single-stage features at different stages. First, low-resolution features Pi are upsampled using linear interpolation, then a 1×1 convolution is performed to ensure the number of channels in the feature map remains consistent. This processed feature is then compared with high-resolution features. Add element-wise to obtain coarse features. To refine the feature and capture context-aware information, a 1×1 convolution is used to compress the feature into a single channel to aggregate spatial information, and then sigmoid activation is used to generate spatial aggregation weights. Features containing spatial information It can be calculated as:

[0086]

[0087]

[0088] in For element-wise multiplication, Sum the elements intelligently. Sigmoid represents an activation function. Conv represents a 1×1 convolutional layer.

[0089] To enhance the correlation between adjacent features, features are spatially aggregated along the channel dimension. The data is divided into n groups, and feature interactions are performed on each group. Specifically, convolutional modules are used to process the feature information of adjacent channels within each group. Further refinement. For each group... The global features of different channels are transformed to generate attention masks that capture the inter-channel relationships. Then use this mask. This is applied to the refined features. Finally, each group of features is connected to form aggregated and highly correlated adjacent features. The entire calculation process is as follows:

[0090]

[0091]

[0092] in and These represent the fusion interaction layer and the convolutional transformation layer, respectively. ∪ stands for concatenation. This represents each highly correlated feature group. Softmax is used as the activation function to generate the attention mask.

[0093] Finally, the grouped features are aggregated. The feature is embedded into a normalization layer that fuses multiple layers of original features. Feature Pg is normalized using the mean and standard deviation of the features, thereby incorporating more spatial location information from a smaller target. This results in features with strong correlation and rich spatial information. , can be represented as:

[0094]

[0095] in and These represent the mean and standard deviation, respectively.

[0096] By employing this method, the semantic information of adjacent layers is fully utilized, and relevant features are extracted across different channels, thereby enhancing the overall feature representation.

[0097] Multi-scale feature reconstruction units can reduce the fusion and extraction of irrelevant features in deep networks, minimizing the loss of target information.

[0098] First, from the perspective of single-level characteristics at different stages and In this process, features are obtained through operations such as upsampling, convolution, and element-wise addition. Next, average pooling and the sigmoid function are used to generate information weights for each channel, which serve as feature weight thresholds. . It can be represented as:

[0099]

[0100] in This indicates average pooling.

[0101] Secondly, single-level features and Each step is processed using batch normalization (BN) and activated by the sigmoid function, generating unique weight information at each spatial location. and This indicates the importance of different feature maps.

[0102]

[0103]

[0104] Where BN represents batch normalization.

[0105] Next, the weight information ωi and ωi−1 at different stages are compared with the feature weight threshold Tc to obtain an attention map that captures the intensity of spatial information. Subsequently, strong and weak features from different layers are aggregated to obtain enriched features and weak features.

[0106]

[0107]

[0108] Step 4: Design of a noise-resistant multimodal remote sensing image matching model based on edge-aware attention fusion

[0109] The overall model structure includes a feature extraction backbone network, a feature preprocessing module, a Transformer encoder, a coarse matching module, a feature fusion module, and a fine matching refinement module. The feature extraction backbone network employs a multi-scale convolutional architecture with shared weights, focusing on extracting robust and alignable semantic and textural representations at 1 / 8 and 1 / 2 scales, providing foundational features for subsequent cross-view modeling. The feature preprocessing module uses multi-scale edge enhancement as its core computational unit. Through edge-guided attention and average pooling amplification mechanisms, it explicitly enhances structural signals such as building outlines, road networks, and water system boundaries at multiple resolutions, effectively suppressing ambiguity caused by noise and radiation differences, and outputting structure-sensitive input representations. The Transformer encoder consists of stacked self-attention and cross-attention mechanisms, combined with 2D positional encoding to achieve intra-image long-range context aggregation and cross-image semantic alignment, capturing long-distance dependencies and cross-view correspondences. The coarse matching module constructs a cross-view similarity matrix on low-resolution features, and, in conjunction with bidirectional consistency checks and confidence filtering, generates broad-coverage coarse matching anchor points, providing reliable priors for subsequent local searches. The feature fusion module is composed of a grouped feature focusing unit and a multi-scale feature reconstruction unit. The former enhances inter-layer / inter-channel correlation and suppresses redundancy and noise through scale and channel alignment, spatial attention, and grouped channel interaction. The latter performs strong and weak information separation and lightweight reconstruction on the fused representation, prioritizing the preservation of details such as small targets and edge textures to improve cross-modal robustness and spatial reconstruction quality. The fine matching refinement module adopts a progressive local search design, constructing a local cost volume or window correlation around the coarse matching position on high-resolution features, and combining interpolation sampling and learnable regression (or soft-argmax) to achieve sub-pixel-level localization. At the same time, confidence estimation and robust filtering are introduced to output high-precision correspondence. Each stage maintains information flow and scale consistency through normalization, position encoding, and skip connections, enabling the overall process to balance efficiency, stability, and sub-pixel matching accuracy in scenarios with complex geometry and radiation differences.

[0110] Step 5: Training a noise-resistant multimodal remote sensing image matching model based on edge-aware attention fusion

[0111] Based on the standardized multimodal sample set constructed in step one, the data is divided into a training set (80%) and a validation set (20%) according to fixed rules, maintaining a 1024×1024 scale, normalization, and a unified coordinate reference. During the training phase, a controllable homography transformation with random sampling is applied to any input image to generate a reference image, and the true correspondence is dynamically derived from the homography matrix to avoid manual annotation errors. At the same time, geometric and photometric enhancements consistent with step one are applied (random rotation, flipping, scaling, translation, and homography parameter sampling, brightness / contrast perturbation, motion blur, Gaussian noise, and speckle noise for SAR) to improve robustness while ensuring effective geometric alignment. The network designed in step four is used for end-to-end training. During the training process, the model performance is evaluated using the validation set after each epoch, and the weights of the best-performing model are saved. A learning rate decay strategy is introduced, and the learning rate is reduced when the validation set performance does not improve for several consecutive epochs.

[0112] Step Six: Model Performance Evaluation and Index Analysis

[0113] To address the challenges of large radiometric differences, strong noise interference, and complex geometric distortions among multi-source images, an integrated framework from preprocessing and feature fusion to coarse-to-fine matching is constructed. By introducing a multi-scale edge enhancement module in the feature preprocessing stage, structural priors are used to explicitly suppress noise and radiometric inconsistencies, significantly improving the structural sensitivity of the initial representation. In the fusion stage, grouped feature focusing and multi-scale feature reconstruction strategies are employed to strengthen inter-layer and inter-channel correlations, separate and refine strong and weak features, and effectively preserve small targets and edge details. In the matching stage, Transformer interaction and coarse-to-fine hierarchical optimization are combined to achieve sub-pixel-level fine localization. Compared with traditional methods, this approach exhibits higher matching accuracy and stability in cross-modal, weak-texture, and strong-noise scenarios, while balancing computational efficiency and resource controllability, reducing reliance on large-scale homogeneous labeled data and complex manual parameter tuning, demonstrating good engineering usability and promotional value.

[0114] To verify the effectiveness of the noise-resistant multimodal remote sensing image matching model based on edge-aware attention fusion, a systematic evaluation of the model's performance was conducted. Four datasets were used: optical-depth, infrared-optical, optical-electronic map, and SAR-optical paired data. Each dataset contained 10 paired images, each with a resolution of 512×512. Depth data was sourced from an airborne LiDAR, infrared data from Landsat TM-5 or an airborne infrared sensor, electronic maps from Google Maps, and SAR data from the GF-3 satellite. These datasets exhibit significant differences in radiometric characteristics and geometric deformation, including nonlinear radiometric distortion, contrast variations, and geometric transformations such as scale, rotation, and displacement, effectively testing the robustness and adaptability of the multimodal matching method. Evaluation metrics included the number of correct matches (NCM), root mean square error (RMSE), and runtime (RT), used to measure matching accuracy, positioning precision, and computational efficiency, respectively. The combined experimental results were used to quantitatively and qualitatively verify the model's effectiveness and robustness under complex modal differences and noisy environments.

[0115] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to specific implementations. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A noise-resistant multimodal remote sensing image matching method based on edge-aware attention fusion, characterized in that, Includes the following steps: Step 1: Establish a training sample set; Step 2: Feature preprocessing module design; Step 3: Feature fusion module design; Step 4: Design of a noise-resistant multimodal remote sensing image matching model based on edge-aware attention fusion; Step 5: Training a noise-resistant multimodal remote sensing image matching model based on edge-aware attention fusion; Step 6: Model performance evaluation and index analysis.

2. The method according to claim 1, characterized in that, Step one includes the following data construction and annotation generation process: acquiring multi-source, multi-modal remote sensing image data, including but not limited to optical, SAR, and infrared images, using high spatial resolution satellite data; performing radiometric correction, geometric correction, and coordinate reference unification on each modality image in sequence to ensure spatial reference consistency, and cropping them into 1024×1024 pixel paired image patches according to a fixed scale to ensure accurate geographical correspondence between optical and non-optical images; pairing samples covering typical land cover types such as towns, roads, water bodies, cultivated land, forest land, and bare land to enhance the diversity and representativeness of the samples; using an automatic registration algorithm to complete coarse registration, combined with manual fine-tuning to optimize the registration results, so that the alignment error of the image pairs is controlled within 1 pixel; performing data augmentation on the registered multi-modal samples, including rotation, flipping, scaling, translation, brightness and contrast perturbation, and noise injection, to simulate various observation conditions and noisy environments while maintaining consistent image geometry; A Python script is used to record the enhancement parameters and geometric relationships for subsequent generation of matching supervision information and consistency verification. The processed multimodal samples are randomly divided into training, validation, and test sets in a 7:2:1 ratio, and corresponding index text files are generated. All samples are normalized to ensure the consistency of different modal inputs on the numerical scale. Based on the Python script, a structured description file required for training is generated for each sample pair: reading data to divide the index, obtaining image pair numbers and metadata, extracting or exporting geometric priors, bounding boxes, and region statistics for matching supervision, and calculating necessary noise annotations and quality labels. The above information is saved as a JSON instance file and a binary index file, which contain fields such as sample number, modality type, dataset partition type, augmentation parameters, pairing relationship and geometric constraint, as the input data basis for model training and evaluation.

3. The method according to claim 1, characterized in that, In step two, based on the features of the remote sensing image, the input remote sensing image is processed through two key components: cross-scale interaction and edge feature enhancement. First, a linear projection layer is used to reduce the feature dimensionality. ； Where, are the parameters of the linear projection layer. After passing through the ReLU activation function, the features are further reshaped into... Spatial information processing; It is a reshaping operation; The reduction factor is used; then, four average pooling layers are used to obtain the multi-scale feature XS. And it uses deep convolutional layers to capture local details: ； in, A convolutional layer is defined, which contains a 1×1 kernel, followed by a... function; It is a deep convolutional layer with one 3×3 kernel and one function; It involves bilinear interpolation of upsampled features to a specific resolution; subsequently, multi-scale features are... , and The following is a fusion: ； in, and It is channel-wise cascading; Finally, Reshape into labeled features; obtain the final output through a linear projection layer and a residual connection: ； in, It is a linear projection layer that restores the feature dimension. To address the challenges of insufficient feature generalization ability and the coupling interference of noise and geometric distortion in multi-source remote sensing images, a multi-scale edge enhancement module was designed as the core component of the feature preprocessing module, as follows: ； ； in, This represents average pooling with 3×3 cores; This represents a 1×1 convolutional layer with batch normalization and a sigmoid function; The scale is Features; then, an edge enhancer is introduced. Enhance the details at each scale: ； in, Features for edge enhancement; the structure of the edge enhancer is represented as: ； ； in, Then, these features are integrated: ； in, This is the feature output of the multi-scale edge enhancement module; thus, It possesses both fine-grained details and multi-scale edge information; these features can complement the overall feature set. The missing information is then concatenated, and two 3×3 convolutional layers and one 1×1 convolutional layer are applied to obtain the final result. : ； 。 4. The method according to claim 1, characterized in that, In step three, the feature fusion module is completed collaboratively by the grouped feature focusing unit and the multi-scale feature reconstruction unit: first, the features of adjacent stages are aligned in size and channel; then, the low-resolution features are upsampled and matched with the high-resolution features element by element after 1×1 convolution to obtain coarse fused features. Subsequently, spatial attention masks are generated using single-channel spatial compression and sigmoid, and position-sensitive weighting is applied to the coarse fused features to obtain context-aware representations. Further, the weighted features are grouped along the channel dimension. Within each group, local patterns are first refined using convolution, then reweighted using cross-channel attention masks. Finally, these features are concatenated by channel to form aggregated features with stronger channel correlation. Normalization is applied by referencing statistics from adjacent layers to stabilize the values ​​and inject positional information of small targets, thus obtaining a fused representation that combines spatial detail and correlation. Based on this, a multi-scale feature reconstruction unit generates channel-level weight thresholds using global average pooling of the fused base features, and adjusts the original features of adjacent layers. The features are batch normalized and Sigmoided separately to obtain spatial location importance maps. Strong and weak responses are separated by comparing location importance with channel thresholds. Strong responses are aggregated to preserve structure and edge details, while weak responses are aggregated to supplement background and global context. Finally, features are reconstructed through multi-scale convolution and element-wise fusion to reduce interference from irrelevant and noise components. Through the above cascaded strategy of "spatial attention - grouped channel interaction - statistical normalization - strong and weak separation - multi-scale reconstruction", effective alignment of semantics between adjacent layers and cross-channel correlation enhancement are achieved, improving the discriminativeness and robustness of multimodal matching features. The optimal fusion weights can be adaptively learned in end-to-end training.

5. The method according to claim 1, characterized in that, In step four, the overall model structure includes a feature extraction backbone network, a feature preprocessing module, a Transformer encoder, a coarse matching module, a feature fusion module, and a fine matching refinement module. The feature extraction backbone network is a multi-scale convolutional architecture with shared weights, focusing on extracting alignable semantic and texture representations at the 1 / 8 and 1 / 2 scales to provide basic features for cross-view modeling. The feature preprocessing module focuses on multi-scale edge enhancement, combining edge-guided attention and average pooling to amplify the sense. This enhances the structural signals of building outlines, road networks, and water system boundaries at multiple resolutions, while suppressing ambiguity caused by noise and radiation differences, outputting structure-sensitive representations. The Transformer encoder consists of stacked self-attention and cross-attention mechanisms, combined with two-dimensional positional encoding to achieve intra-image long-range context aggregation and cross-image semantic alignment, thereby capturing long-distance dependencies and cross-view correspondences. The coarse matching module constructs a cross-view similarity matrix based on low-resolution features and generates broad-coverage coarse matching anchors through bidirectional consistency checks and confidence filtering, providing priors for subsequent local searches. The feature fusion module consists of a grouped feature focusing unit and a multi-scale feature reconstruction unit. The grouped feature focusing unit enhances the correlation between layers and channels and suppresses redundancy and noise by aligning scale and channels, spatial attention and interaction between grouped channels. The multi-scale feature reconstruction unit performs strong and weak information separation and lightweight reconstruction on the fused representation, and prioritizes the preservation of small targets and edge texture details to improve cross-modal robustness and spatial reconstruction quality. The fine matching refinement module adopts a progressive local search strategy, constructs a local cost volume or window correlation around the coarse matching position on high-resolution features, and achieves sub-pixel-level localization by combining interpolation sampling and learnable regression or soft-argmax. It also introduces confidence estimation and robust filtering to output high-precision correspondence. Each stage maintains information flow and scale consistency through normalization, location coding, and skip connections, enabling the overall process to balance efficiency, stability, and sub-pixel matching accuracy in scenarios with complex geometry and radiation differences.

6. The method according to claim 1, characterized in that, In step five, 80% of the data is randomly extracted from the standardized multimodal sample set constructed in step one for training, and the remaining 20% ​​is used for validation. The samples maintain a spatial scale of 1024×1024, are normalized, and have a unified coordinate reference. During training, a randomly sampled controllable homography transformation is applied to the source image in any input image pair to generate a reference image, and the pixel-level true correspondence is derived online from the corresponding homography matrix, thus eliminating the error of manual annotation. At the same time, the geometric and photometric enhancement strategies consistent with those in step one are adopted to improve the model's generalization and noise resistance. Enhancements include random rotation, horizontal / vertical flipping, scaling, translation, homography parameter perturbation, brightness and contrast perturbation, motion blur, Gaussian noise, and speckle noise injection for SAR data, while ensuring effective geometric alignment. The enhanced multimodal image pairs are input into the network architecture designed in step four. They are sequentially passed through the feature extraction backbone network, the feature preprocessing module, and the Transformer encoder to obtain cross-view context representations. In the coarse matching module, candidate correspondences with broad coverage are generated. In the feature fusion module, grouped feature focusing and multi-scale feature reconstruction are realized. In the fine matching refinement module, sub-pixel-level localization is achieved through local cost volume / window correlation and learnable regression or soft-argmax, and the matching results are output. During training, the model performance is evaluated using a validation set after each epoch, and the model weights with the best validation performance are saved. At the same time, a learning rate decay strategy is introduced. When the validation set performance does not improve within several consecutive epochs, the learning rate is automatically reduced to promote convergence and improve generalization performance.

7. The method according to claim 1, characterized in that, In step six, to verify the effectiveness and robustness of the model, a comprehensive evaluation and index analysis of the model performance are conducted. System experiments are carried out on the dataset constructed in step one, with the main quantitative indicators including: root mean square error of localization and number of correct matches (NCM). RMSE measures sub-pixel localization accuracy, and NCM evaluates the trade-off between coverage and reliability. At the same time, computational efficiency and resource consumption are analyzed, including inference speed, number of parameters and memory usage, and comparative tests are conducted under different input resolutions and different modal combinations to verify the adaptability and scalability in cross-modal, weak texture and strong noise scenes.

8. The method according to claim 1, characterized in that, In step six, for qualitative evaluation, the response map of multi-scale edge enhancement, the cost volume / correlation heatmap of Transformer interactive attention and coarse-fine matching are visualized. The effect of structural prior on the suppression of noise and radiation inconsistency, and the fidelity improvement of small targets and edge details by grouped feature focusing and multi-scale feature reconstruction are analyzed. Furthermore, comparative experiments are conducted with existing representative methods to comprehensively evaluate the accuracy, stability and efficiency.

Images