A robust matching method for SAR images based on attention feature aggregation
By using a twin convolutional neural network and graph attention feature aggregation, the problem of balancing accuracy and efficiency in SAR image matching is solved, achieving fast and high-precision matching in complex scenes and eliminating large coarse-grained errors.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIANGTAN UNIV
- Filing Date
- 2026-03-26
- Publication Date
- 2026-06-23
AI Technical Summary
Existing SAR image matching algorithms struggle to balance accuracy and efficiency in complex large-area scenes, and are sensitive to local distortions, making it difficult to achieve fast and high-precision matching.
A feature encoder based on a Siamese convolutional neural network is combined with soft-argmax subpixel localization. The AGNN framework is used for graph attention feature aggregation, and the matching matrix is optimized by the Sinkhorn algorithm. Finally, ED-RANSAC is used for gross error detection to achieve fine matching.
It improves the accuracy and efficiency of SAR image matching, enabling fast and high-precision matching of corresponding points in complex scenes and eliminating potential large and coarse points.
Smart Images
Figure CN122265683A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of remote sensing image processing and computer vision interdisciplinary technology, specifically relating to a robust matching method for SAR images based on attention feature aggregation. Background Technology
[0002] For SAR image matching tasks, region-based feature-driven image matching algorithms are sensitive to scale and rotation and have low computational efficiency, thus being significantly affected by local distortions in SAR images and failing to achieve satisfactory results. While salient feature-based image matching methods show good performance in local SAR images with clear terrain features and simple scenes, there is still no universally applicable algorithm for complex scenes and large-area SAR image matching tasks. Therefore, this method has certain limitations and cannot achieve widely adaptable results.
[0003] Overall, in SAR image matching tasks in large-area complex scenes, feature-driven matching algorithms are “unstable” and it is difficult to simultaneously achieve both image matching accuracy and matching efficiency, making it difficult to meet the requirement of fast and high-precision matching of corresponding points in large-area SAR images. Summary of the Invention
[0004] This invention addresses the need for high-precision matching of tie points in spaceborne SAR images. To resolve the issue of instability in the feature matching process of traditional similarity measures, a robust SAR image matching method based on attention feature aggregation is designed.
[0005] The technical solution adopted by the method of the present invention is as follows, including the following steps:
[0006] Step 1: Input the SAR image pairs to be matched into a feature encoder based on a Siamese convolutional neural network to obtain the depth feature maps of each SAR image pair;
[0007] Step 2: Use Soft-argmax subpixel localization to decode the SAR image depth feature map and obtain subpixel-level feature points and their corresponding depth feature descriptors;
[0008] Step 3: Implement feature aggregation based on graph attention using the AGNN framework;
[0009] Step 4: Based on the soft assignment matrix, use the Sinkhorn algorithm to iteratively solve until convergence to obtain the probability of the registration matrix. Based on the cosine similarity threshold of the matching descriptor, obtain the information of the same point pairs with high matching probability to obtain the final matching matrix.
[0010] Step 5: After the feature points are output from the assigned matrix, gross error detection is performed based on ED-RANSAC to achieve fine matching of feature points in spaceborne SAR images.
[0011] The robust SAR image matching method based on attention feature aggregation comprises three modules:
[0012] Module 1: Feature Detection Module;
[0013] Module 2: Graph Attention Feature Aggregation Module;
[0014] Module 3: Fine matching and gross error detection module.
[0015] As a further improvement to the above technical solution:
[0016] In step 1, the feature encoder based on the Siamese convolutional neural network consists of convolutional layers, pooling layers, and nonlinear activation functions. Dimensionality reduction encoding of image features is performed based on convolutional pooling. The encoder has a total of 8 convolutional layers with 3×3 kernels and a layer size of 64-64-64-128-128-128-128-128. A 2×2 max-pooling layer is applied every two layers. ReLU normalization is performed after each convolutional layer in the network.
[0017] Step 2 includes:
[0018] Step 21: Feature point extraction based on the Soft-argmax subpixel localization module;
[0019] Step 22: Decoding the deep feature description.
[0020] In step 21, feature point extraction involves performing two convolutions on the encoded depth feature map, followed by normalization using the BatchNorm2 function to obtain a score feature map. Finally, by discarding invalid information from the Dustbin channel, a feature map of the same size as the original image is reconstructed to obtain the feature heatmap of the entire image. Non-maximum suppression and a threshold on the heatmap are then used to filter redundant candidate points. Finally, a softened Argmax function, i.e., the Soft-argmax module, is introduced to extract sub-pixel-level feature points. Its expression is:
[0021] (1)
[0022] In a single size On the heat map The processing formula is as follows:
[0023] (2)
[0024] in It is a heat map In position The value, This refers to the heatmap size. In contrast to the more common cross-channel softmax, spatial softmax is used here to ensure that each heatmap is normalized. The expression for the Soft-argmax function defined here is as follows:
[0025] (3)
[0026] Where d is a given component x or y, It corresponds to coordinates of Weight matrix, matrix It can be determined by its components and express, Indicates a single size of On the heat map The processing is defined as follows:
[0027] (4)
[0028] Finally, given the heatmap The regression position of the predicted sub-pixel coordinates of the feature point is given by the following formula:
[0029] (5)
[0030] The Soft-argmax operation can be viewed as a weighted average of points distributed across a uniform grid, where the weights are equal to the corresponding heatmaps. To integrate the Soft-argmax layer into a deep network, it is defined with respect to... The derivative is as follows:
[0031] (6)
[0032] Therefore, by applying backpropagation and the chain rule to Equation (4), the Soft-argmax function can be integrated into a trainable framework. Furthermore, the above equation shows that the higher the gradient value, the more exponentially the gradient increases, resulting in highly distinctive responses at locations of significant features.
[0033] In step 22, the deep feature descriptor is decoded by sequentially passing the encoded deep feature map through a 3×3 convolutional layer and a 1×1 convolutional layer, and finally by bilinear interpolation and L2 regularization to obtain a 256-dimensional deep feature descriptor.
[0034] Step 3 includes:
[0035] Step 31: Feature coupling based on location embedding. The location information of feature points is embedded into a high-dimensional vector using a multilayer perceptron, and then their visual appearance information is fused.
[0036] Step 32: Perform deep feature aggregation based on AGNN, and use graph attention mechanism to fuse the contextual information of feature points to create a more powerful image matching descriptor.
[0037] In step 31, assume that image A and image B have M and N feature points respectively, represented as A = {1, … , M} and B = {1, … , N}. The position of the feature points in each image is represented as follows: The corresponding descriptor is represented as The first in the image Each feature can be represented as , The characteristic response intensity of a feature point. Indicates the position of feature coordinates The descriptor at this location , Indicates the feature dimension.
[0038] By combining the visual appearance and location of feature points with descriptors, keypoint locations are embedded into a multilayer perceptron. In a high-dimensional vector, the coupling between visual appearance and feature point location is achieved, as shown in the formula: ,in For each key point The initial, This represents a multilayer perceptron, whose function is to upscale low-dimensional features. The above formula actually couples visual appearance with feature point location, which allows the subsequent attention mechanism to fully consider the appearance and positional similarity of features. Adding the descriptor to the encoded feature point yields the initial value of the input to the subsequent multilayer GNN:
[0039] (7)
[0040] (8)
[0041] in, , These represent feature descriptors on images A and B, respectively. , These represent the coordinate information of feature points on images A and B, respectively. , These represent the feature response intensity scores of feature points on images A and B, respectively. Function representation (Position embedding, PE) processing.
[0042] 4. In step 32, the feature points of the SAR image pair are processed as a graph neural network. Its nodes are each feature point in the image. The graph structure of the SAR image pair includes two different types of undirected edges: one is the feature connection within the SAR image, denoted as self-edge. One type connects internal feature points from a single SAR image; the other type connects feature points between two SAR images, denoted as cross-edge. It connects feature points in the reference SAR image with all feature points in the SAR image to be matched. The attention mechanism will... and Information aggregation .in It utilizes self-attention. Cross attention was utilized;
[0043] Based on the attention mechanism, if the query Element-based attributes, i.e., keys The values of some elements were retrieved. The formula is as follows:
[0044] (9)
[0045] Attention weight It queries and retrieves the similarity of object key values. ,Right now:
[0046] (10)
[0047] For the attention feature aggregation process of two sets of SAR image feature point map network sequences to be matched, assume a certain depth feature point Located on the reference image M, all the meta-feature points are located on the image to be matched N, where ,but:
[0048] (11)
[0049] (12)
[0050] Each layer Each feature point has its own set of projection parameters, which are shared by all feature points. Here... Corresponding to a feature point on the image to be queried One representation, as well as All of these are image feature points from the recall. One representation (mapping) of this is the self-attention process; The similarity between these two features is represented by... as well as The calculation shows that the larger the value, the more similar the two features are. This similarity score is then used to... Weighted summation This refers to the cross-attention process. Both the self-attention layer and the cross-attention layer employ a multi-head attention structure, but the difference lies in that the former uses the same descriptor as its input, while the latter uses descriptors from different images as its input.
[0051] make Represented as the first image on image A The element in the first... The intermediate form of expression in a layer is information (message). It is the result after aggregating all feature points from image A and image B. Therefore, all features in image A The process of transmitting updated residual information is as follows:
[0052] (13)
[0053] Similarly, let Represented as the first image on image B The element in the first... All features in the intermediate representation of layer B The process of transmitting updated residual information is as follows:
[0054] (14)
[0055] In the above formula, This indicates a concatenated operation, where self-attention and cross-attention are concatenated and alternately updated. The self-attention layer is followed by the cross-attention layer, with a fixed number of L layers for iterative computation. The self-attention layer makes the features more specific for matching, while the cross-attention layer uses these specific points to compare the similarity of features between images. In the specific implementation of the cross-attention module, a residual connection module is introduced to maintain the information flow of the deep network and also to ensure the stability of gradient backpropagation during training.
[0056] Therefore, after performing L self-attention and cross-attention operations on images A and B, the output of the attention GNN can be obtained, defined as:
[0057] (15)
[0058] (16)
[0059] Will , Defined as a matching descriptor, this process realizes the fusion of image feature descriptor information with global features within the image and cross-global features between images. The matching descriptor after feature coupling contains the location information and local appearance information of SAR image feature points, and also integrates the differential information between feature nodes within the image and the homogenization information of similar features with the same name between images.
[0060] In step 4, the soft allocation matrix The matrix has the following relationship: In the assignment matrix, all elements should be 1, and the value of each element represents the matching probability of the corresponding node feature. Ideally, the sum of the rows or columns of matrix P should equal 1. M and N are the number of feature points in image A and image B, respectively. The matching degree assignment matrix is as follows: By maximizing the score The allocation matrix can then be obtained. To enable the matching model to eliminate gross errors, in the scoring matrix... Setting an empty set in the last column / row will give you... Based on the obtained score matrix and allocation matrix Given the constraints, construct the Sinkhorn distance function: The Sinkhorn algorithm is used to solve the problem. The initial values are all set to 1. The probability of obtaining the registration matrix is obtained by iterative solution until convergence. Based on the cosine similarity threshold of the matching descriptor, the information of the same point pairs with high matching probability is obtained to obtain the final matching matrix.
[0061] In step 5, the ED-RANSAC-based gross error detection calculates the intermediate transformation matrix by randomly selecting SAR image matching pairs as samples. Based on the intermediate transformation matrix, the sample set, and the error metric function, a consensus set satisfying the current transformation matrix is calculated, and the optimal consensus set is iteratively updated. Euclidean distance is introduced as a threshold to perform a secondary screening of the optimal consensus set, retaining the matching pairs in the optimal set that satisfy the threshold condition as the final set of corresponding points for calculating the transformation matrix. The formula is as follows:
[0062] (17)
[0063] (18)
[0064] in, The transformation parameters are obtained by least-squares decomposition of four randomly selected points. For the coordinates of the corresponding point on the reference image, These are the coordinates of the corresponding points on the image to be registered after transformation.
[0065] The beneficial effects of this invention are:
[0066] (1) This invention utilizes the Soft-argmax subpixel localization module to achieve subpixel extraction of feature points;
[0067] (2) The present invention achieves the coupling of local and global information of geometrically stable feature points through the graph attention aggregation module, guiding the model to focus on key information in a targeted manner.
[0068] (3) The image matching problem is transformed into an optimization transmission problem in graph matching, and fast feature matching is achieved using the Sinkhorn algorithm. In addition, an ED-RANSAC gross error detection algorithm is designed to eliminate potential large gross errors and further improve the matching accuracy of SAR image tie points. Attached Figure Description
[0069] Figure 1 This is a schematic diagram of the framework of a robust SAR image matching method based on attention feature aggregation according to the present invention. Specific implementation methods
[0070] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0071] A robust SAR image matching method based on attention feature aggregation, characterized by the following steps:
[0072] Step 1: Input the SAR image pairs to be matched into a feature encoder based on a Siamese convolutional neural network to obtain the depth feature maps of each SAR image pair;
[0073] The feature encoder based on a Siamese convolutional neural network consists of convolutional layers, pooling layers, and nonlinear activation functions, using convolutional pooling for dimensionality reduction encoding of image features. The encoder has eight convolutional layers with 3×3 kernels and a layer size of 64-64-64-128-128-128-128-128. Every two layers have a 2×2 max-pooling layer. ReLU normalization is performed after each convolutional layer in the network.
[0074] Step 2: Use Soft-argmax subpixel localization to decode the SAR image depth feature map and obtain subpixel-level feature points and their corresponding depth feature descriptors;
[0075] Step 2 includes the following two steps:
[0076] Step 21: Feature point extraction based on the Soft-argmax subpixel localization module; feature point extraction is performed by convolution twice on the encoded depth feature map, followed by normalization using the BatchNorm2 function to obtain a score feature map. Finally, by discarding invalid information from the Dustbin channel, a feature map of the same size as the original image is reconstructed to obtain the feature heatmap of the entire image. Non-maximum suppression and a threshold on the heatmap are then used to filter redundant candidate points. Finally, the softened Argmax function, i.e., the Soft-argmax module, is introduced to extract subpixel-level feature points. Its expression is:
[0077] (1)
[0078] In a single size On the heat map The processing formula is as follows:
[0079] (2)
[0080] in It is a heat map In position The value, This refers to the heatmap size. In contrast to the more common cross-channel softmax, spatial softmax is used here to ensure that each heatmap is normalized. The expression for the Soft-argmax function defined here is as follows:
[0081] (3)
[0082] Where d is a given component x or y, It corresponds to coordinates of Weight matrix, matrix It can be determined by its components and express, Indicates a single size of On the heat map The processing is defined as follows:
[0083] (4)
[0084] Finally, given the heatmap The regression position of the predicted sub-pixel coordinates of the feature point is given by the following formula:
[0085] (5)
[0086] The Soft-argmax operation can be viewed as a weighted average of points distributed across a uniform grid, where the weights are equal to the corresponding heatmaps. To integrate the Soft-argmax layer into a deep network, it is defined with respect to... The derivative is as follows:
[0087] (6)
[0088] Therefore, by applying backpropagation and the chain rule to Equation (4), the Soft-argmax function can be integrated into a trainable framework. Furthermore, the above equation shows that the higher the gradient value, the more exponentially the gradient increases, resulting in highly distinctive responses at locations of significant features.
[0089] Step 22: Decoding the deep feature descriptor; The deep feature descriptor is decoded by sequentially passing the encoded deep feature map through a 3×3 convolutional layer, a 1×1 convolutional layer, and finally through bilinear interpolation and L2 regularization to obtain a 256-dimensional deep feature descriptor.
[0090] Step 3: Implement feature aggregation based on graph attention using the AGNN framework;
[0091] Step 3 includes the following two steps:
[0092] Step 31: Assume that image A and image B have M and N feature points respectively, represented as A = {1, … , M}, B = {1, … , N}. The position of the feature points in each image is represented as… The corresponding descriptor is represented as The first in the image Each feature can be represented as , The characteristic response intensity of a feature point. Indicates the position of feature coordinates The descriptor at this location , Indicates the feature dimension.
[0093] By combining the visual appearance and location of feature points with descriptors, keypoint locations are embedded into a multilayer perceptron. In a high-dimensional vector, the coupling between visual appearance and feature point location is achieved, as shown in the formula: ,in For each key point Initially, MLP stands for Multilayer Perceptron, which is used to upscale low-dimensional features. The above formula actually couples visual appearance with feature point location. This encoding form allows the subsequent attention mechanism to fully consider the appearance and positional similarity of features. Adding the descriptor to the encoded feature point yields the initial value of the input to the subsequent multilayer GNN:
[0094] (7)
[0095] (8)
[0096] in, , These represent feature descriptors on images A and B, respectively. , These represent the coordinate information of feature points on images A and B, respectively. , These represent the feature response intensity scores of feature points on images A and B, respectively. Function representation (Position embedding, PE) processing.
[0097] In step 32, the input feature points of the SAR image pair are processed as a graph neural network. Its nodes are each feature point in the image. The graph structure of the SAR image pair includes two different types of undirected edges: one is the feature connection within the SAR image, denoted as self-edge. One type connects internal feature points from a single SAR image; the other type connects feature points between two SAR images, denoted as cross-edge. It connects feature points in the reference SAR image with all feature points in the SAR image to be matched. The attention mechanism will... and Information aggregation .in It utilizes self-attention. It utilizes cross attention.
[0098] Based on the attention mechanism, if the query Element-based attributes, i.e., keys The values of some elements were retrieved. The formula is as follows:
[0099] (9)
[0100] Attention weight It is the Softmax of the similarity between the key values of the retrieved and unretrieved objects, that is:
[0101] (10)
[0102] For the attention feature aggregation process of two sets of SAR image feature point map network sequences to be matched, assume a certain depth feature point Located on the reference image M, all the meta-feature points are located on the image to be matched N, where ,but:
[0103] (11)
[0104] (12)
[0105] Each layer Each feature point has its own set of projection parameters, which are shared by all feature points. Here... Corresponding to a feature point on the image to be queried One representation, as well as All of these are image feature points from the recall. One representation (mapping) of this is the self-attention process; The similarity between these two features is represented by... as well as The calculation shows that the larger the value, the more similar the two features are. This similarity score is then used to... Weighted summation This refers to the cross-attention process. Both the self-attention layer and the cross-attention layer employ a multi-head attention structure, but the difference lies in that the former uses the same descriptor as its input, while the latter uses descriptors from different images as its input.
[0106] make Represented as the first image on image A The element in the first... The intermediate form of expression in a layer is information (message). It is the result after aggregating all feature points from image A and image B. Therefore, all features in image A The process of transmitting updated residual information is as follows:
[0107] (13)
[0108] Similarly, let Represented as the first image on image B The element in the first... All features in the intermediate representation of layer B The process of transmitting updated residual information is as follows:
[0109] (14)
[0110] In the above formula, This indicates a concatenated operation, where self-attention and cross-attention are concatenated and updated alternately, starting with a self-attention layer followed by a cross-attention layer, for a fixed number of L layers in total. The self-attention layer makes features more specific for matching, while the cross-attention layer uses these specific points to compare the similarity of features between images. In the specific implementation of the cross-attention module, a residual connection module is introduced to maintain the information flow of the deep network and also to ensure the stability of gradient backpropagation during training.
[0111] Therefore, after performing L self-attention and cross-attention operations on images A and B, the output of the attention GNN can be obtained, defined as:
[0112] (15)
[0113] (16)
[0114] Will , Defined as a matching descriptor, this process realizes the fusion of image feature descriptor information with global features within the image and cross-global features between images. The matching descriptor after feature coupling contains the location information and local appearance information of SAR image feature points, and also integrates the differential information between feature nodes within the image and the homogenization information of similar features with the same name between images.
[0115] Step 4: Based on the soft assignment matrix, use the Sinkhorn algorithm to iteratively solve until convergence to obtain the probability of the registration matrix. Based on the cosine similarity threshold of the matching descriptor, obtain the information of the same point pairs with high matching probability to obtain the final matching matrix.
[0116] Among them, soft allocation matrix The matrix has the following relationship: In the assignment matrix, all elements should be 1, and the value of each element represents the matching probability of the corresponding node feature. Ideally, the sum of the rows or columns of matrix P should equal 1. M and N are the number of feature points in image A and image B, respectively. The matching degree assignment matrix is as follows: By maximizing the score The allocation matrix can then be obtained. To enable the matching model to eliminate gross errors, in the scoring matrix... Setting an empty set in the last column / row will give you... Based on the obtained score matrix and allocation matrix Given the constraints, construct the Sinkhorn distance function: The Sinkhorn algorithm is used to solve the problem. The initial values are all set to 1. The probability of obtaining the registration matrix is obtained by iterative solution until convergence. Based on the cosine similarity threshold of the matching descriptor, the information of the same point pairs with high matching probability is obtained to obtain the final matching matrix.
[0117] Step 5: After the feature points are output from the assigned matrix, gross error detection is performed based on ED-RANSAC to achieve fine matching of feature points in spaceborne SAR images.
[0118] In this method, ED-RANSAC-based gross error detection calculates intermediate transformation matrices by randomly selecting SAR image matching pairs as samples. Based on the intermediate transformation matrix, the sample set, and the error metric function, a consensus set satisfying the current transformation matrix is calculated, and the optimal consensus set is iteratively updated. Euclidean distance is introduced as a threshold to perform a secondary screening of the optimal consensus set, retaining the matching pairs satisfying the threshold condition in the optimal set as the final set of corresponding points for calculating the transformation matrix. The formula is as follows:
[0119] (17)
[0120] (18)
[0121] in, The transformation parameters are obtained by least-squares decomposition of four randomly selected points. For the coordinates of the corresponding point on the reference image, These are the coordinates of the corresponding points on the image to be registered after transformation.
[0122] The robust SAR image matching method based on attention feature aggregation includes three modules:
[0123] Module 1: Feature Detection Module;
[0124] Module 2: Graph Attention Feature Aggregation Module;
[0125] Module 3: Fine matching and gross error detection module.
Claims
1. A robust SAR image matching method based on attention feature aggregation, characterized in that, Includes the following steps: Step 1: Input the SAR image pairs to be matched into a feature encoder based on a Siamese convolutional neural network to obtain the depth feature maps of each SAR image pair; Step 2: Use Soft-argmax subpixel localization to decode the SAR image depth feature map and obtain subpixel-level feature points and their corresponding depth feature descriptors; Step 3: Implement feature aggregation based on graph attention using the AGNN framework; Step 4: Based on the soft assignment matrix, use the Sinkhorn algorithm to iteratively solve until convergence to obtain the probability of the registration matrix. Based on the cosine similarity threshold of the matching descriptor, obtain the information of the same point pairs with high matching probability to obtain the final matching matrix. Step 5: After the feature points are output from the assigned matrix, gross error detection is performed based on ED-RANSAC to achieve fine matching of feature points in the spaceborne SAR image; The robust SAR image matching method based on attention feature aggregation comprises three modules: Module 1: Feature Detection Module; Module 2: Graph Attention Feature Aggregation Module; Module 3: Fine matching and gross error detection module.
2. The robust SAR image matching method based on attention feature aggregation according to claim 1, characterized in that: In step 1, the feature encoder based on the Siamese convolutional neural network consists of convolutional layers, pooling layers, and nonlinear activation functions. Dimensionality reduction encoding of image features is performed based on convolutional pooling. The encoder has a total of 8 convolutional layers with 3×3 kernels and a layer size of 64-64-64-128-128-128-128-128. A 2×2 max-pooling layer is applied every two layers. ReLU normalization is performed after each convolutional layer in the network.
3. The robust SAR image matching method based on attention feature aggregation according to claim 2, characterized in that: Step 2 includes: Step 21: Feature point extraction based on the Soft-argmax subpixel localization module; Step 22: Decoding the deep feature description.
4. The robust SAR image matching method based on attention feature aggregation according to claim 3, characterized in that: In step 21, feature point extraction involves performing two convolutions on the encoded depth feature map, followed by normalization using the BatchNorm2 function to obtain a score feature map. Finally, by discarding invalid information from the Dustbin channel, a feature map of the same size as the original image is reconstructed to obtain the feature heatmap of the entire image. Non-maximum suppression and a threshold on the heatmap are then used to filter redundant candidate points. Finally, a softened Argmax function, i.e., the Soft-argmax module, is introduced to extract sub-pixel-level feature points. Its expression is: (1) In a single size On the heat map The processing formula is as follows: (2) in It is a heat map In position The value, This refers to the heatmap size. In contrast to the more common cross-channel softmax, spatial softmax is used here to ensure that each heatmap is normalized. The expression for the Soft-argmax function defined here is as follows: (3) Where d is a given component x or y, It corresponds to coordinates of Weight matrix, matrix It can be determined by its components and express, Indicates a single size of On the heat map The processing is defined as follows: (4) Finally, given the heatmap The regression position of the predicted sub-pixel coordinates of the feature point is given by the following formula: (5) The Soft-argmax operation can be viewed as a weighted average of points distributed across a uniform grid, where the weights are equal to the corresponding heatmaps. To integrate the Soft-argmax layer into a deep network, it is defined with respect to... The derivative is as follows: (6) Therefore, by applying backpropagation and the chain rule to Equation (4), the Soft-argmax function can be integrated into a trainable framework. Furthermore, the above equation shows that the higher the gradient value, the more exponentially the gradient increases, resulting in highly distinctive responses at locations of significant features.
5. The robust SAR image matching method based on attention feature aggregation according to claim 4, characterized in that: In step 22, the deep feature descriptor is decoded by sequentially passing the encoded deep feature map through a 3×3 convolutional layer and a 1×1 convolutional layer, and finally by bilinear interpolation and L2 regularization to obtain a 256-dimensional deep feature descriptor.
6. The robust SAR image matching method based on attention feature aggregation according to claim 1, characterized in that: Step 3 includes: Step 31: Feature coupling based on location embedding. The location information of feature points is embedded into a high-dimensional vector using a multilayer perceptron, and then their visual appearance information is fused. Step 32: Perform deep feature aggregation based on AGNN, and use graph attention mechanism to fuse the contextual information of feature points to create a more powerful image matching descriptor.
7. The robust SAR image matching method based on attention feature aggregation according to claim 6, characterized in that: In step 31, assume that image A and image B have M and N feature points respectively, represented as A = {1, … , M} and B = {1, … , N}. The position of the feature points in each image is represented as follows: The corresponding descriptor is represented as The first in the image Each feature can be represented as , The characteristic response intensity of a feature point. Indicates the position of feature coordinates The descriptor at this location , Indicates the feature dimension; By combining the visual appearance and location of feature points with descriptors, keypoint locations are embedded into a multilayer perceptron. In a high-dimensional vector, the coupling between visual appearance and feature point location is achieved, as shown in the formula: ,in For each key point Initially, MLP stands for Multilayer Perceptron, which is used to upscale low-dimensional features. The above formula actually couples visual appearance with feature point location. This encoding form allows the subsequent attention mechanism to fully consider the appearance and positional similarity of features. Adding the descriptor to the encoded feature point yields the initial value of the input to the subsequent multilayer GNN: (7) (8) in, , These represent feature descriptors on images A and B, respectively. , These represent the coordinate information of feature points on images A and B, respectively. , These represent the feature response intensity scores of feature points on images A and B, respectively. Function representation (Position embedding, PE) processing.
8. The robust SAR image matching method based on attention feature aggregation according to claim 6, characterized in that: In step 32, the input feature points of the SAR image pair are processed as a graph neural network. Its nodes are each feature point in the image. The graph structure of the SAR image pair includes two different types of undirected edges: one is the feature connection within the SAR image, denoted as self-edge. One type connects internal feature points from a single SAR image; the other type connects feature points between two SAR images, denoted as cross-edge. It connects feature points in the reference SAR image with all feature points in the SAR image to be matched. The attention mechanism will... and Information aggregation .in It utilizes self-attention. It utilizes cross attention; Based on the attention mechanism, if the query Element-based attributes, i.e., keys The values of some elements were retrieved. The formula is as follows: (9) Attention weight It is the Softmax of the similarity between the key values of the retrieved and queryed objects, that is: (10) For the attention feature aggregation process of two sets of SAR image feature point map network sequences to be matched, assume a certain depth feature point Located on the reference image M, all the meta-feature points are located on the image to be matched N, where ,but: (11) (12) Each floor Each feature point has its own set of projection parameters, which are shared by all feature points. Here... Corresponding to a feature point on the image to be queried One representation, as well as All of these are image feature points from the recall. One representation (mapping) of this is the self-attention process; The similarity between these two features is represented by... as well as The calculation shows that the larger the value, the more similar the two features are. This similarity score is then used to... Weighted summation This refers to the cross-attention process. Both the self-attention layer and the cross-attention layer employ a multi-head attention structure, but the difference lies in that the former uses the same descriptor as its input, while the latter uses descriptors from different images as its input. make Represented as the first image on image A The element in the first... The intermediate form of expression in a layer is information (message). It is the result after aggregating all feature points from image A and image B. Therefore, all features in image A The method for transmitting updated residual information is: (13) Similarly, let Represented as the first image on image B The element in the first... All features in the intermediate representation of layer B The method for transmitting updated residual information is: (14) In the above formula, This indicates a concatenated operation, where self-attention and cross-attention are concatenated and alternately updated. The self-attention layer is followed by the cross-attention layer, with a fixed number of L layers for iterative computation. The self-attention layer makes the features more specific for matching, while the cross-attention layer uses these specific points to compare the similarity of features between images. In the specific implementation of the cross-attention module, a residual connection module is introduced to maintain the information flow of the deep network and also to ensure the stability of gradient backpropagation during training. Therefore, after performing L self-attention and cross-attention operations on images A and B, the output of the attention GNN can be obtained, defined as: (15) (16) Will , Defined as a matching descriptor, this process realizes the fusion of image feature descriptor information with global features within the image and cross-global features between images. The matching descriptor after feature coupling contains the location information and local appearance information of SAR image feature points, and also integrates the differential information between feature nodes within the image and the homogenization information of similar features with the same name between images.
9. The robust SAR image matching method based on attention feature aggregation according to claim 1, characterized in that: In step 4, the soft allocation matrix The matrix has the following relationship: In the assignment matrix, all elements should be 1, and the value of each element represents the matching probability of the corresponding node feature. Ideally, the sum of the rows or columns of matrix P should be equal to 1. M and N are the number of feature points in image A and image B, respectively. The matching degree assignment matrix is as follows: By maximizing the score The allocation matrix can then be obtained. To enable the matching model to eliminate gross errors, in the scoring matrix... Setting an empty set in the last column / row will give you... Based on the obtained score matrix and allocation matrix Given the constraints, construct the Sinkhorn distance function: The Sinkhorn algorithm is used to solve the problem. The initial values are all set to 1. The probability of obtaining the registration matrix is obtained by iterative solution until convergence. Based on the cosine similarity threshold of the matching descriptor, the information of the same point pairs with high matching probability is obtained to obtain the final matching matrix.
10. The robust SAR image matching method based on attention feature aggregation according to claim 1, characterized in that: In step 5, the ED-RANSAC-based gross error detection calculates the intermediate transformation matrix by randomly selecting SAR image matching pairs as samples. Based on the intermediate transformation matrix, the sample set, and the error metric function, a consensus set satisfying the current transformation matrix is calculated, and the optimal consensus set is iteratively updated. Euclidean distance is introduced as a threshold to perform a secondary screening of the optimal consensus set, retaining the matching pairs in the optimal set that satisfy the threshold condition as the final set of corresponding points for calculating the transformation matrix. The formula is as follows: (17) (18) in, The transformation parameters are obtained by least-squares decomposition of four randomly selected points. For the coordinates of the corresponding point on the reference image, These are the coordinates of the corresponding points on the image to be registered after transformation.