A reliable image registration method and system for calibrating attention networks

Abstract

This invention discloses an image registration method using a reliability-calibrated attention network, belonging to the field of computer vision and image processing technology. The method extracts feature points and descriptors from two images to establish an initial corresponding point set; it employs a three-level progressive pruning structure consisting of an initial stage and two refinement stages; each stage utilizes a learnable soft-thresholding Transformer to extract features, combining spatial attention with learnable soft-thresholding channel attention to adaptively generate thresholds based on the current corresponding point set, suppressing weak channel correlations before attention normalization; through a prior-guided cross-stage aggregation module, it constructs an affinity prior using initial stage features and injects cross-stage attention, aggregating early complementary information to the final stage; a weighted eight-point algorithm is used to estimate the essential matrix, and full-size geometric verification is performed, outputting reliable matching point pairs and the essential matrix. This invention reduces outlier interference and improves the accuracy and robustness of interior point selection and geometric relationship estimation in image registration.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, and specifically relates to an image registration method and system for reliably calibrating attention networks. Background Technology

[0002] Image registration is a fundamental task in computer vision and image processing. A crucial step is establishing reliable correspondences between feature points in two related images and estimating their geometric relationships, thus supporting 3D reconstruction, camera pose estimation, and visual localization. In feature-based image registration, feature points and their descriptors are first extracted from the two images, and then an initial match is established based on descriptor similarity. Due to factors such as viewpoint variations, repetitive textures, and occlusion, the initial match often contains erroneous correspondences, i.e., outliers. These erroneous matches interfere with geometric model estimation; therefore, it is necessary to remove outliers and retain correct correspondences that satisfy geometric consistency, i.e., interior points.

[0003] In existing technologies, traditional methods such as RANSAC estimate the geometric model from a randomly sampled minimum subset and select the model supported by the largest consensus set; MAGSAC++ reduces the sensitivity to manually set inlier thresholds by marginalizing noise scales. However, when the proportion of outliers is high, the computational cost of sampling methods increases and they are easily affected by the quality of model assumptions. Deep learning-based methods such as OANet, CL-Net, ConvMatch, BCLNet, MambaMatch, LeCoT, and MGCA-Net improve the ability to distinguish between correct and incorrect matches by learning the features and contextual information of initial matching point pairs. However, the above methods still have shortcomings: local graph reasoning mainly aggregates neighborhood information and has limited ability to model long-range associations between different matching point pairs; dense channel attention causes weak correlation responses induced by outliers to participate in feature aggregation, reducing feature quality; multi-stage pruning produces correspondence features with different degrees of refinement, but it is difficult to effectively utilize the complementary information retained in the early stages by simply passing these features sequentially or directly fusing them. The aforementioned problems limit the selection of reliable matching point pairs, thereby affecting the accuracy and robustness of essential matrix estimation in image registration. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an image registration method and system for reliable calibration attention networks, so as to improve the accuracy and robustness of reliable matching point pair selection and geometric relationship estimation in image registration with a high proportion of outliers.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: an image registration method for a reliability-calibrated attention network, comprising the following steps:

[0006] S1. Obtain an image to be registered, extract feature points and descriptors from the two images respectively, and establish an initial set of corresponding points based on the nearest neighbor matching of the descriptors;

[0007] S2. Input the initial corresponding point set into the reliability calibration attention network. The reliability calibration attention network adopts a three-level progressive pruning structure consisting of an initial stage and two refinement stages. The initial stage processes the initial corresponding point set. The refinement stage concatenates the original corresponding point set, the confidence score predicted in the previous stage, and the geometric feedback along the feature dimension as input. The confidence score is obtained by transforming the in-point score in the previous stage. The geometric feedback is calculated based on the essential matrix estimated in the previous stage. Gradient propagation of the confidence score and the geometric feedback is stopped before concatenation.

[0008] S3. At each stage, the input is projected onto the feature space, and a learnable soft threshold Transformer is used to extract features within the stage to obtain stage features. The learnable soft threshold Transformer includes spatial attention and learnable soft threshold channel attention. The learnable soft threshold channel attention adaptively generates a threshold based on the current input corresponding point set and suppresses weak channel correlation before attention normalization to model the long-range spatial and channel dependencies between correspondences.

[0009] S4. For each stage, the corresponding relationships are filtered based on the predicted interior point scores, the essential matrix of the stage is estimated using the weighted eight-point algorithm, and the geometric residual of the original correspondence is calculated.

[0010] S5. The cross-stage aggregation module guided by priors fuses the stage features of the three stages: a correspondence affinity prior is constructed based on the initial stage features, the features of the first two stages are fused to obtain early fusion features, a query is generated with the final stage features, and keys and values ​​are generated with the early fusion features, and cross-stage attention is calculated; the correspondence affinity prior is injected into the cross-stage attention after a learnable transformation, and the value features are weighted and aggregated after normalization, and the aggregation result is projected and added to the residual of the final stage features to obtain the cross-stage aggregated features; the cross-stage aggregated features are refined by residual blocks with channel recalibration.

[0011] S6. Based on the refined features, predict the interior point weights for all initial correspondences, estimate the final essential matrix using the weighted eight-point algorithm, and use the final essential matrix to perform full-size verification on the initial correspondence point set. Filter reliable interior points according to the symmetric epipolar distance to obtain reliable matching point pairs and the essential matrix describing the geometric relationship between images.

[0012] Further, in step S1, for two images in a set of images to be registered, SIFT or SuperPoint is used to extract feature points and their descriptors from the two images respectively, and then an initial set of corresponding points S is established by nearest neighbor matching of the descriptors:

[0013]

[0014]

[0015] Where N represents the number of initial correspondences. This represents the i-th correspondence, where each correspondence consists of the normalized two-dimensional coordinates of matching feature points in two images to be registered.

[0016] Furthermore, in step S2, the input for the refinement stage is:

[0017]

[0018] Where t=1,2, , The interior point score predicted in the previous stage. The symmetric polar residuals are calculated based on the essential matrix estimated in the previous stage. and Stop gradient propagation before splicing.

[0019] Furthermore, in step S2, after each stage projects the input into the feature space, the refinement stage combines the projected features of the current stage, the graph representation of the previous stage, and the output of the previous stage to construct a neighborhood graph, and aggregates them in a multi-graph fusion manner; the second refinement stage also introduces the output of the initial stage; within the stage, the corresponding relationship features are extracted by combining point convolutional networks, learnable soft threshold Transformers, and local graph modules.

[0020] The local graph module constructs a dynamic neighborhood and fuses the maximum aggregation and ring aggregation results through local and global gating. Then, it compresses and restores the corresponding relationship representation through sequential perceptual pooling, filtering and inverse pooling, and then inputs another learnable soft threshold Transformer to combine local geometric consistency with long-range spatial and channel interactions.

[0021] Further, in step S3, the learnable soft-threshold Transformer includes two consecutive spatial-channel layers, each of which performs spatial attention and learnable soft-threshold channel attention in sequence, and a feedforward network and residual connection are set after each attention sub-layer;

[0022] The spatial attention is configured to: calculate the similarity between the query and the key along the correspondence dimension, and then perform weighted aggregation of the value features after normalization to model the long-range spatial association between different matching point pairs;

[0023] The learnable soft-threshold channel attention is configured as follows: Input features Z are acquired, with dimensions B×N×C, where B, N, and C represent the batch size, the number of correspondences, and the number of feature channels, respectively; after 1×1 projection, the features are divided into H attention heads, where the query, key, and value features of the h-th head are respectively... , , Perform L2 normalization on the query and key along the correspondence dimension, and calculate the channel affinity matrix of the h-th head:

[0024]

[0025] in, Let h be the learnable temperature parameter for the h-th attention head; This indicates that L2 normalization is performed along the correspondence dimension;

[0026] Global average pooling is performed on the input correspondence features, and a threshold τ and a differentiable soft mask related to the current sample are generated by a threshold predictor. :

[0027]

[0028] Where ψ is the threshold predictor, which uses a multilayer perceptron; σ represents the sigmoid function, and α controls the slope of the sigmoid relaxation. The soft mask for the h-th header;

[0029] The channel affinity scores are masked and modulated, and then Softmax normalized sum feature aggregation is performed.

[0030]

[0031]

[0032]

[0033] in, The channel affinity score is the result of masking, where κ is a positive masking constant, and ⊙ denotes element-wise multiplication. Cat represents the aggregation result of the h-th head, where Cat indicates that the outputs of each attention head are concatenated. This indicates the output projection function. This represents a learnable soft-threshold channel attention map. This represents the output feature after the input feature Z has been processed by the learnable soft-threshold channel attention mapping; by first concatenating the aggregation results of each attention head, then projecting the output, and finally adding it to the input feature residual, the corresponding output feature is obtained.

[0034] In the learnable soft-threshold channel attention, the comparison between the threshold and the channel affinity score gives stronger correlations a larger retention weight, while weaker correlations are suppressed; the soft mask modulates the channel affinity score before attention normalization.

[0035] Furthermore, in step S4, the stage features of each stage retain the correspondence index consistent with the original corresponding point set; for each supervised output, the correspondences are sorted according to the prediction score, and the top half of the correspondences are selected to form the system. The dual-channel prediction head outputs a mask score and a weight score. The product of the mask score and the weight score is normalized to obtain the weight. The essential matrix is ​​estimated using the weighted eight-point algorithm. And calculate the geometric residuals of each original correspondence. , where g represents the weighted eight-point algorithm and d represents the symmetric epipolar distance.

[0036] Furthermore, in step S5, the prior-guided cross-stage aggregation module, based on the initial stage characteristics... Constructing correspondence affinity priors The features from the first two stages are then added together to obtain the early fusion features. Where C is the number of feature channels; based on the final stage features Generate query Q to fuse features early. Generate key K and value V, and calculate cross-stage attention score. ; Perform a learnable affine transformation and Leaky ReLU activation on the prior P to obtain the transformed prior:

[0037]

[0038] in, and These are the learnable scaling and offset parameters, respectively;

[0039] Will Incorporating cross-stage attention scores to obtain a prior-guided attention matrix Then, the aggregated feature values ​​are obtained as follows:

[0040]

[0041] in, Indicates output projection. This is a cross-stage aggregation feature.

[0042] Furthermore, in step S5, the cross-stage aggregated features are refined by two residual blocks with channel recalibration; each residual block with channel recalibration is configured to recalibrate the input features in the channel dimension and add the recalibrated features to the input feature residuals.

[0043] Further, in step S6, the full-size verification includes: calculating the symmetric epipolar distance of each original correspondence using the final estimated essential matrix, determining the correspondences whose symmetric epipolar distance satisfies the inlier threshold as reliable inliers, and retaining the corresponding matching point pairs and the final essential matrix.

[0044] The present invention also provides an image registration system for reliably calibrating attention networks, for performing the above-described method, comprising:

[0045] The initial matching module is used to obtain the image pair to be registered, extract the feature points and descriptors of the two images respectively, and establish an initial corresponding point set based on the nearest neighbor matching of the descriptors;

[0046] A progressive pruning network module is used to process the initial corresponding point set using a three-level progressive pruning structure consisting of an initial stage and two refinement stages. The initial stage processes the initial corresponding point set, and the refinement stage concatenates the original corresponding point set, the confidence score predicted in the previous stage, and the geometric feedback along the feature dimension as input. The confidence score is obtained by transforming the intra-point scores in the previous stage, and the geometric feedback is calculated based on the essential matrix estimated in the previous stage. Gradient propagation of the confidence score and the geometric feedback is stopped before concatenation. Each stage uses a learnable soft-thresholding Transformer for intra-stage feature extraction. The learnable soft-thresholding Transformer includes spatial attention and learnable soft-thresholding channel attention. The learnable soft-thresholding channel attention adaptively generates a threshold based on the current input corresponding point set and suppresses weak channel correlations before attention normalization.

[0047] A priori-guided cross-stage aggregation module is used to fuse stage features from three stages: a correspondence affinity prior is constructed based on initial stage features; the features of the first two stages are fused to obtain early fusion features; queries are generated using the final stage features, and keys and values ​​are generated using the early fusion features; cross-stage attention is calculated; the correspondence affinity prior is injected into the cross-stage attention after a learnable transformation; after normalization, the value features are weighted and aggregated; the aggregation result is projected and added to the residual of the final stage features to obtain the cross-stage aggregated features; the cross-stage aggregated features are refined by residual blocks with channel recalibration.

[0048] The model estimation and validation module is used to predict the interior point weights of all initial correspondences based on the refined features, estimate the final essential matrix using the weighted eight-point algorithm, and perform full-size validation of the initial correspondence point set using the final essential matrix. Reliable interior points are selected based on the symmetric epipolar distance to obtain reliable matching point pairs and an essential matrix describing the geometric relationship between images.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. This invention designs a learnable soft-threshold Transformer that combines spatial attention with learnable soft-threshold channel attention. Internally, LST-CA generates adaptive thresholds for samples based on the current set of corresponding points, suppressing weak channel correlations before attention normalization, reducing the propagation of unreliable responses induced by outliers, and simultaneously modeling long-range spatial and channel dependencies between correspondences to improve the reliability of matching features within a stage, thereby enhancing the ability to distinguish between correct and incorrect matches in image registration.

[0051] 2. This invention designs a prior-guided cross-stage aggregation module. It constructs a correspondence affinity prior using initial stage features, uses the fusion result of the features from the first two stages as the key and value, and uses the final stage features as the query. Before Softmax normalization, the prior, after learnable affine transformation, is added to the attention score to guide the aggregation of early complementary information towards the final representation. The final stage features are preserved through residual connections, allowing complementary information from different stages to be used together for matching reliability judgment, thereby improving interior point selection and essential matrix estimation in image registration.

[0052] 3. The image registration method using a reliability-calibrated attention network proposed in this invention integrates a learnable soft-threshold Transformer and a prior-guided cross-stage aggregation module into a three-level progressive correspondence pruning framework, thereby improving the reliability of intra-stage attention interactions and cross-stage feature fusion. This method takes the initial matching point pairs of two images to be registered as input, predicts the inlier weights of the correspondence, and estimates the essential matrix using a weighted eight-point algorithm. By performing geometric verification on the initial correspondence, erroneous matches are eliminated and reliable inliers are retained, thereby improving the accuracy and robustness of match filtering and geometric relationship estimation in image registration. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the overall architecture of the image registration method for a reliability calibration attention network provided in an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the learnable soft-threshold Transformer and its channel attention structure in an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the priori-guided cross-stage aggregation module structure in an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram showing the visualization comparison results of outlier removal in image registration in an embodiment of the present invention. Detailed Implementation

[0057] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0058] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0059] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0060] I. Overall Architecture of Image Registration Using Reliability Calibration Attention Network

[0061] like Figure 1 As shown, this embodiment provides an image registration method for a reliability calibration attention network. Its overall architecture includes: inputting an initial set of corresponding points, passing through a three-stage progressive pruning structure (Pruning Stage #0, Pruning Stage #1, Pruning Stage #2), each stage including a learnable soft threshold Transformer (LST-former), a local graph module, sequential pooling, etc.; then fusing the features of the three stages through a priori-guided cross-stage aggregation module (PGCA), and then passing through model estimation and full-size verification to output reliable matching point pairs and an essence matrix.

[0062] Given two images to be registered, first use SIFT or SuperPoint to extract feature points and their descriptors from the two images respectively, and then establish an initial set of corresponding points S by nearest neighbor matching of descriptors:

[0063] (1)

[0064] (2)

[0065] Where N represents the number of initial correspondences. Let represent the i-th correspondence. Each correspondence consists of the normalized two-dimensional coordinates of matching feature points in two images to be registered. The network takes this set of correspondence points as input, predicts inlier scores for each matching point pair to distinguish between correct and incorrect matches, and estimates the essential matrix using the retained reliable correspondences.

[0066] Next, to progressively filter reliable matches and improve geometric model estimation, a three-level progressive pruning structure consisting of an initial stage and two refinement stages is adopted. The initial stage directly processes S; the two refinement stages respectively use the original set of corresponding points, the confidence scores predicted in the previous stage, and the geometric feedback concatenated along the feature dimension as input:

[0067] (3)

[0068] (4)

[0069] (5)

[0070] Where t represents the stage number. This represents the interior point score predicted in the previous stage. The confidence level is derived from this score. This is the symmetric epipolar residual calculated based on the essential matrix estimated in the previous stage. Gradient propagation is stopped for both the confidence level and the residual before splicing to prevent the feedback path from altering the previous stage during backpropagation.

[0071] Each stage first projects the input into a C-dimensional feature space (C is 128 in this embodiment). To preserve cross-stage context, the refinement stage combines the projected features of the current stage, the graph representation of the previous stage, and the output of the previous stage to construct a neighborhood graph, which is then aggregated using a multi-graph fusion method; the second refinement stage also introduces the output of the initial stage. Within each stage, PointCN, a Learnable Soft-Threshold Transformer (LST-former), and a local graph module are used to extract correspondence features. The local graph module constructs a dynamic neighborhood, fusing the maximum aggregation and ring aggregation results through local and global gating. Subsequently, sequential perceptual pooling, filtering, and inverse pooling are used to compress and recover the correspondence representation, which is then input into the second LST-former, thereby combining local geometric consistency with long-range spatial and channel interactions.

[0072] For each supervised output, the correspondences are sorted according to the predicted score, and the top half of the correspondences are selected to form the basis of the analysis. The dual-channel predictor outputs the mask score and weight score, and their product is normalized to obtain... Subsequently, the essential matrix is ​​estimated using the weighted eight-point algorithm, and the geometric residuals of each original correspondence are calculated:

[0073] (6)

[0074] Where g represents the weighted eight-point algorithm, and d represents the symmetric epipolar distance. This is the essential matrix estimated for stage t. The above filtering is used for geometric model estimation, and the features of the three stages still retain the correspondence index consistent with the original corresponding point set.

[0075] Finally, the Prior-Guided Cross-Stage Aggregation (PGCA) module is integrated. , and Then, by refining and fusing features through two residual blocks with channel recalibration, the prediction of all correspondences and the estimation of the essential matrix are completed.

[0076] The original corresponding point set is fully validated using the estimated essential matrix. Reliable interior points are selected based on the epipolar distance of the correspondence, thereby obtaining reliable matching point pairs and the essential matrix for image registration.

[0077] II. Learnable Soft Threshold Transformer

[0078] To address the issues in image registration where local neighborhood information is insufficient to fully determine matching reliability and where incorrect matching can easily interfere with feature interaction, this invention designs a learnable soft-threshold Transformer (LST-former), such as... Figure 2 As shown, this module models long-range associations between different matching point pairs through spatial attention and enhances the separability of correct and incorrect matches by suppressing unreliable responses through soft thresholding in channel attention. Two LST-formers are set up for each pruning stage, one before local graph inference and the other after sequence-aware aggregation. Each LST-former contains two consecutive spatial-channel layers, each performing spatial attention and Learnable Soft-Threshold Channel Attention (LST-CA) sequentially, with a feedforward network and residual connections after each attention sub-layer.

[0079] First, let Z denote the normalized correspondence feature of the input LST-CA, with dimensions B×N×C, where B, N, and C represent the batch size, the number of correspondences, and the number of feature channels, respectively. After 1×1 projection, the feature is divided into H attention heads. The query, key, and value features of the h-th head are as follows:

[0080] (7)

[0081] (8)

[0082] The three feature projection functions generate query, key, and value representations, respectively, and `Split` indicates the selection of the h-th attention head. L2 normalization is applied to the query and key along the correspondence dimension, and the channel affinity matrix for each head is calculated.

[0083] (9)

[0084] in, Let h be the learnable temperature parameter for the h-th attention head. This indicates that L2 normalization is performed along the correspondence dimension. The channel affinity matrix satisfies... , This matrix represents the number of channels for each attention head, thus describing the correlation between channels. Subsequently, global average pooling is performed on the input correspondence features, and a threshold and differentiable soft mask related to the current sample are generated by a threshold predictor.

[0085] (10)

[0086] Where ψ represents the multilayer perceptron, τ is the adaptive threshold predicted based on the current corresponding point set, σ represents the Sigmoid function, and α controls the slope of the Sigmoid relaxation. This is the soft mask for the h-th head. By comparing the threshold with the channel affinity score, stronger correlations are given a larger retention weight, while weaker correlations are suppressed.

[0087] After masking the affinity scores, softmax normalized summation feature aggregation is then performed:

[0088] (11)

[0089] (12)

[0090] (13)

[0091] in, The channel affinity score is the result of masking, where κ is a positive masking constant, and ⊙ denotes element-wise multiplication. Cat represents the aggregation result of the h-th head, where Cat indicates that the outputs of each attention head are concatenated. This indicates the output projection function. This represents a learnable soft-threshold channel attention map. The output feature represents the input feature Z after being processed by a learnable soft-threshold channel attention mapping. The output feature is obtained by first concatenating the aggregation results of each attention head, then projecting the output, and finally adding it to the input feature residual. By simultaneously applying soft masking weighting and low-response suppression before Softmax, LST-CA can adjust channel dependence according to different input samples, mitigating the propagation of weak channel responses induced by outliers. Spatial attention is used to preserve the long-range context between different matching point pairs. Both enhance the correspondence features, providing a basis for selecting reliable matching point pairs in image registration.

[0092] III. Prior-guided cross-stage aggregation module

[0093] In the progressive correspondence filtering of image registration, different pruning stages retain matching features with varying degrees of refinement. To fully utilize the complementary information within these features and improve the final interior point discrimination capability, this invention designs a priori-guided cross-stage aggregation module (PGCA), such as... Figure 3 As shown, this module extracts the corresponding affinity prior from the initial stage features and injects the transformed prior into the cross-stage attention score, guiding the information from the early stage to the final stage before normalization.

[0094] First, reshape the output of the three pruning stages into... , and Each of these features has dimensions B×N×C. An affinity prior P is constructed based on the feature correlations between different correspondences in the initial stage, and the features from the first two stages are added together to obtain the early fusion features. :

[0095] (14)

[0096] Where P has a dimension of B×N×N, representing the correlation between correspondences calculated from the initial stage features, and C is the number of feature channels. The final stage features are then used. Generate queries using early fusion features Generate keys and values:

[0097] (15)

[0098] (16)

[0099] Here, Q, K, and V represent query, key, and value, respectively, and A is the unnormalized score of cross-stage attention. All three projection functions are learnable mappings. Since the final stage features have been sufficiently refined, querying earlier stage features with them can extract complementary context that is helpful to the final representation. Before normalization, a learnable affine transformation and LeakyReLU activation are applied to the initial stage affinity prior, and then added to the cross-stage attention score.

[0100] (17)

[0101] (18)

[0102] in, For the transformed prior, and These are the learnable scaling and offset parameters, respectively. This is the prior-guided attention matrix. Softmax normalizes the matrix along the key-to-key correspondence dimension. Then, this attention matrix is ​​used to perform weighted aggregation of the value features, and the projected result is added to the feature residuals of the final stage.

[0103] (19)

[0104] Where ρ represents the output projection. This represents the cross-stage aggregated features output by PGCA. The aggregated results are then processed through two residual blocks with channel recalibration before being used for correspondence prediction. By introducing an affinity prior constructed from the initial stage features before attention normalization, PGCA can guide... and The complementary context is passed to the final representation, while the residual connection is preserved. The refined features in the image are used to enhance the matching features used for interior point discrimination, thus providing support for reliable correspondence screening and essential matrix estimation in image registration.

[0105] IV. Experimental Verification

[0106] To verify the performance of the proposed Reliability-Calibrated Attention Network (RCANet) in image registration for screening reliable matches and estimating geometric relationships, this embodiment compares it with methods such as RANSAC, OANet, MSA-Net, U-Match, ConvMatch, NCMNet, DeMatch, BCLNet, CGR-Net, CSBCNet, MambaMatch, and LeCoT. The experiments use the outdoor scene dataset YFCC100M and the indoor scene dataset SUN3D, establishing initial correspondences using SIFT and SuperPoint, respectively. The experiments evaluate the geometric relationship estimation and matching reliability: relative pose estimation uses the mean precision (mAP) at 5° and 20° error thresholds; outlier removal uses precision (P), recall (R), and F-score, all expressed as percentages.

[0107] Table 1 shows the relative pose estimation results of different methods using SIFT features on the YFCC100M and SUN3D datasets. The evaluation metrics are mAP5° and mAP20°, with the best results indicated in bold.

[0108]

[0109] As shown in Table 1, RCANet achieved the best results in seven out of the eight SIFT pose estimation settings. In the YFCC100M unknown scene, mAP5° and mAP20° reached 74.88% and 88.61%, respectively, representing improvements of 2.05 and 2.34 percentage points compared to MambaMatch's 72.83% and 86.27%. In the SUN3D unknown scene, the two metrics were 23.67% and 49.81%, with mAP5° achieving the highest result, while mAP20° was slightly lower than DeMatch's 49.84%. These results demonstrate that the proposed method possesses good pose estimation capabilities in both known and unknown scenes, providing support for geometric relationship estimation in image registration.

[0110] Given that the geometric relationship estimation in image registration is affected by the initial matching quality, this embodiment further uses SuperPoint to generate the initial correspondence and verifies it on known and unknown outdoor scenes of YFCC100M. The results are shown in Table 2.

[0111] Table 2 shows the relative pose estimation results of different methods using SuperPoint features in outdoor scenes. The evaluation metrics are mAP5° and mAP20°, with the best results indicated in bold.

[0112]

[0113] As shown in Table 2, when using SuperPoint, the proposed method achieves mAP5° and mAP20° accuracy of 54.06% and 72.48% respectively in known scenarios, representing improvements of 3.76 and 2.36 percentage points compared to MambaMatch. In unknown scenarios, these two metrics are 60.88% and 82.99%, improvements of 3.10 and 1.95 percentage points respectively. RCANet achieves the best results in all four settings, indicating that the proposed method can adapt to different correspondence distributions generated by handcrafted and learned features, providing effective matching filtering and geometric relationship estimation for image registration under different feature extraction methods.

[0114] To further verify the ability of this method to retain correct matches and remove incorrect matches in image registration, this embodiment conducted outlier removal experiments on known and unknown scenes in YFCC100M and SUN3D, and used precision, recall and F-score as evaluation metrics.

[0115] Table 3 shows the outlier removal results on the YFCC100M and SUN3D datasets. The evaluation metrics are precision (P), recall (R), and F-score (F). The bolded results represent the best results.

[0116]

[0117] As shown in Table 3, RCANet achieved the highest precision and F-score across all four scene settings. In the YFCC100M known and unknown scenes, the precision was 86.31% and 85.12%, respectively, with F-scores of 84.69% and 82.46%. In the SUN3D known and unknown scenes, the precision was 76.06% and 72.10%, respectively, with F-scores of 75.21% and 71.62%. Compared to the comparative method with the highest F-score in each setting, these represent improvements of 3.07, 3.42, 5.70, and 6.97 percentage points, respectively. Although the recall was not the highest across all settings, the precision and F-score results indicate that this method can more effectively reduce erroneous correspondences while preserving reliable inliers, thus improving the reliability of matching point pairs used for image registration.

[0118] To more intuitively demonstrate RCANet's matching and filtering performance in image registration, this embodiment also includes a visual comparison experiment. For example... Figure 4As shown, from left to right, the results are from OANet, BCLNet, LeCoT, and RCANet. The first two rows represent unknown outdoor scenes from YFCC100M, and the last two rows represent unknown indoor scenes from SUN3D. Green lines indicate correct correspondences, and red lines indicate incorrect correspondences. These examples demonstrate that this method can eliminate many incorrect correspondences while retaining reliable interior points, providing reliable matching point pairs for image registration.

[0119] This embodiment also provides an image registration system for a reliability-calibrated attention network to perform the above-described method, including an initial matching module, a progressive pruning network module, a priori-guided cross-stage aggregation module, and a model estimation and validation module.

[0120] The initial matching module is used to obtain the image pair to be registered, extract the feature points and descriptors of the two images respectively, and establish an initial corresponding point set based on the nearest neighbor matching of the descriptors.

[0121] The progressive pruning network module is used to process the initial corresponding point set using a three-level progressive pruning structure consisting of an initial stage and two refinement stages. The initial stage processes the initial corresponding point set, and the refinement stage concatenates the original corresponding point set, the confidence score predicted in the previous stage, and the geometric feedback along the feature dimension as input. The confidence score is obtained by transforming the intra-point score in the previous stage, and the geometric feedback is calculated based on the essential matrix estimated in the previous stage. Gradient propagation of the confidence score and the geometric feedback is stopped before concatenation. Each stage uses a learnable soft-thresholding Transformer to extract intra-stage features. The learnable soft-thresholding Transformer includes spatial attention and learnable soft-thresholding channel attention. The learnable soft-thresholding channel attention adaptively generates a threshold based on the current input corresponding point set and suppresses weak channel correlations before attention normalization.

[0122] The prior-guided cross-stage aggregation module is used to fuse the stage features of three stages: constructing a correspondence affinity prior based on the initial stage features, fusing the features of the first two stages to obtain early fusion features, generating queries with the final stage features, generating keys and values ​​with the early fusion features, and calculating cross-stage attention; injecting the correspondence affinity prior into the cross-stage attention after a learnable transformation, performing weighted aggregation on the value features after normalization, projecting the aggregation result and adding it to the residual of the final stage features to obtain the cross-stage aggregated features; the cross-stage aggregated features are refined by residual blocks with channel recalibration.

[0123] The model estimation and verification module is used to predict the interior point weights of all initial correspondences based on the refined features, estimate the final essential matrix using the weighted eight-point algorithm, and use the final essential matrix to perform full-size verification on the initial correspondence point set. Reliable interior points are selected based on the symmetric epipolar distance to obtain reliable matching point pairs and an essential matrix describing the geometric relationship between images.

[0124] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0125] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0126] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An image registration method for reliability-calibrated attention networks, characterized in that, Includes the following steps: S1. Obtain an image to be registered, extract feature points and descriptors from the two images respectively, and establish an initial set of corresponding points based on the nearest neighbor matching of the descriptors; S2. Input the initial corresponding point set into the reliability calibration attention network. The reliability calibration attention network adopts a three-level progressive pruning structure consisting of an initial stage and two refinement stages. The initial stage processes the initial corresponding point set. The refinement stage concatenates the original corresponding point set, the confidence score predicted in the previous stage, and the geometric feedback along the feature dimension as input. The confidence score is obtained by transforming the in-point score in the previous stage. The geometric feedback is calculated based on the essential matrix estimated in the previous stage. Gradient propagation of the confidence score and the geometric feedback is stopped before concatenation. S3. At each stage, the input is projected onto the feature space, and a learnable soft threshold Transformer is used to extract features within the stage to obtain stage features. The learnable soft-threshold Transformer includes spatial attention and learnable soft-threshold channel attention. The learnable soft-threshold channel attention adaptively generates a threshold based on the current input corresponding point set and suppresses weak channel correlations before attention normalization in order to model the long-range spatial and channel dependencies between correspondences. S4. For each stage, the corresponding relationships are filtered based on the predicted interior point scores, the essential matrix of the stage is estimated using the weighted eight-point algorithm, and the geometric residual of the original correspondence is calculated. S5. The cross-stage aggregation module guided by priors fuses the stage features of the three stages: a correspondence affinity prior is constructed based on the initial stage features, the features of the first two stages are fused to obtain early fusion features, a query is generated with the final stage features, and a key and value are generated with the early fusion features, and cross-stage attention is calculated; the correspondence affinity prior is injected into the cross-stage attention after a learnable transformation, and the value features are weighted and aggregated after normalization, and the aggregation result is projected and added to the residual of the final stage features to obtain the cross-stage aggregated features; The cross-stage aggregation features are refined by residual blocks with channel recalibration; S6. Based on the refined features, predict the interior point weights for all initial correspondences, estimate the final essential matrix using the weighted eight-point algorithm, and use the final essential matrix to perform full-size verification on the initial correspondence point set. Filter reliable interior points according to the symmetric epipolar distance to obtain reliable matching point pairs and the essential matrix describing the geometric relationship between images.

2. The image registration method for reliability calibration attention network according to claim 1, characterized in that, In step S1, for two images in a set of images to be registered, SIFT or SuperPoint is used to extract feature points and their descriptors from the two images respectively, and then an initial set of corresponding points S is established by nearest neighbor matching of the descriptors: Where N represents the number of initial correspondences. This represents the i-th correspondence, where each correspondence consists of the normalized two-dimensional coordinates of matching feature points in two images to be registered.

3. The image registration method for reliability calibration attention network according to claim 1, characterized in that, In step S2, the input for the refinement stage is: Where t=1,2, , The interior point score predicted in the previous stage. The symmetric polar residuals are calculated based on the essential matrix estimated in the previous stage. and Stop gradient propagation before splicing.

4. The image registration method for reliability calibration attention network according to claim 1, characterized in that, In step S2, after each stage projects the input into the feature space, the refinement stage combines the projected features of the current stage, the graph representation of the previous stage, and the output of the previous stage to construct a neighborhood graph, and aggregates them in a multi-graph fusion manner. The second refinement stage also incorporates the output from the initial stage; within the stage, point convolutional networks, learnable soft-thresholding Transformers, and local graph modules are combined to extract corresponding features; The local graph module constructs a dynamic neighborhood and fuses the maximum aggregation and ring aggregation results through local and global gating. Then, it compresses and restores the corresponding relationship representation through sequential perceptual pooling, filtering and inverse pooling, and then inputs another learnable soft threshold Transformer to combine local geometric consistency with long-range spatial and channel interactions.

5. The image registration method for reliability calibration attention network according to claim 1, characterized in that, In step S3, the learnable soft-threshold Transformer contains two consecutive spatial-channel layers. Each layer sequentially performs spatial attention and learnable soft-threshold channel attention, and a feedforward network and residual connections are set after each attention sub-layer. The spatial attention is configured to: calculate the similarity between the query and the key along the correspondence dimension, and then perform weighted aggregation of the value features after normalization to model the long-range spatial association between different matching point pairs; The learnable soft-threshold channel attention is configured as follows: Input features Z are acquired, with dimensions B×N×C, where B, N, and C represent the batch size, the number of correspondences, and the number of feature channels, respectively; after 1×1 projection, the features are divided into H attention heads, where the query, key, and value features of the h-th head are respectively... , , Perform L2 normalization on the query and key along the correspondence dimension, and calculate the channel affinity matrix of the h-th head: in, Let h be the learnable temperature parameter for the h-th attention head; This indicates that L2 normalization is performed along the correspondence dimension; Global average pooling is performed on the input correspondence features, and a threshold τ and a differentiable soft mask related to the current sample are generated by a threshold predictor. : Where ψ is the threshold predictor, which uses a multilayer perceptron; σ represents the sigmoid function, and α controls the slope of the sigmoid relaxation. The soft mask for the h-th header; The channel affinity scores are masked and modulated, and then Softmax normalized sum feature aggregation is performed. in, The channel affinity score is the result of masking, where κ is a positive masking constant, and ⊙ denotes element-wise multiplication. Cat represents the aggregation result of the h-th head, where Cat indicates that the outputs of each attention head are concatenated. This indicates the output projection function. This represents a learnable soft-threshold channel attention map. This represents the output feature after the input feature Z has been processed by a learnable soft-threshold channel attention mapping. The corresponding output feature is obtained by first concatenating the aggregation results of each attention head, then projecting the output, and finally adding it to the input feature residual. In the learnable soft-threshold channel attention, the comparison between the threshold and the channel affinity score gives stronger correlations a larger retention weight, while weaker correlations are suppressed; the soft mask modulates the channel affinity score before attention normalization.

6. The image registration method for reliability calibration attention network according to claim 1, characterized in that, In step S4, the stage features of each stage retain the correspondence index consistent with the original corresponding point set; for each supervised output, the correspondences are sorted according to the prediction score, and the top half of the correspondences are selected to form the system. The dual-channel prediction head outputs a mask score and a weight score. The product of the mask score and the weight score is normalized to obtain the weight. The essential matrix is ​​estimated using the weighted eight-point algorithm. And calculate the geometric residuals of each original correspondence. , where g represents the weighted eight-point algorithm and d represents the symmetric epipolar distance.

7. The image registration method for reliability calibration attention network according to claim 1, characterized in that, In step S5, the prior-guided cross-stage aggregation module, based on the initial stage characteristics... Constructing correspondence affinity priors The features from the first two stages are then added together to obtain the early fusion features. Where C is the number of feature channels; Characteristics of the final stage Generate query Q to fuse features early. Generate key K and value V, and calculate cross-stage attention score. ; Perform a learnable affine transformation and Leaky ReLU activation on the prior P to obtain the transformed prior: in, and These are the learnable scaling and offset parameters, respectively; Will Incorporating cross-stage attention scores to obtain a prior-guided attention matrix Then, the aggregated feature values ​​are obtained as follows: in, Indicates output projection. This is a cross-stage aggregation feature.

8. The image registration method for reliability calibration attention network according to claim 1, characterized in that, In step S5, the cross-stage aggregated features are refined by two residual blocks with channel recalibration; each residual block with channel recalibration is configured to recalibrate the input features in the channel dimension and add the recalibrated features to the input feature residuals.

9. The image registration method for reliability calibration attention network according to claim 1, characterized in that, In step S6, the full-size verification includes: calculating the symmetric epipolar distance of each original correspondence using the final estimated essential matrix, determining the correspondences whose symmetric epipolar distance satisfies the inlier threshold as reliable inliers, and retaining the corresponding matching point pairs and the final essential matrix.

10. An image registration system for reliably calibrating an attention network, characterized in that, For performing the method as described in any one of claims 1-9, comprising: The initial matching module is used to obtain the image pair to be registered, extract the feature points and descriptors of the two images respectively, and establish an initial corresponding point set based on the nearest neighbor matching of the descriptors; A progressive pruning network module is used to process the initial corresponding point set using a three-level progressive pruning structure consisting of an initial stage and two refinement stages. The initial stage processes the initial corresponding point set, and the refinement stage concatenates the original corresponding point set, the confidence score predicted in the previous stage, and the geometric feedback along the feature dimension as input. The confidence score is obtained by transforming the intra-point scores in the previous stage, and the geometric feedback is calculated based on the essential matrix estimated in the previous stage. Gradient propagation of the confidence score and the geometric feedback is stopped before concatenation. Each stage uses a learnable soft-thresholding Transformer for intra-stage feature extraction. The learnable soft-thresholding Transformer includes spatial attention and learnable soft-thresholding channel attention. The learnable soft-thresholding channel attention adaptively generates a threshold based on the current input corresponding point set and suppresses weak channel correlations before attention normalization. A priori-guided cross-stage aggregation module is used to fuse stage features from three stages: a correspondence affinity prior is constructed based on initial stage features; the features of the first two stages are fused to obtain early fusion features; queries are generated using the final stage features, and keys and values ​​are generated using the early fusion features; cross-stage attention is calculated; the correspondence affinity prior is injected into the cross-stage attention after a learnable transformation; after normalization, the value features are weighted and aggregated; the aggregation result is projected and added to the residual of the final stage features to obtain the cross-stage aggregated features; the cross-stage aggregated features are refined by residual blocks with channel recalibration. The model estimation and validation module is used to predict the interior point weights of all initial correspondences based on the refined features, estimate the final essential matrix using the weighted eight-point algorithm, and perform full-size validation of the initial correspondence point set using the final essential matrix. Reliable interior points are selected based on the symmetric epipolar distance to obtain reliable matching point pairs and an essential matrix describing the geometric relationship between images.

Images