Resolution-adaptive end-to-end sub-pixel-level feature matching method

Abstract

The invention provides a resolution-adaptive end-to-end sub-pixel-level feature matching method. The method comprises the following steps: step 1, carrying out feature sub-joint detection and description based on single-head coupling output; step 2, carrying out improved sub-pixel-level key point positioning based on a micro-key point detection algorithm; step 3, executing an adaptive mechanism for different resolutions or texture densities; and step 4, executing end-to-end gradient closed loop based on back propagation. According to the method, an effective solution is provided for the core problems that a traditional feature matching method is insufficient in sub-pixel-level positioning precision, detection and description modules are split, scene adaptability is poor and the like, the key point positioning error is remarkably reduced to the sub-pixel level from the pixel level, and the geometric precision of three-dimensional reconstruction is remarkably improved. A single-head coupling output structure realizes collaborative optimization of detection and description tasks, feature expression is more discriminative through a shared gradient, and parameter redundancy is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional reconstruction, and in particular relates to a resolution-adaptive end-to-end sub-pixel feature matching method. Background Art

[0002] 3D reconstruction is a core task in computer vision, aiming to recover the 3D structure of a scene from multi-view images. It is widely used in fields such as autonomous driving, AR / VR, and digital twins. As the cornerstone of 3D reconstruction, feature matching plays a key role in ensuring data association, reconstruction accuracy, and efficiency. In recent years, feature matching algorithms have become a key path to improving the accuracy and efficiency of 3D reconstruction, driving new demands for high precision, robustness, and versatility.

[0003] The current mainstream approaches to feature matching include those based on manual design and deep learning. Hand-crafted feature matching methods perform well in terms of computational efficiency and interpretability, but they are less robust to complex scenarios such as perspective changes, illumination changes, and texture loss, and rely on a large number of manual parameter adjustments and heuristic rules, which limits their performance ceiling. For example, methods such as SIFT, SURF, and ORB extract local feature points from images and calculate their descriptors, then use nearest neighbor search or ratio testing for matching. In contrast, deep learning-based methods significantly improve the robustness and accuracy of matching by automatically learning feature representations and matching strategies, forming mainstream frameworks such as graph matching frameworks, self-supervised or unsupervised learning frameworks, detection-description joint learning frameworks, and matching optimization frameworks. However, deep learning methods usually require a large amount of labeled data and high computing resources, and the model complexity is high, which limits their application performance.

[0004] The Superglue feature matching method (reference: Sarlin, PE, DeTone, D., Malisiewicz, T., & Rabinovich A. Superglue: Learning feature matching with graph neural networks [C]. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2020: 4938-4947.) is a deep learning feature matching method based on graph neural networks (GNNs) and an attention mechanism for optimized matching. Its core process involves inputting feature points and descriptors from two images, modeling global contextual relationships using self-attention and cross-attention, then normalizing the matching score matrix using the Sinkhorn algorithm to generate optimal matching pairs. False matches are then eliminated by combining geometric validation techniques such as RANSAC. This mechanism enables excellent performance in scenes with occlusion, viewpoint changes, and low texture, and has been widely used in visual localization, structure from motion (SfM), and augmented reality (AR). However, this method still has three major limitations: 1) It inherits the computational complexity of the earlier Transformer, with its computational cost growing quadratically with the number of keypoints; 2) it is highly dependent on labeled data, making training difficult; and 3) detector quality directly impacts matching performance. Matching accuracy is limited to the pixel level, making it inadequate for complex scenes, and generalization stability needs improvement. Therefore, optimizing the Superglue feature matching method to address these issues while maintaining its advantages has become a major research challenge.

[0005] LightGlue (Lindenberger, P. -E. Sarlin, M. Pollefeys, LightGlue: Local Feature Matching at Light Speed[C], 2023 IEEE / CVF International Conference on Computer Vision (ICCV), 2023: 17581-17592) and LoFTR (Sun, Z. Shen, Y. Wang, H. Bao and X. Zhou, LoFTR: Detector-Free Local Feature Matching with Transformers[C], 2021 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 8918-8927) further explore reducing computational complexity and improving the stability of feature matching in low-texture scenes, respectively. LightGlue significantly improves computational efficiency while maintaining high matching accuracy by introducing adaptive depth and width mechanisms. Its adaptive nature enables faster inference when processing simple image pairs. However, LightGlue is very sensitive to detector quality. If the front-end feature point detection is unstable, it may lead to excessive pruning and loss of valid matches. In addition, threshold tuning is difficult in extremely dynamic scenes. LoFTR adopts a detector-free design. Through the global receptive field and position encoding of the Transformer, combined with a multi-scale feature pyramid and channel attention mechanism, it can generate dense matches in low-texture areas, solving the problem that traditional detectors fail in repetitive patterns or weak-texture areas. However, although LoFTR performs better than SuperGlue in matching low-texture areas, its computational complexity is high, requiring a sophisticated iterative correction strategy, and its real-time performance is low. Summary of the Invention

[0006] Purpose of the invention: The problems of insufficient sub-pixel positioning accuracy, separation of detection and description modules, and poor resolution adaptability in feature matching are the core bottlenecks that restrict the improvement of feature matching performance and are also the technical difficulties that the present invention focuses on overcoming. These problems not only reduce the accuracy and robustness of feature matching, but also limit the practical application effect of the algorithm in complex scenarios. The present invention specifically provides a resolution-adaptive end-to-end sub-pixel feature matching method, comprising the following steps: Step 1: Joint detection and description of feature sub-units based on single-head coupling output; Step 2: Perform sub-pixel key point positioning based on an improved differentiable key point detection algorithm; Step 3, performing an adaptive mechanism for different resolutions or texture densities; Step 4: Perform end-to-end gradient closure based on back-propagation.

[0007] In step 1, a single-head coupled output structure is adopted to synchronously generate key point confidence and descriptors through a single set of convolution parameters, and a gradient coupling mechanism is designed to enable the detection and description tasks to share homologous features and be optimized collaboratively.

[0008] Step 1 includes: First, a lightweight convolutional neural network (CNN) is used to generate a feature map F and a keypoint score map S. The convolutional neural network (CNN) consists of four depthwise separable convolutional blocks with a maximum receptive field of 204×204. Use s to represent the confidence of the key point at each pixel position after activation by the Sigmoid function. It is regarded as a probability value during calculation and s is used as the key point score. Input feature map Shared convolution kernel Processing, there are D+1 channels in total, the output channel tensor Z has a total of D+1, coordinates The mapping of the channel tensor Z at is defined as: (1), in is convolution, is the weight matrix of the convolution kernel, b is the bias term, and its value is determined by back propagation during training. and b form a shared convolution kernel The set of trainable parameters.

[0009] In step 1, record the coordinates At , the normalized unit descriptor vector for: (2), in, Represents the first D channels of the output tensor Z at coordinate (x, y), and the length of the vector is calculated by the L2 norm (Euclidean norm); The dimension is D×1.

[0010] In step 1, record the coordinates Key point score for: (3), in Represents the D+1th channel of the feature tensor Z; is the Sigmoid function; The gradients of the descriptor loss and the detection loss are coupled to the shared convolution kernel through the chain rule. The same set of trainable parameters for adjusting and b, using Represents the gradient coupling formula: (4), in represents partial differential; Represents the gradient of the descriptor loss with respect to the descriptor d; represents the gradient of the detection loss with respect to the score s; Indicates the response of the descriptor normalization operation to the original descriptor gradient; Represents the Sigmoid activation response to the original score gradient; Represents the convolutional layer output Z pair parameter set gradient; Key point score With the descriptor vector Originated from shared convolution kernel , while accepting the gradients from the descriptor loss and the keypoint loss.

[0011] Step 2 includes: Step 2-1, preliminary detection of whole pixel key points: Use the non-maximum suppression strategy NMS, set the global threshold τ, the image coordinate system takes the upper left corner as the origin (0,0), u represents the horizontal coordinate of the pixel in the image, and v represents the vertical coordinate of the pixel in the image. The non-maximum suppression strategy searches for pixel coordinates (u, v) that meet the following two conditions on the entire score map: Score condition: the key point score s(i,j) of coordinate (i,j) is greater than the preset global threshold τ; Local maximum condition: In a local neighborhood N with a radius of 2r+1 and a pixel coordinate (u, v) as the center, the key point score s(i, j) of the coordinate (i, j) is the maximum value of the key point scores of all neighboring pixels: The conditional formula is: , (5), Where s(u,v) represents the keypoint score of pixel coordinate (u,v), N represents the local neighborhood centered at pixel coordinate (u,v), and is a window of size (2r+1)×(2r+1). The global threshold τ is a configurable hyperparameter whose value is determined by cross-validation based on the training dataset and ranges from [0.3, 0.7]. max represents the maximum value function. The non-maximum suppression strategy only retains the pixels with the maximum score, and treats the filtered pixel coordinates (u, v) and the key point scores s(u, v) corresponding to the coordinates as constants in the back propagation; Step 2-2, the differentiable key point detection strategy DKD implements sub-pixel fine-tuning: For the key point score map S obtained by the lightweight convolutional neural network CNN, As the center, a local window P of size N×N is cut on the key point score map S, and the probability weighted average soft-argmax function is applied to all scores in the local window P to transform the key point score map S into a spatial probability distribution , at coordinates The spatial probability distribution value at The calculation is as follows: (6), in is the maximum score in the window, is the temperature coefficient, exp represents the natural exponential function; Indicates that within the local window P, the coordinates The key point score at express The key point scores at other coordinates (p,q) in the same local window P; Then use the coordinate The spatial probability distribution value at As weights, calculate the expected coordinates of the key points in the window and the sub-pixel position p: (7), Finally, the comprehensive center coordinates and sub-pixel positions , the expected coordinates Mapping back to the entire image coordinate system, the final sub-pixel key point coordinates for: (8), in It is the coordinate of the center of the local window in its own coordinate system.

[0012] Step 3 includes: Define discrete candidate sets , represents the predefined optional value of the window size N; for the k-th candidate window currently being calculated , logarithmic weights can be learned by gradient descent update during training , and use Gumbel-Softmax to make the discrete sampling distribution continuous (Gumbel-Softmax, Chinese translation: Gumbel-Softmax, is a discrete variable modeling technology used in deep learning, which achieves differentiable approximate sampling by introducing Gumbel noise while retaining gradient information), and adjust the temperature parameter Control sampling and select weights in Gumbel distribution (extreme value distribution) and sampled random noise as follows: (9), in Indicates that when the summation term in the denominator traverses all candidate windows, the jth candidate window The corresponding learnable logarithmic weights, Indicates that when the summation term in the denominator traverses all candidate windows, the jth candidate window Random noise sampled using Gumbel distribution; For each Calculate pixel-level soft coordinates Finally, the pixel-level soft coordinates are weighted and fused to update the sub-pixel position p to achieve differentiable selection: (10), (11), Let the probability weighted average Soft-argmax temperature coefficient ,in, Used to control the concentration of coordinate probability distribution, It is a learnable parameter with an initial value of 0; the softplus function is a flexible plus activation function. Will Map to , so that the temperature parameter Always positive.

[0013] The neural network of the entire resolution adaptive end-to-end sub-pixel feature matching model is learned through training , automatically adjust To adapt to the texture complexity of different image regions.

[0014] Step 4 includes: Through end-to-end joint detection and description, each position i corresponds to a candidate coordinate , mark the sub-pixel key point coordinates corresponding to position i as , the corresponding key point score Redefine it through the Sigmoid function, denoted as , indicating that the location The probability of the existence of a true key point is used for Bernoulli loss calculation, and the calculation formula is: (12), in, Indicates that the sub-pixel key point coordinates The integer pixel coordinates obtained after rounding.

[0015] Design a supervisory signal based on Bernoulli distribution. For position i, if position i falls within the geometric threshold neighborhood of a true key point In the label , can be re-invested; otherwise, let the label 0; Confidence loss is calculated using Bernoulli negative log-likelihood , during the model training phase, enforce continuous confidence Fitting discrete labels : (13), To achieve collaborative optimization of detection and description tasks, a joint loss function is constructed : (14), in, Represents the reprojection loss The weight coefficient of Indicates the dispersion peak loss The weight coefficient of Denotes the descriptor loss The weight coefficient of Reliability loss The weight coefficient of Represents confidence loss The weight coefficient of Formulas (4), (9) to (14) are combined to form a differentiable computational graph. The gradient is fully refluxed through soft-argmax and Bernoulli, achieving integrated learning of detection and description: (15), in, represents the gradient returned by the descriptor d, Represents the gradient of loss L with respect to descriptor d; Describes the descriptor d pair gradient; represents the gradient of the key point score s backpropagation, represents the gradient of loss L with respect to key point score s, Represents the key point score s pair The gradient, represents the sum of the gradients over all spatial coordinates.

[0016] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the method.

[0017] The present invention also provides a storage medium storing a computer program or instruction, which executes the steps of the method when the computer program or instruction is run on a computer.

[0018] Beneficial effects: The present invention provides an effective solution to the core problems of traditional feature matching methods, such as insufficient sub-pixel positioning accuracy, separation of detection and description modules, and poor scene adaptability. Through the differentiable sub-pixel positioning mechanism combined with dynamic window selection, the model significantly reduces the key point positioning error from the pixel level to the sub-pixel level, significantly improving the geometric accuracy of three-dimensional reconstruction. The single-head coupled output structure realizes the collaborative optimization of detection and description tasks, makes the feature expression more discriminative by sharing gradients, and reduces parameter redundancy. The adaptive mechanism enables the model to intelligently adapt to input scenes of different resolutions and texture densities through learnable window size and temperature parameters, and can still maintain stable matching performance in low-texture areas. The final output joint features have sub-pixel positioning accuracy, highly discriminative descriptors and calibrated reliable confidence indicators, providing a higher-quality data foundation for tasks such as three-dimensional reconstruction and visual positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the overall flow chart of the present invention.

[0020] Figure 2 This is the effect diagram of the reconstructed model.

[0021] Figure 3 Detail comparison of the reconstructed model. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0023] like Figure 1 As shown, an embodiment of the present invention provides a resolution-adaptive end-to-end sub-pixel feature matching method, comprising the following steps: Step 1: Joint detection and description of feature sub-elements based on single-head coupling output; In traditional key point detection and descriptor extraction tasks, a dual-branch structure (detection and description are separated) is often used, resulting in inconsistent characteristics of the two problems. The detection branch focuses on position saliency, while the description branch focuses on matching discriminability. There is an inherent conflict in the optimization objectives of the two, and the gradient is split into different parameter sets during backpropagation, which weakens the model convergence efficiency and feature alignment capabilities. To overcome the above-mentioned shortcomings, the present invention proposes a single-head coupled output structure. Its core idea is to synchronously generate key point confidence and descriptors through a single set of convolution parameters, and design a gradient coupling mechanism so that the detection and description tasks share homologous features and are optimized collaboratively.

[0024] First, a lightweight convolutional neural network (CNN) is used to generate a downsampled feature map F and a keypoint score map S. This network consists of four depthwise separable convolutional blocks with a maximum receptive field of 204×204. The score s of each pixel coordinate is activated by a sigmoid function. The activated score s represents the confidence level of the keypoint at that coordinate. Input feature map Shared convolution kernel Processing, output (D+1) channel tensor. The mapping is defined as: (1), in is convolution; Let the D channel be: (2), The D+1th channel is recorded as: (3), The gradients of the descriptor loss and the detection loss are coupled to the same parameter set through the chain rule. : (4), Detection score and descriptors From the same convolution kernel , while simultaneously accepting the gradients from both the descriptor loss and the keypoint loss. This allows a single image to simultaneously produce both the descriptor graph for matching and the score graph for detection. This avoids the gradient splitting and potential direction conflicts inherent in the "two-headed" model, while maintaining semantic consistency between detection and description. This allows the model to share features and gradients, reducing overall parameters and making end-to-end training more stable.

[0025] Step 2: Sub-pixel keypoint localization based on improved differentiable keypoint detection algorithm; The core goal of the Differentiable Keypoint Detection (DKD) algorithm is to achieve sub-pixel accuracy in keypoint localization. This process is typically divided into two stages: initial screening of whole-pixel locations and local sub-pixel fine-tuning.

[0026] (1) Preliminary detection of whole pixel key points; To identify significant and unique candidate points, the algorithm uses the non-maximum suppression (NMS) strategy. The idea behind NMS is to select the best candidate box most likely to represent a true object from a large number of overlapping candidate boxes, while removing all other redundant, highly overlapping candidate boxes. The screening steps and conditions are as follows: A global threshold τ (valued at 0.2) is set. NMS searches the entire score map for pixel coordinates (u, v) that simultaneously meet the following two conditions: Score condition: the key point score s(i,j) of coordinate (i,j) is greater than the preset global threshold τ; Local maximum condition: In a local neighborhood N with a radius of 2r+1 and a pixel coordinate (u, v) as the center, the key point score s(i, j) of the coordinate (i, j) is the maximum value of the key point scores of all neighboring pixels: The conditional formula is: , (5), Where s(u,v) represents the keypoint score of pixel coordinate (u,v), N represents the local neighborhood centered at pixel coordinate (u,v), and is a window of size (2r+1)×(2r+1). The global threshold τ is a configurable hyperparameter whose value is determined by cross-validation based on the training dataset and ranges from [0.3, 0.7]. max represents the maximum value function. The non-maximum suppression strategy only retains the pixels with the maximum score, and treats the filtered pixel coordinates (u, v) and the key point scores s(u, v) corresponding to the coordinates as constants in the back propagation;

[0027] (2) Local soft-argmax to achieve sub-pixel fine-tuning; In order to obtain higher geometric accuracy, DKD targets the score map S with Centered on the original score map S, a local window P of size N×N is cut, and the soft-argmax function is applied to all scores within the window P to convert it into a spatial probability distribution: (6), Then use the generated probability distribution as the weight to calculate the expected coordinates of the key points in the window : (7), Finally, the calculated local expected coordinates Map back to the entire image coordinate system. is the offset relative to the local window center (u, v) (with the window center as the origin), and the final sub-pixel key point coordinates for: (8), in are the coordinates of the center of the local window in its own coordinate system.

[0028] Sub-pixel keypoint localization, based on an improved differentiable keypoint detection algorithm, combines discrete NMS screening with continuous soft-argmax regression. NMS efficiently provides reliable whole-pixel candidate points, while the learnable local soft-argmax algorithm uses the distribution of scores within a small neighborhood of these candidate points to make fine adjustments and calculate sub-pixel coordinates with decimals. This significantly improves the geometric accuracy of keypoint localization while maintaining the model's end-to-end trainability.

[0029] Step 3: Adaptive mechanism for different resolutions or texture densities; In traditional sub-pixel positioning, the fixed window size N and the Softmax function are difficult to adapt to scenes with different resolutions or texture densities. Therefore, the present invention proposes an adaptive mechanism for different resolutions or texture densities to achieve parameter adaptive optimization.

[0030] (1) Learnable window N and temperature parameters ; The fixed windows used in traditional methods often cannot adapt to changes in target scale. For example, densely textured scenes require smaller windows to avoid interference from neighboring points, while sparse textures in aerial photography require larger windows to capture information. Manually switching windows is labor-intensive, while pyramid multi-scale methods are computationally intensive. The learnable window proposed in this paper achieves soft switching of different sizes through Gumbel-Softmax and can backpropagate gradients, addressing two key issues in traditional methods: poor generalization and insufficient scene adaptability caused by fixed hyperparameters.

[0031] Define discrete candidate sets , representing the predefined optional value of the window size N, for each Maintaining learnable logarithmic weights , and use Gumbel-Softmax to make the discrete sampling distribution continuous, and use Control sampling: (9), For each Calculate soft coordinates (soft-argmax of formula (2)), the final coordinates are weighted fused to achieve differentiable selection: (10), (11); Let the temperature of Soft-argmax be , Controls the concentration of the coordinate probability distribution. When at high temperature ( When it is much larger than 1), the coordinate probability distribution is flat, the coordinates are smooth but easy to deviate from the true peak; when it is at low temperature ( When is much smaller than 1, the coordinate probability distribution becomes sharp, sensitive to noise, and has unstable gradients. The learnable mechanism transforms parameter selection into an optimization problem, allowing the network to autonomously find the optimal balance. Can be learned and Epoch thaw. Softplus guaranteed And the gradient is smooth. Faced with feature point sets with different resolutions or texture densities, the model can automatically select the most appropriate window size N and soft-argmax temperature , and remain divisible.

[0032] Step 4: End-to-end gradient closure based on back-propagation; In the task of keypoint detection in computer vision, traditional methods often couple confidence scores with feature descriptor training, making it difficult for confidence scores to accurately reflect the actual reliability of keypoints. To address this issue, this paper proposes an innovative confidence normalization mechanism. Its core goal is to model whether a location is a true keypoint as an explicit Bernoulli event, enabling the network to distinguish between "reliable keypoints" and "pseudo-keypoints" (such as texture noise or edge responses). Confidence only affects the weighted detection loss and threshold selection during inference, and no longer directly affects the descriptor learning process.

[0033] The network outputs a score at each candidate position . Redefine the mathematical meaning of this fraction, The Bernoulli success probability interpreted as "there is a true keypoint at this position": (12), By constraining its value range to the probability space, the numerical normalization of the confidence is achieved, which can be directly used for probability comparison and threshold decision making, significantly improving the accuracy of the confidence. To train this probability representation, the present invention designs a supervisory signal based on the Bernoulli distribution. For position i, if it falls within the geometric threshold neighborhood of a certain true value key point, In the label , can be re-invested; otherwise, let the label 0. Calculated using Bernoulli negative log-likelihood: (13), The network output score It is trained as the Bernoulli success probability of "this point is a true key point", thereby learning a confidence indicator with calibrated probability meaning, numerical normalization and clear decision boundaries, which is used to determine the detection loss weight and screening during inference without affecting the descriptor itself.

[0034] In order to achieve collaborative optimization of detection and description tasks, a joint loss function is constructed: (14), Default weight .

[0035] The submodules of formulas (4), (9) to (14) are combined to form a differentiable computational graph. The gradient is fully refluxed through soft-argmax and Bernoulli, realizing the integrated learning of detection and description: (15).

[0036] In one embodiment of the present invention, the method comprises the following steps: Step 1: Data preprocessing based on multi-scale feature extraction and enhancement; Select an open source dataset and input image pairs and First, Z-Score normalization and downsampling methods are used for preprocessing to generate a multi-scale image pyramid , where S represents the scale index. The purpose of this step is to provide a basis for subsequent multi-scale feature extraction. Compared with traditional single-scale input methods, the present invention is more robust to scale changes.

[0037] Step 2: Joint detection and description of feature sub-elements based on single-head coupling output; First, a lightweight convolutional neural network (CNN) is used to generate a downsampled feature map F and a keypoint score map S. This network consists of four depthwise separable convolutional blocks with a maximum receptive field of 204×204. The feature map is fed into a single convolutional head, where features are extracted synchronously using shared weights. The first D channels of the convolutional head output are L2-normalized to generate a descriptor map of size H×W×D. The D-dimensional vector at each spatial location in this descriptor represents the features of the local region. The D+1th channel of the convolutional head output is sigmoid-activated to generate a score map S of size H×W×1. Each pixel value in this descriptor represents the Bernoulli probability of a keypoint at that location, which is used for localization screening and loss weighting. This design avoids the parameter redundancy of traditional two-branch architectures (independent detection and description) and forces the two tasks to share the same underlying features.

[0038] Step 3: sub-pixel key point detection, extraction and adaptive optimization; Perform non-maximum suppression (NMS) on the score map S, set the geometric neighborhood radius r and the global threshold. Only retain the pixels that meet both the confidence condition (s>τ) and the local maximum condition (maximum value in the (2r+1)×(2r+1) window), and obtain the integer pixel coordinates as the candidate key point set {(u,v)}. With each candidate point (u,v) as the center, cut an adaptive window of size N×N (N∈{3,5,7}) on the score map S, and dynamically select the window size through learnable weights. Apply the soft-argmax algorithm in the window to convert the score in the window into a spatial probability distribution, and use the high temperature parameter t det Control the smoothness of the distribution, calculate the probability-weighted expected coordinates, and obtain the sub-pixel offset. Superimpose the offset to the integer pixel coordinates and output the final sub-pixel coordinate p. For the descriptor d, perform bilinear interpolation based on the sub-pixel coordinate p on the descriptor graph D, extract the D-dimensional vector and perform L2 normalization to ensure feature scale consistency. For the confidence s i , directly obtain the original score value at position p from the score map S as the key point reliability indicator, and supervise its probability calibration through Bernoulli loss.

[0039] Step 4, feature matching; Feature matching utilizes Superglue's matching framework. Keypoints p and their descriptors d from two images are received and encoded into high-dimensional vectors, while the keypoints' positions are incorporated to enhance spatial perception. Next, a graph structure is constructed containing two types of edges: self-edges between keypoints within the same image, which aggregate local contextual information; and cross-edges between keypoints from different images, which establish cross-image associations. By alternating self-attention and cross-attention mechanisms, the graph neural network progressively updates the feature representation of each keypoint, autonomously optimizing it through a backpropagation loss function, ultimately outputting filtered, high-precision matching pairs.

[0040] like Figure 3 As shown in Figure 2, sub-pixel key points are densely and accurately distributed in texture-rich areas (such as edges and corners), while key points in weak texture areas (such as walls) are significantly reduced but reasonably distributed. The average positioning error of the sub-pixel coordinate p is ≤ 0.3 pixels (such as Figure 2 As shown, compared to existing technologies (such as SuperPoint+superglue, which only studies whole-pixel positioning errors), this method achieves more accurate edge alignment and outperforms existing technologies on open-source datasets, achieving a precision of 88.51%, a recall of 83.39%, and an F1 score of 85.82% on the open-source dataset Hpatches. Compared to existing technologies (such as SuperPoint+superglue), this method achieves high keypoint accuracy and greater robustness to scale variations through multi-scale feature enhancement, joint detection-description optimization, adaptive sub-pixel positioning, and graph matching fusion. It achieves a high and uniform number of correct matches and minimal false matches, making it particularly robust in areas with repetitive textures and dynamic occlusion.

[0041] The present invention provides a resolution-adaptive, end-to-end sub-pixel feature matching method. There are numerous methods and approaches for implementing this technical solution. The foregoing description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A resolution-adaptive end-to-end sub-pixel feature matching method, characterized in that: The following steps are involved: Step 1: Joint detection and description of feature sub-units based on single-head coupling output; Step 2: Perform sub-pixel key point positioning based on an improved differentiable key point detection algorithm; Step 3, performing an adaptive mechanism for different resolutions or texture densities; Step 4: Perform end-to-end gradient closure based on back-propagation.

2. The method according to claim 1, characterized in that In step 1, a single-head coupled output structure is adopted to synchronously generate key point confidence and descriptors through a single set of convolution parameters, and a gradient coupling mechanism is designed to enable the detection and description tasks to share homologous features and be optimized collaboratively.

3. The method according to claim 2, characterized in that Step 1 includes: First, a lightweight convolutional neural network (CNN) is used to generate a feature map F and a keypoint score map S. The convolutional neural network (CNN) consists of four depthwise separable convolutional blocks with a maximum receptive field of 204×204. Use s to represent the confidence of the key point at each pixel position after activation by the Sigmoid function, and use s as the key point score; Input feature map Shared convolution kernel Processing, there are D+1 channels in total, the output channel tensor Z has a total of D+1, coordinates The mapping of the channel tensor Z at is defined as: （1）， in is convolution, is the weight matrix of the convolution kernel, b is the bias term, and b form a shared convolution kernel The set of trainable parameters.

4. The method according to claim 3, characterized in that In step 1, record the coordinates At , the normalized unit descriptor vector for: （2）， in, Represents the first D channels of the output tensor Z at coordinate (x, y), and the length of the vector is calculated by the L2 norm; The dimension is D×1.

5. The method according to claim 4, characterized in that In step 1, record the coordinates Key point score for: （3）， in Represents the D+1th channel of the feature tensor Z; is the Sigmoid function; The gradients of the descriptor loss and the detection loss are coupled to the shared convolution kernel through the chain rule. The same set of trainable parameters for adjusting and b, using Represents the gradient coupling formula: （4）， in represents partial differential; Represents the gradient of the descriptor loss with respect to the descriptor d; represents the gradient of the detection loss with respect to the score s; Indicates the response of the descriptor normalization operation to the original descriptor gradient; Represents the Sigmoid activation response to the original score gradient; Represents the convolutional layer output Z pair parameter set gradient; Key point score With the descriptor vector From shared convolution kernel , while accepting the gradients from the descriptor loss and the keypoint loss.

6. The method according to claim 5, characterized in that Step 2 includes: Step 2-1, preliminary detection of integer pixel key points: adopt the non-maximum suppression strategy, set the global threshold τ, the image coordinate system takes the upper left corner as the origin (0,0), u represents the horizontal coordinate of the pixel in the image, and v represents the vertical coordinate of the pixel in the image. The non-maximum suppression strategy searches for pixel coordinates (u,v) that meet the following two conditions on the entire score map: Score condition: the key point score s(i,j) of the coordinate (i,j) is greater than the preset global threshold τ; Local maximum condition: In a local neighborhood N with a radius of 2r+1 and a pixel coordinate (u, v) as the center, the key point score s(i, j) of the coordinate (i, j) is the maximum value of the key point scores of all neighboring pixels: The conditional formula is: , （5）， Where s(u,v) represents the key point score of pixel coordinate (u,v), N represents the local neighborhood centered at pixel coordinate (u,v) and is a window of size (2r+1)×(2r+1); max represents the maximum value function; The non-maximum suppression strategy only retains the pixels with the maximum score, and treats the filtered pixel coordinates (u, v) and the key point scores s(u, v) corresponding to the coordinates as constants in the back propagation; Step 2-2, the differentiable key point detection strategy DKD implements sub-pixel fine-tuning: For the key point score map S obtained by the lightweight convolutional neural network CNN, As the center, a local window P of size N×N is cut on the key point score map S, and the probability weighted average soft-argmax function is applied to all scores in the local window P to transform the key point score map S into a spatial probability distribution , at coordinates The spatial probability distribution value at The calculation is as follows: （6）， in is the maximum score in the window, is the temperature coefficient, exp represents the natural exponential function; Indicates that within the local window P, the coordinates The key point score at express The key point scores at other coordinates (p,q) in the same local window P; Then use the coordinate The spatial probability distribution value at As weights, calculate the expected coordinates of the key points in the window and the sub-pixel position p: （7）， Finally, the comprehensive center coordinates and sub-pixel positions , the expected coordinates Mapping back to the entire image coordinate system, the final sub-pixel key point coordinates for: （8）， in It is the coordinate of the center of the local window in its own coordinate system.

7. The method according to claim 6, characterized in that Step 3 includes: Define discrete candidate sets , represents the predefined optional value of the window size N; for the k-th candidate window currently being calculated , logarithmic weights can be learned by gradient descent update during training , and use Gumbel-Softmax to make the discrete sampling distribution continuous, and adjust the temperature parameter Control sampling and select weights in Gumbel distribution and sampled random noise as follows: （9）， in Indicates that when the summation term in the denominator traverses all candidate windows, the jth candidate window The corresponding learnable logarithmic weights, Indicates that when the summation term in the denominator traverses all candidate windows, the jth candidate window Random noise sampled using Gumbel distribution; For each Calculate pixel-level soft coordinates Finally, the pixel-level soft coordinates are weighted and fused to update the sub-pixel position p to achieve differentiable selection: （10）， （11）， Let the probability weighted average Soft-argmax temperature coefficient ,in, Used to control the concentration of coordinate probability distribution, It is a learnable parameter with an initial value of 0; the softplus function is a flexible plus activation function. Will Map to , so that the temperature parameter Always positive.

8. The method according to claim 7, characterized in that Step 4 includes: Through end-to-end joint detection and description, each position i corresponds to a candidate coordinate , mark the sub-pixel key point coordinates corresponding to position i as , the corresponding key point score Redefine it through the Sigmoid function, denoted as , indicating that the location The probability of the existence of a true key point is used for Bernoulli loss calculation, and the calculation formula is: （12）， in, Indicates that the sub-pixel key point coordinates The integer pixel coordinates obtained after rounding; Design a supervisory signal based on Bernoulli distribution. For position i, if position i falls within the geometric threshold neighborhood of a true key point In the label Otherwise, let the label 0; Confidence loss is calculated using Bernoulli negative log-likelihood , during the model training phase, enforce continuous confidence Fitting discrete labels : （13）， To achieve collaborative optimization of detection and description tasks, a joint loss function is constructed : （14）， in, Represents the reprojection loss The weight coefficient of Indicates the dispersion peak loss The weight coefficient of Denotes the descriptor loss The weight coefficient of Reliability loss The weight coefficient of Represents confidence loss The weight coefficient of Formulas (4), (9) to (14) are combined to form a differentiable computational graph. The gradient is fully refluxed through soft-argmax and Bernoulli, achieving integrated learning of detection and description: （15）， in, represents the gradient returned by the descriptor d, Represents the gradient of loss L with respect to descriptor d; Describes the descriptor d pair gradient; represents the gradient of the key point score s backpropagation, represents the gradient of loss L with respect to key point score s, Represents the key point score s pair The gradient, represents the sum of the gradients over all spatial coordinates.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: A computer program or instruction is stored, and when the computer program or instruction is run on a computer, the steps of the method according to any one of claims 1 to 8 are executed.

Images