A detectorless image feature matching method based on gray level prior
By constructing a gray-level prior matrix and extracting features using a convolutional neural network, combined with attention calculation, the robustness and efficiency issues of image matching in complex scenarios in existing technologies are solved, achieving high-precision image feature matching suitable for autonomous driving and robotic tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-26
- Publication Date
- 2026-05-29
AI Technical Summary
Existing image matching methods struggle to robustly acquire reproducible feature points in scenarios with significant changes in viewpoint, marked differences in illumination, sparse textures, and repetitive patterns. Furthermore, dense detection methods are computationally time-consuming and difficult to apply to latency-sensitive tasks.
By acquiring the grayscale and planar information association matrices of the image, a comprehensive prior matrix is constructed. Image features are extracted by combining convolutional neural networks, and prior-guided geometric self-attention and cross-attention calculations are performed to obtain coarse matching pairs. Then, fine-tuning is performed on the fine feature pairs to achieve image feature matching.
It improves the accuracy and efficiency of image matching, and provides an efficient and robust image matching solution in scenarios with large changes in viewpoint, significant differences in lighting, sparse texture and repetitive patterns. It is suitable for real-time sensitive fields such as robotics and autonomous driving.
Smart Images
Figure CN122115912A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and digital image processing technology, and more specifically to a detectorless image feature matching method based on gray-level prior. Background Technology
[0002] Currently, most existing image matching methods typically employ a two-step process: first, a feature extractor detects and describes feature points in the image; then, a matcher matches these descriptors. These extractors and matchers include both hand-designed and learning-based methods. While these detector-based approaches can efficiently obtain sparse matches between images, feature detectors often struggle to robustly obtain sufficient reproducible feature points when faced with challenging scenarios such as significant changes in viewpoint, substantial differences in illumination, sparse textures, and repetitive patterns.
[0003] However, detector-free methods no longer follow the traditional step of detection followed by description, but instead learn feature representations directly from images, which alleviates the aforementioned problems to some extent. Dense detector-free methods are computationally expensive and difficult to apply in a timely manner to latency-sensitive tasks such as autonomous driving and robotics. However, semi-dense methods achieve a better balance between accuracy and efficiency and are thus more widely used.
[0004] Therefore, how to effectively and interpretably utilize the prior information of the image itself to further improve the robustness and accuracy of image matching is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of the above problems, the present invention is proposed to provide a detectorless image feature matching method based on gray-level prior to overcome or at least partially solve the above problems.
[0006] A detectorless image feature matching method based on gray-level prior specifically includes the following steps:
[0007] S1. Obtain image pairs containing shared viewing regions, convert the images into grayscale images, calculate the grayscale information correlation matrix and planar information correlation matrix between pixel blocks based on the grayscale images, fuse the grayscale information correlation matrix and the planar information correlation matrix to construct a comprehensive prior matrix. S2. Use a convolutional neural network to extract coarse and fine feature pairs of image pairs; S3. Perform prior-guided geometric self-attention calculation on the coarse feature pairs of the image, then obtain the coarse matching confidence matrix through cross-attention calculation, and extract coarse matching pairs based on the coarse matching confidence matrix. S4. Select the corresponding region on the fine feature pair of the image based on the coarse matching pair, and perform fine matching processing to obtain the fine matching pair of image features.
[0008] Preferably, S1 specifically includes: S11. Convert the input image to a grayscale image, calculate the grayscale difference between any two pixels, and calculate the grayscale information correlation matrix. S12. Calculate the Manhattan distance between any two pixels and construct a planar information correlation matrix; S13. Weighted fusion of the grayscale information association matrix and the planar information association matrix to construct a comprehensive prior matrix.
[0009] Preferably, the convolutional neural network in S2 adopts a RepVGG structure, wherein the shallow features of the RepVGG structure output a high-resolution image and extract fine feature pairs of the image, and the deep features of the RepVGG structure output a low-resolution image and extract coarse feature pairs of the image.
[0010] Preferably, S3 specifically includes: S31. Calculate prior-guided geometric self-attention using coarse image features extracted by convolutional networks; S32. Perform cross-attention calculation on coarse feature pairs of the image, and repeat the prior-guided geometric self-attention calculation and cross-attention calculation. S33. Obtain the coarse matching confidence matrix through correlation operation and dual-headed Softmax operation, and use the mutual nearest neighbor algorithm to obtain the matching confidence from the coarse matching confidence matrix. After comparing the matching confidence with the set threshold, extract the coarse matching pairs.
[0011] Preferably, in S31, the prior-guided geometric self-attention calculation is obtained by decomposing the comprehensive prior matrix into a horizontal prior matrix and a vertical prior matrix. The horizontal prior matrix is calculated using horizontal self-attention, and the vertical prior matrix is calculated using vertical self-attention. The horizontal self-attention calculation and the vertical self-attention calculation are then fused to obtain the prior-guided geometric self-attention calculation.
[0012] The preferred formula for calculating horizontal self-attention is:
[0013] In the formula, Indicates horizontal self-attention. The attenuation coefficient and , This represents the comprehensive prior matrix, since the comprehensive prior matrix... The elements in the middle are non-negative numbers. Its diagonals are all 1. This indicates that attention is being paid to calculations along the horizontal direction. softmax ( ) indicates normalization. This represents the element-wise multiplication operation of matrices. Indicates the horizontal attention decay weight; The formula for calculating vertical self-attention is:
[0014] In the formula, Indicates vertical self-attention. This indicates that the attention is focused on calculations in the vertical direction. Indicates the vertical self-attention decay weight; The formula for calculating geometric self-attention, obtained by fusing horizontal and vertical self-attention calculations, is as follows:
[0015] In the formula, Let V represent geometric self-attention, and let V represent the value matrix.
[0016] Preferably, S4 specifically includes: S41. Based on the receptive field of the coarse matching pair, extract the corresponding feature blocks from the fine feature pairs of the image; S42. Perform correlation calculation and dual-headed softmax operation on the feature blocks to construct a fine feature confidence matrix, and use the nearest neighbor algorithm to obtain preliminary refined matching pairs; S43. Perform local expansion and correlation adjustment on the feature vectors in the preliminary refined matching pairs to obtain image feature fine matching pairs.
[0017] Preferably, the method also includes a neural network training step, wherein the loss function for neural network training includes the negative log-likelihood loss of the coarse matching stage, the loss of the first stage of fine matching, and the L2 loss of the second stage of fine matching, and the weighted sum of the above three losses is used as the total loss.
[0018] Preferably, the formula for calculating the negative log-likelihood loss in the coarse matching stage is:
[0019] In the formula, For coarse matching truth values, This represents the number of coarse matching logs. The formula for calculating the loss in the first stage of fine matching is:
[0020] In the formula, To fine-tune the truth value, For fine-grained matching of image features, For fine matching logarithms; The formula for calculating total loss is:
[0021] In the formula, and For preset coefficients, To refine the matching, the second stage uses the L2-norm loss.
[0022] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a detectorless image feature matching method based on gray-level prior. The beneficial effects of the above technical solutions provided by the embodiments of the present invention include at least the following: 1. By converting images to grayscale images, grayscale information association matrices and planar information association matrices are obtained. These matrices are used to directly adjust attention learning weights during the attention extraction process, preserving the physical and semantic information of the matrices and making the network more interpretable.
[0023] 2. In the attention calculation process, attention calculation is decomposed into horizontal and vertical components, which greatly saves computing resources and enhances image matching efficiency, showing significant application prospects in real-time sensitive fields such as robotics and autonomous driving.
[0024] 3. It effectively improves the accuracy of image matching, providing an efficient and robust image matching solution for challenging scenarios such as significant changes in viewing angle, obvious differences in lighting, sparse textures, and repetitive patterns. Attached Figure Description
[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0026] Figure 1 This is a flowchart of a detectorless image feature matching method based on gray-level prior provided in an embodiment of the present invention. Detailed Implementation
[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0028] This invention discloses a detectorless image feature matching method based on gray-level prior.
[0029] Specifically, the following steps are included: S1. Obtain image pairs containing shared viewing regions, convert the images into grayscale images, calculate the grayscale information correlation matrix and planar information correlation matrix between pixel blocks based on the grayscale images, fuse the grayscale information correlation matrix and the planar information correlation matrix to construct a comprehensive prior matrix. S2. Use a convolutional neural network to extract coarse and fine feature pairs of image pairs; S3. Perform prior-guided geometric self-attention calculation on the coarse feature pairs of the image, then obtain the coarse matching confidence matrix through cross-attention calculation, and extract coarse matching pairs based on the coarse matching confidence matrix. S4. Select the corresponding region on the fine feature pair of the image based on the coarse matching pair, and perform fine matching processing to obtain the fine matching pair of image features.
[0030] This invention effectively improves the accuracy of image matching and is an efficient and robust image matching solution for challenging scenarios such as large changes in viewing angle, significant differences in illumination, sparse textures, and repetitive patterns. The invention is described below with specific embodiments.
[0031] like Figure 1 As shown, a detectorless image feature matching method based on gray-level prior is adopted to achieve high-precision image matching. Specifically, this embodiment mainly includes six core parts: extracting image pairs containing shared viewing regions using a camera; converting the images into grayscale images; calculating the gray-level information correlation matrix and planar information correlation matrix between pixel blocks based on the grayscale images; extracting coarse and fine feature pairs of the image using a convolutional neural network; and obtaining the coarse matching confidence matrix by performing prior-guided self-attention and cross-attention calculations on the coarse features. and obtain coarse matching pairs. Based on the coarse matching pairs, corresponding regions are selected on the fine feature pairs, and a two-step refinement process is performed to obtain the final image fine feature matching pairs. Neural network training; image matching during the inference phase.
[0032] A specific example will be described in detail: Step 1: Use the camera to extract image pairs containing shared viewing areas, convert the images into grayscale images, and calculate the grayscale information correlation matrix and planar information correlation matrix between pixel blocks based on the grayscale images.
[0033] Step 1 includes: S11, defining the width of the input image as... The height is Convert the input image into a grayscale image, corresponding to the pixel coordinates. The grayscale is For higher resolution images, average pooling can be performed on the grayscale of a single pixel block, and then the operation can be applied to the entire pixel block. The grayscale difference between pixels is used to describe the relative relationship between pixels.
[0034] In the formula, It is a symmetrical shape with both length and width being equal to 1. The grayscale information correlation matrix has all diagonal elements as 0.
[0035] S12. Considering that the grayscale information correlation matrix does not contain planar distance information between pixels, this invention uses Manhattan distance to describe the planar distance relationship between pixels.
[0036] In the formula, It is a symmetrical shape with both length and width being equal to 1. The image planar information correlation matrix has all diagonal elements as 0.
[0037] S13. The attention relationship between any two pixels can be fused from the gray-level information correlation matrix and the planar information correlation matrix. Therefore, we can construct a comprehensive prior matrix. :
[0038] In the formula, and For preset coefficients, and .
[0039] Step 2: Extract coarse and fine feature pairs from the image using a convolutional neural network. Specifically, this example uses a 3-layer RepVGG convolutional neural network with a stride of 2. The output of the first layer of the RepVGG convolutional neural network is used as shallow features, outputting a high-resolution image and extracting fine feature pairs, with a resolution of 1 / 2 of the input image. The output of the third layer of the RepVGG convolutional neural network is used as deep features, outputting a low-resolution image and extracting coarse feature pairs, with a resolution of 1 / 8 of the input image.
[0040] Step 3: Perform prior-guided geometric self-attention and cross-attention calculations on the coarse features to obtain the coarse matching confidence matrix. and obtain coarse matching pairs. .
[0041] Furthermore, step 3 includes: S31. Perform prior-guided geometric self-attention calculation on the coarse image feature pairs extracted by the convolutional neural network.
[0042] The formula for calculating self-attention guided by prior knowledge is:
[0043] In the formula, Represents the query matrix. V represents the key matrix, and V represents the value matrix. The attenuation coefficient and Due to the comprehensive prior matrix Since the elements in the middle are non-negative, Its diagonals are all 1.
[0044] The prior-guided geometric self-attention computation in S31 is performed by synthesizing the prior matrix. Decomposed into level prior matrices and vertical prior matrix The level prior matrix It is a size of A matrix representing each coordinate as Pixel and corresponding number The relationship between all pixels in the row.
[0045] Similarly, the vertical prior matrix It is a size of A matrix representing each coordinate as Pixel and corresponding number The relationship between all pixels.
[0046] Furthermore, the horizontal prior matrix is computed using horizontal self-attention, and the vertical prior matrix is computed using vertical self-attention. The horizontal and vertical self-attention computes are then fused to obtain prior-guided geometric self-attention computed.
[0047] The formula for calculating horizontal self-attention is as follows:
[0048] In the formula, Indicates horizontal self-attention. The attenuation coefficient and , This represents the comprehensive prior matrix, since the comprehensive prior matrix... The elements in the middle are non-negative numbers. Its diagonals are all 1. This indicates that attention is being paid to calculations along the horizontal direction. softmax ( ) indicates normalization. This represents the element-wise multiplication operation of matrices. Indicates the horizontal attention decay weight; The formula for calculating vertical self-attention is:
[0049] In the formula, Indicates vertical self-attention. This indicates that the attention is focused on calculations in the vertical direction. Indicates the vertical self-attention decay weight; The formula for calculating geometric self-attention, obtained by fusing horizontal and vertical self-attention calculations, is as follows:
[0050] In the formula, Let V represent geometric self-attention, and let V represent the value matrix.
[0051] S32. Perform cross-attention calculation on the coarse feature pairs of the image, and repeat the prior self-attention and cross-attention calculations. Second-rate.
[0052] S33. The matching confidence matrix is obtained by using correlation calculation and bi-headed softmax operation. Finally, the mutual nearest neighbor algorithm is used to obtain the matching confidence from the coarse matching confidence matrix, and the matching confidence is compared with a set threshold to extract coarse matching pairs.
[0053] Furthermore, when the matching confidence is greater than a set threshold, fine feature matching pairs are obtained by refining the coarse features; when the matching confidence is less than a set threshold, coarse features are discarded.
[0054] Step 4: Select the corresponding region on the fine feature pair of the image based on the coarse matching pair, and perform two-step refinement to obtain the final fine feature matching pair of the image. .
[0055] Furthermore, S4 includes: S41. Based on the receptive field corresponding to the obtained coarse matching, extract the corresponding feature block from the shallow fine matching features.
[0056] S42. Perform correlation calculations and dual-headed Softmax operations on the fine feature blocks to construct the fine feature confidence matrix. The mutual nearest neighbor algorithm is used to obtain preliminary refined matching pairs.
[0057] S43. Expand the feature vectors in the initial refined matching pair to 3 on the fine feature blocks. For each feature block, the corresponding feature vector is correlated with the feature block, and the position of the feature vector is adjusted on the feature block according to the correlation, resulting in a fine-grained matching pair of image features. .
[0058] Step 5: Neural Network Training. During neural network training, this invention standardizes the longest side of the input image to 832 pixels and defines the coarse matching truth value as... There are a total of For a match, the truth value of a fine match is There are a total of For the coarse matching stage, this invention supervises the process by minimizing the negative log-likelihood loss function; for the fine matching stage, this invention supervises the process by using the first-stage loss function and the second-stage L2 loss function.
[0059] The formula for calculating the negative log-likelihood loss in the coarse matching stage is as follows:
[0060] In the formula, For coarse matching truth values, This represents the number of coarse matching logs.
[0061] The formula for calculating the loss in the first stage of fine matching is:
[0062] In the formula, To fine-tune the truth value, For fine-grained matching of image features, For fine matching logarithms.
[0063] The formula for calculating the L2 loss in the second stage of fine matching is:
[0064] This represents the total number of fine-matching features. The total loss is calculated by weighted summation of the three types of loss. The formula for calculating the total loss is as follows:
[0065] In the formula, and These are preset coefficients, set to 1.0 and 0.25 respectively. To refine the matching, the second stage uses the L2-norm loss.
[0066] The specific implementation methods of each step have been described in the embodiments and will not be detailed here. To overcome the shortcomings of current semi-dense image matching techniques in effectively integrating prior information, this invention innovatively proposes a novel matching framework based on visual images. Its core lies in: obtaining physical priors through grayscale images and integrating them as weight adjustment factors into the attention extraction module, making the network learning process more interpretable; simultaneously, by decomposing attention calculation into horizontal and vertical directions, computational efficiency is greatly optimized. This invention achieves a significant improvement in accuracy, especially in robustly handling challenging environments such as large viewpoint changes, significant lighting differences, weak textures, and repetitive textures, providing an efficient image matching solution for real-time applications such as autonomous driving and mobile robots.
[0067] The semi-dense, detectorless image feature matching neural network method based on gray-level prior in the above embodiments is programmed and implemented. The computer processor or embedded device is used to perform tasks such as image data processing, neural network training, and neural network inference calculation, forming a semi-dense, detectorless image feature matching neural network system based on gray-level prior, enabling the terminal system to have high-precision image feature matching capabilities.
[0068] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by combining program instructions with hardware. The aforementioned program can be stored in a computer-readable storage medium and, when executed, completes the various steps of the above method embodiments. The aforementioned storage medium includes, but is not limited to, mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, or other media that can store program code.
[0069] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A detectorless image feature matching method based on gray-level prior, characterized in that, Includes the following steps: S1. Obtain image pairs containing shared viewing regions, convert the images into grayscale images, calculate the grayscale information correlation matrix and planar information correlation matrix between pixel blocks based on the grayscale images, fuse the grayscale information correlation matrix and the planar information correlation matrix to construct a comprehensive prior matrix. S2. Use a convolutional neural network to extract coarse and fine feature pairs of image pairs; S3. Perform prior-guided geometric self-attention calculation on the coarse feature pairs of the image, then obtain the coarse matching confidence matrix through cross-attention calculation, and extract coarse matching pairs based on the coarse matching confidence matrix. S4. Select the corresponding region on the fine feature pair of the image based on the coarse matching pair, and perform fine matching processing to obtain the fine matching pair of image features.
2. The detectorless image feature matching method based on gray-level prior according to claim 1, characterized in that, S1 specifically includes: S11. Convert the input image to a grayscale image, calculate the grayscale difference between any two pixels, and calculate the grayscale information correlation matrix. S12. Calculate the Manhattan distance between any two pixels and construct a planar information correlation matrix; S13. Weighted fusion of the grayscale information association matrix and the planar information association matrix to construct a comprehensive prior matrix.
3. The detectorless image feature matching method based on gray-level prior according to claim 1, characterized in that, The convolutional neural network in S2 adopts the RepVGG structure. The shallow features of the RepVGG structure output a high-resolution image and extract fine feature pairs of the image, while the deep features of the RepVGG structure output a low-resolution image and extract coarse feature pairs of the image.
4. The detectorless image feature matching method based on gray-level prior according to claim 1, characterized in that, S3 specifically includes: S31. Calculate prior-guided geometric self-attention using coarse image features extracted by convolutional networks; S32. Perform cross-attention calculation on coarse feature pairs of the image, and repeat the prior-guided geometric self-attention calculation and cross-attention calculation. S33. Obtain the coarse matching confidence matrix through correlation operation and dual-headed Softmax operation, and use the mutual nearest neighbor algorithm to obtain the matching confidence from the coarse matching confidence matrix. After comparing the matching confidence with the set threshold, extract the coarse matching pairs.
5. A detectorless image feature matching method based on gray-level prior according to claim 4, characterized in that, In S31, the prior-guided geometric self-attention computation decomposes the comprehensive prior matrix into a horizontal prior matrix and a vertical prior matrix. The horizontal prior matrix is computed using horizontal self-attention, and the vertical prior matrix is computed using vertical self-attention. The horizontal self-attention computation and the vertical self-attention computation are then fused to obtain the prior-guided geometric self-attention computation.
6. A detectorless image feature matching method based on gray-level prior according to claim 5, characterized in that, The formula for calculating horizontal self-attention is: In the formula, Indicates horizontal self-attention. The attenuation coefficient and , This represents the comprehensive prior matrix, since the comprehensive prior matrix... The elements in the middle are non-negative numbers. Its diagonals are all 1. This indicates that attention is being paid to calculations along the horizontal direction. softmax ( ) indicates normalization. This represents the element-wise multiplication operation of matrices. Indicates the horizontal attention decay weight; The formula for calculating vertical self-attention is: In the formula, Indicates vertical self-attention. This indicates that the attention is focused on calculations in the vertical direction. Indicates the vertical self-attention decay weight; The formula for calculating geometric self-attention, obtained by fusing horizontal and vertical self-attention calculations, is as follows: In the formula, Let V represent geometric self-attention, and let V represent the value matrix.
7. A detectorless image feature matching method based on gray-level prior according to claim 1, characterized in that, S4 specifically includes: S41. Based on the receptive field of the coarse matching pair, extract the corresponding feature blocks from the fine feature pairs of the image; S42. Perform correlation calculation and dual-headed softmax operation on the feature blocks to construct a fine feature confidence matrix, and use the nearest neighbor algorithm to obtain preliminary refined matching pairs; S43. Perform local expansion and correlation adjustment on the feature vectors in the preliminary refined matching pairs to obtain image feature fine matching pairs.
8. A detectorless image feature matching method based on gray-level prior according to claim 1, characterized in that, It also includes a neural network training step. The loss function for neural network training includes the negative log-likelihood loss of the coarse matching stage, the loss of the first stage of fine matching, and the L2 loss of the second stage of fine matching. The total loss is obtained by weighted summation of the above three losses.
9. A detectorless image feature matching method based on gray-level prior according to claim 8, characterized in that, The formula for calculating the negative log-likelihood loss in the coarse matching stage is: In the formula, For coarse matching truth values, This represents the number of coarse matching logs. The formula for calculating the loss in the first stage of fine matching is: In the formula, To fine-tune the truth value, For fine-grained matching of image features, For fine matching logarithms; The formula for calculating total loss is: In the formula, and For preset coefficients, To refine the matching, the second stage uses the L2-norm loss.