ORB feature extraction and matching method based on adaptive threshold and geometric constraint

Through adaptive threshold and geometric constraint model, the problems of instability and unevenness of feature point distribution in the lighting change environment of traditional feature extraction algorithms are solved, and the accuracy and real-time nature of feature matching are improved, and it is suitable for real-time image processing systems. .

CN119068215APending Publication Date: 2024-12-03CHANGCHUN UNIV OF SCI & TECH
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411122394.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The traditional feature extraction algorithm is unstable in an environment with significant lighting changes, the feature points are unevenly distributed, and the mismatch problem is serious, which affects the accuracy and real-timeness of the visual odometer.

Method used

Adaptive threshold adjustment based on image grayscale mean and quadtree management feature point distribution are adopted, and feature matching is combined with geometric constraint models based on slope, distance and descriptor distance to improve the adaptability and accuracy of the algorithm.

Benefits of technology

It improves the stability and accuracy of feature extraction, reduces the error matching rate, improves processing speed and positioning accuracy, and is suitable for real-time image processing systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068215A_ABST
    Figure CN119068215A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of image processing, and relates to an ORB feature extraction and matching method based on a self-adaptive threshold value and geometric constraints, which comprises the following steps of: dynamically adjusting a feature extraction threshold value in an ORB algorithm according to an average gray value and a gray variance of an image through a self-adaptive threshold value algorithm; image feature extraction under different illumination conditions can be adapted; a quadtree structure is adopted to manage feature points, and uniform distribution of the feature points is achieved; a feature matching process is optimized by utilizing geometric constraints based on the slope, the distance and the descriptor distance between feature point pairs, and mismatching points are removed; and finally, performing a visual odometer task by using the processed feature point pairs, thereby improving the accuracy and robustness of positioning and navigation. Compared with the prior art, the method can provide more stable and uniform feature extraction in different illumination environments, and greatly improves the efficiency and accuracy of feature matching at the same time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to feature extraction and feature matching technology in visual odometer. Background Art

[0002] In the field of visual odometry / visual SLAM, feature extraction and feature matching are the most critical steps. These technologies are widely used in various scenarios such as autonomous driving. Traditional feature extraction algorithms such as SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Features) can provide highly robust feature points, but these algorithms have high computational costs and long processing times, and are not suitable for applications that require real-time response.

[0003] The ORB (Oriented FAST and Rotated BRIEF) algorithm has been widely used in real-time systems due to its high efficiency. The ORB algorithm combines FAST feature point detection and BRIEF descriptor, and through its specific design, it significantly improves the calculation speed. However, the ORB algorithm does not work well when dealing with scenes with obvious lighting changes. This is because it uses a fixed threshold for feature point detection and fails to adapt to changes in image brightness. In addition, the ORB algorithm also has the problem of uneven distribution of feature points, which causes feature points to easily gather in areas with rich textures, while feature points are scarce in areas with less texture.

[0004] In terms of feature matching, common methods such as brute force matching based on Euclidean distance or tree-based search algorithms can find accurate matching points, but they are not ideal when dealing with data containing a lot of noise and mismatches. The existence of mismatches will significantly affect subsequent position estimation and other results. To solve this problem, the RANSAC algorithm is widely used to remove a match, but the efficiency and robustness of the algorithm are still limited in a high mismatch environment.

[0005] Therefore, a new feature extraction and matching method is needed, which should not only remain stable under changing lighting conditions, but also improve the uniformity of feature distribution, computational efficiency, and matching accuracy to meet the needs of real-time image processing systems. Summary of the invention

[0006] 1. Technical issues to be solved

[0007] In view of the above problems, the present invention provides an ORB feature extraction and matching method based on adaptive threshold and geometric constraints. The existing ORB feature extraction algorithm usually uses a fixed threshold to detect feature points, which leads to a significant decrease in the stability and accuracy of feature extraction in an environment with significant changes in illumination. The present invention improves the adaptability and robustness of the algorithm under different illumination conditions by introducing an adaptive threshold adjustment mechanism based on the image grayscale mean. In the traditional feature extraction process, feature points tend to gather in areas with rich textures, while feature points are scarce in areas with less texture. The present invention manages the distribution of feature points by using a quadtree to ensure that feature points are evenly distributed in the image, thereby improving the accuracy of feature matching. In the existing feature matching technology, the problem of mismatching is particularly prominent in complex scenes, affecting the accuracy of visual odometers. The present invention introduces a geometric constraint model based on slope, distance and descriptor distance to effectively screen out real matching point pairs, reduce the mismatch rate, and improve the processing speed.

[0008] (II) Technical solution

[0009] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0010] An ORB feature extraction and matching method based on adaptive threshold and geometric constraints includes the following steps:

[0011] S1. Preprocessing the input image, using an improved adaptive threshold algorithm to extract ORB features, wherein the adaptive threshold algorithm uses the grayscale mean of the image as an evaluation function;

[0012] S2. Use quadtree to manage feature points to ensure uniform distribution of feature points;

[0013] S3, adopts the external point removal algorithm based on geometric constraints, and constructs the constraint set by matching the slope, distance and descriptor distance between feature points.

[0014] Furthermore, the specific steps of S1 are as follows:

[0015] S11, using a monocular camera to acquire a continuous image sequence;

[0016] S12, performing grayscale and denoising processing on each input image;

[0017] S13. Calculate the grayscale mean of the image The threshold T for feature extraction is determined by the following formula:

[0018]

[0019] Among them, I(x i) is the gray value of the i-th pixel, α and β are proportional coefficients;

[0020] S14. When extracting feature points, T is used to determine whether a pixel point is a feature point, thereby achieving adaptive adjustment to illumination changes.

[0021] Furthermore, the specific steps of S2 are as follows:

[0022] S21, initialize the image as a node, and divide it into four quadrants, each quadrant as a child node, and continue to further divide;

[0023] S22, for each child node, determine the number of feature points contained therein, if the number of feature points is greater than 1, continue to divide the child node into four child nodes, and repeat this process until the expected number of feature points or division depth is reached;

[0024] S23. In each child node, calculate the Harris response values ​​of all feature points, and use the non-maximum suppression method to retain the feature point with the largest response value. The formula is as follows:

[0025] R = det(M) - k(trace(m)) 2

[0026] Where m is the gradient matrix and k is an empirical parameter used to balance the influence of the determinant and the trace and further adjust the calculation of the response value;

[0027] S24, the gradient matrix M can be calculated by the following formula:

[0028]

[0029] Where ω(x,y) is the window function, I x is the gradient of the feature point in the horizontal direction, I y is the gradient of the feature point in the vertical direction;

[0030] S25. Through the non-maximum suppression method, the feature point with the largest Harris response value in each child node is retained, redundant feature points are removed, and the uniform distribution of feature points is ensured.

[0031] Furthermore, the specific steps of S3 are as follows:

[0032] S31, let the distance between ORB descriptors calculated during feature matching be des i ;

[0033] S32, calculate the slope k of each pair of matching feature points on the pixel plane i and distance d i :

[0034]

[0035] in and is the pixel coordinates of a pair of matching feature points, and the corresponding pixel points are represented as and

[0036] S33. Convert the slope to an angle using the following formula:

[0037]

[0038] S34. Establish a geometric constraint set through S31, S32 and S33:

[0039]

[0040] Where k is the slope reference value, ε k is the slope threshold, d i is the distance reference value, ε d is the distance threshold, ε des is the descriptor distance threshold;

[0041] S35. Based on the feature point pairs after geometric constraint screening, the RANSAC algorithm is further used for secondary screening to remove more mismatched points.

[0042] (III) Beneficial effects

[0043] Compared with the prior art, the present invention provides an ORB feature extraction and matching method based on adaptive threshold and geometric constraints, which has the following beneficial effects:

[0044] The present invention can dynamically adjust the feature extraction threshold according to the grayscale mean of the image through an adaptive threshold adjustment mechanism, so that the feature detection process has stronger adaptability to lighting changes. The use of quadtree to manage feature points improves the uniformity of the distribution of feature points in the image, which not only improves the accuracy of feature matching, but also enhances the positioning accuracy of the visual odometer. The introduction of feature mismatch filtering based on geometric constraints effectively reduces the occurrence of mismatches. This screening mechanism based on slope, distance and descriptor distance can accurately identify and send out matching point pairs that do not meet the characteristics, thereby improving the accuracy of the matching results. The present invention optimizes the calculation process of feature extraction and matching, reduces unnecessary calculations, and makes the entire processing process more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in the field, other drawings can be obtained based on these drawings without creative labor.

[0046] Figure 1 is a flow chart of the method of the present invention;

[0047] Figure 2 The algorithm flow chart of the combination of adaptive threshold adjustment and quadtree feature point management provided by the present invention;

[0048] Figure 3 Flowchart of the feature matching algorithm that combines the geometric constraint model with RANSAC; DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The present invention will be further described below in conjunction with the embodiments and the accompanying drawings.

[0051] Example

[0052] like Figure 1 The present invention provides an ORB feature extraction and matching method based on adaptive threshold and geometric constraints, and its implementation method is as follows:

[0053] S1. Preprocess the input image and use an improved adaptive threshold algorithm to extract ORB features. The adaptive threshold algorithm uses the grayscale mean of the image as an evaluation function. The implementation method is as follows:

[0054] S11, using a monocular camera to acquire a continuous image sequence;

[0055] S12, performing grayscale and denoising processing on each input image;

[0056] S13. Calculate the grayscale mean of the image The threshold T for feature extraction is determined by the following formula:

[0057]

[0058] Among them, I(x i) is the gray value of the i-th pixel, α and β are proportional coefficients;

[0059] S14. When extracting feature points, T is used to determine whether a pixel point is a feature point, thereby achieving adaptive adjustment to illumination changes.

[0060] S2. Use quadtree to manage feature points to ensure uniform distribution of feature points. The implementation method is as follows:

[0061] S21, initialize the image as a node, and divide it into four quadrants, each quadrant as a child node, and continue to further divide;

[0062] S22, for each child node, determine the number of feature points contained therein, if the number of feature points is greater than 1, continue to divide the child node into four child nodes, and repeat this process until the expected number of feature points or division depth is reached;

[0063] S23. In each child node, calculate the Harris response values ​​of all feature points, and use the non-maximum suppression method to retain the feature point with the largest response value. The formula is as follows:

[0064] R = det(M) - k(trace(m)) 2

[0065] Where m is the gradient matrix and k is an empirical parameter used to balance the influence of the determinant and the trace and further adjust the calculation of the response value;

[0066] S24, the gradient matrix M can be calculated by the following formula:

[0067]

[0068] Where ω(x,y) is the window function, I x is the gradient of the feature point in the horizontal direction, I y is the gradient of the feature point in the vertical direction;

[0069] S25. Through the non-maximum suppression method, the feature point with the largest Harris response value in each child node is retained, redundant feature points are removed, and the uniform distribution of feature points is ensured.

[0070] S3, using the geometric constraint-based external point removal algorithm, constructing the constraint set by matching the slope, distance and descriptor distance between feature points, the implementation method is as follows:

[0071] S31, let the distance between ORB descriptors calculated during feature matching be des i ;

[0072] S32, calculate the slope k of each pair of matching feature points on the pixel planei and distance d i :

[0073]

[0074] in and is the pixel coordinates of a pair of matching feature points, and the corresponding pixel points are represented as and

[0075] S33. Convert the slope to an angle using the following formula:

[0076]

[0077] S34. Establish a geometric constraint set through S31, S32 and S33:

[0078]

[0079] Where k is the slope reference value, ε k is the slope threshold, d i is the distance reference value, ε d is the distance threshold, ε des is the descriptor distance threshold;

[0080] S35. Based on the feature point pairs after geometric constraint screening, the RANSAC algorithm is further used for secondary screening to remove more mismatched points.

[0081] In this embodiment, Figure 2 As shown, the process of combining the adaptive threshold adjustment with the quadtree feature point management algorithm is shown, which is specifically as follows:

[0082] Read the input image and convert it into a grayscale image, calculate the grayscale mean of the image for adaptive threshold adjustment. Use the adjusted threshold to apply the FAST algorithm for preliminary extraction of feature points. After the feature point extraction is completed, initialize the root node of the quadtree, assign all extracted feature points to the root node, and start recursive node division. For each node, if the number of feature points in the node is greater than 1 or the current division depth does not reach the maximum depth, the node is further divided into four child nodes. The four child nodes are: upper left child node, upper right child node, lower left child node, and lower right child node. In each child node, calculate the Harris response values ​​of all feature points, apply the non-maximum suppression method, and retain the feature point with the largest response value in each child node.

[0083] In this embodiment, Figure 3 As shown, the process of the feature matching algorithm combining the geometric constraint model with RANSAC is shown, which is specifically as follows:

[0084] Perform preliminary feature point matching, and retain preliminary matching pairs by calculating the descriptor distance of each pair of feature points. Calculate geometric constraint parameters for these preliminary matching pairs, including the slope, distance, and angle of each pair of feature points. Based on these geometric parameters, establish a set constraint set. Compare the slope, distance, and descriptor distance of each pair of feature points with the preset threshold to filter out matching pairs that meet the geometric constraints. Finally, apply the RANSAC algorithm to further remove mismatched point pairs, so as to further extract reliable matching point pairs.

[0085] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An ORB feature extraction and matching method based on adaptive threshold and geometric constraints, characterized in that: The following steps are involved: S1. Preprocessing the input image, using an improved adaptive threshold algorithm to extract ORB features, wherein the adaptive threshold algorithm uses the grayscale mean of the image as an evaluation function; S2. Use quadtree to manage feature points to ensure uniform distribution of feature points; S3, adopts the external point removal algorithm based on geometric constraints, and constructs the constraint set by matching the slope, distance and descriptor distance between feature points.

2. The ORB feature extraction and matching method based on adaptive threshold and geometric constraints according to claim 1, characterized in that: The specific steps of S1 are as follows: S11, using a monocular camera to acquire a continuous image sequence; S12, performing grayscale and denoising processing on each input image; S13. Calculate the grayscale mean of the image The threshold T for feature extraction is determined by the following formula: Among them, I(x i ) is the gray value of the i-th pixel, α and β are proportional coefficients; S14. When extracting feature points, T is used to determine whether a pixel point is a feature point, thereby achieving adaptive adjustment to illumination changes.

3. The ORB feature extraction and matching method based on adaptive threshold and geometric constraints according to claim 1, characterized in that: The specific steps of S2 are as follows: S21, initialize the image as a node, and divide it into four quadrants, each quadrant as a child node, and continue to further divide; S22, for each child node, determine the number of feature points contained therein, if the number of feature points is greater than 1, continue to divide the child node into four child nodes, and repeat this process until the expected number of feature points or division depth is reached; S23. In each child node, calculate the Harris response values ​​of all feature points, and use the non-maximum suppression method to retain the feature point with the largest response value. The formula is as follows: R=det(M)-k(trace(M)) 2 Where M is the gradient matrix and k is an empirical parameter used to balance the influence of the determinant and the trace and further adjust the calculation of the response value; S24, the gradient matrix M can be calculated by the following formula: Where ω(x, y) is the window function, I x is the gradient of the feature point in the horizontal direction, I y is the gradient of the feature point in the vertical direction; S25. By using the non-maximum suppression method, the feature point with the largest Harris response value in each child node is retained, redundant feature points are removed, and the uniform distribution of feature points is ensured.

4. The ORB feature extraction and matching method based on adaptive threshold and geometric constraint according to claim 1, characterized in that: The specific steps of S3 are as follows: S31, let the distance between ORB descriptors calculated during feature matching be des i ; S32, calculate the slope k of each pair of matching feature points on the pixel plane i and distance d i : in and is the pixel coordinates of a pair of matching feature points, and the corresponding pixel points are represented as and S33. Convert the slope to an angle using the following formula: S34. Establish a geometric constraint set through S31, S32 and S33: Where k is the slope reference value, ε k is the slope threshold, d i is the distance reference value, ε d is the distance threshold, ε des is the descriptor distance threshold; S35. Based on the feature point pairs after geometric constraint screening, the RANSAC algorithm is further used for secondary screening to remove more mismatched points.

Citation Information

Cited By

  • Method and system for rapidly extracting and matching features in low-illumination weak-texture environment

    CN119540576A

  • Self-adaptive balanced visual feature matching method for unmanned aerial vehicle in dark illumination scene

    CN120235781A