A feature point mismatch elimination method applied to visual slam

Abstract

The application relates to a feature point mismatch elimination method applied to visual SLAM and belonging to the technical field of computer vision. The method comprises the following steps: extracting ORB feature points from images collected by a camera, matching feature points of a current frame with feature points of key frames, calculating the Euclidean distance of matched feature point pairs, roughly eliminating outliers of the Euclidean distance by using a boxplot method, performing histogram statistics on the Euclidean distance of the feature point pairs after rough elimination, performing data processing on the histogram, and eliminating matching point pairs with poor quality according to the histogram processing result. The beneficial effects are that the feature point matching error is eliminated, the robustness of the SLAM system is improved, the accuracy of subsequent mapping and positioning is improved, the calculation amount and storage space are reduced, and the efficiency of the system is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for removing mismatched feature points applied to visual SLAM. Background Technology

[0002] SLAM (Simultaneous Localization and Mapping) is a technology that uses one or more cameras to perform localization and map building in unknown environments. Based on the type of sensor used, SLAM can be divided into visual SLAM and laser SLAM. Currently, visual SLAM technology has been widely used in fields such as autonomous driving, robot navigation, and virtual reality, helping to achieve functions such as autonomous navigation and map building, improving production and life efficiency, and has broad application prospects.

[0003] Currently, mainstream visual SLAM feature point matching algorithms include feature point descriptor matching, direct matching, and deep learning. Feature point descriptor matching calculates feature descriptors for the regions surrounding each feature point, representing each feature point as a vector, and then performs matching by calculating the distance or similarity between vectors. Direct matching differs from traditional feature point-based methods; it does not require feature extraction and matching, but directly uses pixel grayscale information for matching. Deep learning-based end-to-end matching networks can directly learn matching functions from image pairs, achieving end-to-end feature point matching. The advantage of this method is that it avoids the limitations of manually designed feature descriptors and can improve matching accuracy and robustness by training the network with large amounts of data.

[0004] Although feature point matching algorithms perform well in many cases, they are very sensitive to changes in lighting, object occlusion, and camera or object motion blur. These factors can lead to inaccuracies in feature point extraction and matching. Furthermore, when dealing with dynamic objects, the feature points of the dynamic objects may change, which may cause matching failures. Therefore, directly using the matched feature points for subsequent processing may introduce noise into the system. Summary of the Invention

[0005] This invention provides a feature point mismatch removal method for visual SLAM, which solves the problem of mismatch in current feature point matching. By removing mismatched feature points, the robustness and accuracy of the visual SLAM system are improved.

[0006] The technical solution adopted by this invention includes the following steps:

[0007] (1) Extract ORB feature points from the images captured by the camera;

[0008] (2) Match the feature points of the current frame with the feature points of the key frame;

[0009] (3) Calculate the Euclidean distance between the matching feature point pairs;

[0010] (4) Use the boxplot method to roughly remove outliers from the Euclidean distance;

[0011] (5) Perform histogram statistics on the Euclidean distance of all feature points;

[0012] (6) Process the histogram data;

[0013] (7) Remove the mismatched point pairs from the matching relationship based on the histogram statistics.

[0014] Step (1) of this invention involves extracting ORB feature points from the image captured by the camera, specifically including:

[0015] The acquired RGB images were converted to grayscale images; a total of 1000 feature points were extracted using an 8-layer pyramid, where the number of feature points extracted from the i-th layer was:

[0016]

[0017] Where s is the scaling factor, taking The FAST algorithm is used to extract feature points for each layer of the pyramid.

[0018] The orientation vector of each feature point is calculated using the gray-scale centroid method, and the brief descriptor of each feature point is calculated, where the template orientation vector of the brief descriptor is consistent with the orientation vector of the feature point.

[0019] Step (2) of this invention involves matching the feature points of the current frame with the feature points of the keyframe, specifically including:

[0020] Camera initialization and local map point creation are completed based on the ORB-SLAM2 algorithm framework. The similarity of the brief descriptor is compared between the current frame pixel and the key frame pixel. In order to accelerate the extraction, only the pixel region with a radius of 20 with the current frame pixel is considered, and the key frame pixel with the highest similarity is selected as the matching point pair. All pixels are traversed in a loop to find the matching key frame pixel.

[0021] Step (3) of the present invention, which calculates the Euclidean distance between matching feature point pairs, specifically includes:

[0022] Define a point set P = {p1, p2, ..., p...} that has already undergone feature matching. n}, Q={q1,q2,...,q n}, where P is the feature point set of the current frame, Q is the feature point set of the keyframe, n is the index of the feature point, and p is the element of the point set. n and q n For the matched feature points, the set of Euclidean distances between feature point pairs is denoted as Dis = {dis1, dis2, ..., dis...} n The formula is Dis = ||PQ|| 2 That is, calculate the Euclidean distance of each pair of feature points to obtain the set of Euclidean distances Dis for the matching pairs.

[0023] Step (4) of this invention uses the boxplot method to roughly remove outliers from the Euclidean distance, specifically including:

[0024] Arrange the data in set Dis in ascending order. Let Q1 be the lower quartile, Q2 be the median, Q3 be the upper quartile, and IQR be the interquartile range, where IQR = Q3 - Q1. Take the upper limit of the data as Q3 + 1.5IQR and the lower limit of the data as Q1 - 1.5IQR. That is, remove the data in set Dis that are not within Q1 - 1.5IQR to Q3 + 1.5IQR as outliers. Let Dis be the result of removing the data after removing the outliers. Then remove the corresponding data from point sets P and Q. Let P1 be the result of removing the outliers from point set P and Q1 be the result of removing the outliers from point set Q.

[0025] Step (5) of this invention involves performing histogram statistics on the Euclidean distances of all feature points, specifically including:

[0026] Taking the minimum value of the histogram as 0 pixels, the maximum value as max pixels, and the group interval as k pixels, we can see that there are a total of For each set of data, the Euclidean distance of all point pairs is plotted in a histogram. If the Euclidean distance is greater than the maximum value, the pair is placed in the last group. Specifically, first, it is determined whether the Euclidean distance *d* of the currently matched point pair is greater than the maximum value; if it is, it is placed in the last group. Group, otherwise put in the first group. Group the results and round them down to the nearest integer.

[0027] Step (6) of this invention involves data processing of the histogram, specifically including:

[0028] First, count the frequencies of the histogram and record the frequency of each group as . Where M h Let h represent the frequency of the h-th group, and then normalize the histogram. The histogram normalization formula is denoted as: Where X h M represents the normalized value of the h-th group. a M represents the normalized values ​​of the a-th group. hLet M represent the normalized values ​​of the h-th group, and denote the group with the highest histogram frequency as M. ε Furthermore, since all pairs of points with a Euclidean distance greater than the maximum are placed in the last group, and the distance in the last group is too large, the last group is directly considered as an exterior point. Therefore, at this point... A strategy of alternating selection is adopted, that is, M is selected first. ε Take M again ε-1 Take M again ε+1 The process continues until the sum of the frequencies of all the histogram data obtained is greater than 70%. The remaining data is considered to be incorrect matches, and the Euclidean distance index of the selected group is recorded and denoted as set N.

[0029] Step (7) of this invention involves removing feature point pairs using the histogram data processing results, specifically including:

[0030] Traverse and query the Euclidean distance set. If the element Dis1 is not in the set N, then the matching point pair is regarded as an outside point and the matching relationship is removed. Specifically, delete the corresponding element in set P1 and set Q1. Let the point set after removing point set P1 be P2 and the point set after removing point set Q1 be Q2.

[0031] The beneficial effects of this invention are as follows: It utilizes the boxplot method and histograms to further screen matched feature point pairs and eliminate mismatches; it extracts ORB feature points from images captured by the camera; it performs feature point matching between feature points in the current frame and feature points in keyframes; it calculates the Euclidean distance between matched feature point pairs; it uses the boxplot method to coarsely eliminate outliers based on the Euclidean distance; it performs histogram statistics on the Euclidean distances of all feature point pairs, calculates the highest score in the histogram, and uses this highest score as a benchmark. Matching point pairs near this benchmark are considered correct matching point pairs, while point pairs far from the benchmark are considered mismatches, and the mismatched point pairs are de-matched. This invention eliminates mismatches in feature point matching, improves the robustness of the SLAM system, enhances the accuracy of subsequent mapping and localization, reduces computational load and storage space, and improves system efficiency. Attached Figure Description

[0032] Figure 1 This is a flowchart of the present invention;

[0033] Figure 2 This is a comparison diagram of the trajectory in the three-dimensional coordinate system without using the original algorithm of this invention and the true trajectory in an example of this invention;

[0034] Figure 3 This is a comparison diagram of the trajectory after using the present invention in a three-dimensional coordinate system and the true trajectory in an example of the present invention. Detailed Implementation

[0035] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the drawings, not the entire structure.

[0036] like Figure 1 The flowchart of a feature point mismatch removal method applied to visual SLAM according to the present invention is shown below, and the specific steps are as follows:

[0037] S1. Extract ORB feature points from the images captured by the camera;

[0038] S2. Match the feature points of the current frame with the feature points of the key frame;

[0039] S3. Calculate the Euclidean distance between matching feature point pairs;

[0040] S4. Use the boxplot method to roughly remove outliers from the Euclidean distance.

[0041] S5. Perform histogram statistics on the Euclidean distance of all feature points;

[0042] S6. Process the histogram data;

[0043] S7. Based on the histogram statistics, remove the matching relationship from the mismatched point pairs;

[0044] In this embodiment, step S1 specifically includes the following:

[0045] Convert the acquired RGB image into a grayscale image;

[0046] A total of 1000 feature points were extracted using an 8-layer pyramid, where the i-th layer of the pyramid extracted: There are , where s is the scaling factor, and is taken as .

[0047] The FAST algorithm was used to extract feature points for each layer of the pyramid.

[0048] The direction vector of each feature point is calculated using the gray-scale centroid method.

[0049] Calculate the brief descriptor for each feature point and keep the template direction vector of the brief descriptor consistent with the direction vector of the feature point.

[0050] In the specific implementation process, after extracting the feature points using the FAST algorithm, the quadtree search method is used to search for the number of feature points to be extracted at each layer. When calculating the direction vector using the gray-scale centroid method, a circle with a radius of 12 is designed to calculate the centroid of the gray-scale pixel values ​​inside the circle. The line connecting the center and the centroid is used as the direction vector of the feature point. When calculating the brief descriptor, a 16*16 rectangular area is used as a template, and points are randomly selected within the template according to a fixed sequence. In this example, the sequence is a random point sequence that conforms to a Gaussian distribution.

[0051] In this embodiment, since only ORB feature points of a single image are extracted in step S1, feature point matching is performed in step S2 using feature points of the current frame and feature points of the keyframe, as follows:

[0052] Camera initialization and local map point creation were completed based on the ORB-SLAM2 framework.

[0053] The similarity of the brief descriptor is compared between the current frame pixel and the key frame pixel. In order to speed up the extraction, only the key frame pixel with the highest similarity to the current frame pixel with a radius of 20 is considered as the matching point pair.

[0054] Iterate through all the pixels to find the keyframe pixel that matches it.

[0055] In the specific implementation process, when comparing the brief descriptors of the current frame pixels with the key frame pixels, in order to improve the distinguishability, the best distance and the second best distance of similarity are recorded. Only when the best distance is less than the second best distance * 0.9 will it be considered whether it is a matching point pair.

[0056] In this embodiment, since there is a lack of screening and elimination of matching point pairs in step S1, the Euclidean distance of the matching feature point pairs is calculated in step S3 as follows:

[0057] Define a point set P = {p1, p2, ..., p...} that has already undergone feature matching. n}, Q={q1,q2,...,q n}, where P is the feature point set of the current frame, Q is the feature point set of the keyframe, n is the index of the feature point, and p is the element of the point set. n and q n For the matched feature points, the set of Euclidean distances between feature point pairs is denoted as Dis = {dis1, dis2, ..., dis...} n The formula is Dis = ||PQ|| 2 The Euclidean distance of each pair of feature points is calculated to obtain the set of Euclidean distances Dis for the matching pairs.

[0058] In this embodiment, in step S4, the boxplot method is used to roughly remove outliers from the Euclidean distance, as follows:

[0059] Arrange the data in set Dis in ascending order. Let Q1 be the lower quartile, Q2 be the median, Q3 be the upper quartile, and IQR be the interquartile range, where IQR = Q3 - Q1. Take the upper limit of the data as Q3 + 1.5IQR and the lower limit of the data as Q1 - 1.5IQR. That is, remove the data in set Dis that are not within Q1 - 1.5IQR to Q3 + 1.5IQR as outliers. Let Dis be the result of removing the data after removing the outliers. Then remove the corresponding data from point sets P and Q. Let P1 be the result of removing the outliers from point set P and Q1 be the result of removing the outliers from point set Q.

[0060] In this embodiment, step S5 involves performing histogram statistics on the Euclidean distances of all feature points, as detailed below:

[0061] The histogram is designed with a minimum value of 0 pixels, a maximum value of max pixels, and a group interval of k pixels. It is known that there are a total of... For each set of data, the Euclidean distance of all point pairs is plotted in a histogram. If the Euclidean distance is greater than the maximum value, the pair is placed in the last group. Specifically, first, it is determined whether the Euclidean distance *d* of the currently matched point pair is greater than the maximum value; if it is, it is placed in the last group. Group, otherwise put in the first group. Group the results and round them down to the nearest integer.

[0062] In this example, the maximum value max = 150 is selected, the class interval k = 10, and the number of classes is 10.

[0063] In this embodiment, step S6 involves data processing of the histogram, specifically as follows:

[0064] Count the frequencies of the histogram and record the frequency of each group as . Where M h This represents the frequency of the h-th group. The histogram is then normalized using the formula:

[0065] Where X h M represents the normalized value of the h-th group. a M represents the normalized values ​​of the a-th group. h The h-th group represents the normalized values, and the group with the highest histogram frequency is denoted as M. ε Furthermore, since all pairs of points with a Euclidean distance greater than the maximum are placed in the last group, and the distance in the last group is too large, the last group is directly considered as exterior points. Therefore, at this point... A strategy of alternating selection is adopted, that is, M is selected first. ε Take M againε-1 Take M again ε+1 The process continues until the sum of the frequencies of all the histogram data obtained is greater than 70%. The remaining data is considered to be incorrect matches, and the Euclidean distance index of the selected group is recorded and denoted as set N.

[0066] In this embodiment, in step S7, feature point pairs are removed using the histogram data processing results, as follows:

[0067] Traverse and query the Euclidean distance set. If the element Dis1 is not in the set N, then the matching point pair is regarded as an outside point and the matching relationship is removed. Specifically, delete the corresponding element in set P1 and set Q1. Let the point set after removing point set P1 be P2 and the point set after removing point set Q1 be Q2.

[0068] To better demonstrate the effectiveness of this invention, the original ORB-SLAM2 algorithm and the improved ORB-SLAM2 with this method were tested on an Ubuntu 20.04 system. The test hardware system consisted of a 12th Gen Intel Core CPU. TM For an i9-12900KF×24, the required third-party libraries are Pangolin, OpenCV, and Eigen3. The absolute pose error (APE) and relative pose error (RPE) are measured to evaluate the SLAM system error.

[0069] APE (Aspect-Oriented Position Error) is a metric for measuring the absolute attitude error of a SLAM (Simultaneous Localization and Mapping) system, typically calculated using Euclidean distance or attitude difference. In SLAM, APE is usually represented as the average distance error between all known locations on the map and their corresponding ground truth locations. A smaller APE indicates higher absolute positioning accuracy of the SLAM system.

[0070] RPE (Relative Position Error) is a metric that measures the relative attitude error of a SLAM (Simultaneous Localization and Mapping) system. It is typically calculated using methods such as difference vectors and angle differences. In SLAM, RPE is usually expressed as the error between the estimated pose and the true pose at adjacent time steps. The smaller the RPE, the higher the relative positioning accuracy of the SLAM system.

[0071] The specific test data is shown in Table 1. The data shows that the feature point mismatch removal method used in visual SLAM can improve the robustness and accuracy of the SLAM system.

[0072] Table 1. Comparison of APE Errors in Experiments

[0073]

[0074] Table 2. Comparison of RPE Errors in Experiments

[0075]

[0076] Figure 2 This is a comparison diagram of the trajectory without using the original algorithm of this invention in the three-dimensional coordinate system and the true trajectory in an example of this invention; Figure 3 This is a comparison diagram of the trajectory after using the present invention in a three-dimensional coordinate system and the true trajectory in an example of the present invention.

[0077] The dashed line represents the true pose, and the solid line represents the actual pose calculated by the algorithm. The figure clearly shows that the feature point mismatch removal method used in this invention for visual SLAM can improve pose estimation in the SLAM system, thereby enhancing the system's robustness and accuracy.

Claims

1. A method for removing mismatched feature points in visual SLAM, characterized in that, Includes the following steps: (1) Extract ORB feature points from the images captured by the camera; (2) Match the feature points of the current frame with the feature points of the key frame; (3) Calculate the Euclidean distance between the matching feature point pairs; (4) Use the boxplot method to roughly remove outliers from the Euclidean distance; (5) Perform histogram statistics on the Euclidean distances of the feature point pairs retained after the rough removal in step (4); (6) Process the histogram data; First, count the frequencies of the histogram and record the frequency of each group as . , Let h represent the frequency of the h-th group, and then normalize the histogram. The histogram normalization formula is denoted as: ,in This represents the normalized value of the h-th group. Let represent the frequency of group a, and denot the group with the highest frequency in the histogram as . All Euclidean distances greater than All point pairs are placed in the last group, and the last group is considered as exterior points. A cross-selection strategy is adopted, first selecting... Take again Take again The process continues until the sum of the frequencies of all the histogram data obtained is greater than 70%. The remaining data are considered incorrect matches, and the Euclidean distance indices of the selected groups are recorded, denoted as set . ; (7) Remove the mismatched point pairs from the matching relationship based on the histogram statistics.

2. The feature point mismatch removal method applied to visual SLAM according to claim 1, characterized in that, Step (1) involves extracting ORB feature points from the images captured by the camera, specifically including: The acquired RGB images were converted to grayscale images; a total of 1000 feature points were extracted using an 8-layer pyramid, where the number of feature points extracted from the i-th layer was: ； Where s is the scaling factor, taking... The FAST algorithm is used to extract feature points for each layer of the pyramid. The orientation vector of each feature point is calculated using the gray-scale centroid method, and the brief descriptor of each feature point is calculated, where the template orientation vector of the brief descriptor is consistent with the orientation vector of the feature point.

3. The feature point mismatch removal method applied to visual SLAM according to claim 1, characterized in that, Step (2) involves matching the feature points of the current frame with the feature points of the keyframe, specifically including: Camera initialization and local map point creation are completed based on the ORB-SLAM2 algorithm framework. The similarity of the brief descriptor is compared between the current frame pixel and the key frame pixel. In order to accelerate the extraction, only the pixel region with a radius of 20 with the current frame pixel is considered, and the key frame pixel with the highest similarity is selected as the matching point pair. All pixels are traversed in a loop to find the matching key frame pixel.

4. The feature point mismatch removal method applied to visual SLAM according to claim 1, characterized in that, Step (3) calculates the Euclidean distance between the matching feature point pairs, specifically including: Define the set of points for which feature matching has been completed. , Where P is the feature point set of the current frame, Q is the feature point set of the keyframe, n is the index of the feature point, and the element of the point set is... and For the matched feature points, the set of Euclidean distances between feature point pairs is denoted as . The formula is That is, calculating the Euclidean distance of each pair of feature points to obtain the set of Euclidean distances for the matching pairs. .

5. The feature point mismatch removal method applied to visual SLAM according to claim 1, characterized in that, Step (4) uses the boxplot method to roughly remove outliers from the Euclidean distance, specifically including: set The data are arranged in ascending order, and the lower quartile of the data is denoted as . The median is The upper quartile is Interquartile range is ,in The upper limit of the data is The lower limit of the data is Soon to gather Not in the middle Data within the range is discarded as outgoing points. After removing the data And remove the corresponding data from point sets P and Q, and denote the point set after removing the data from point set P as . The point set after removing point set Q is .

6. The feature point mismatch removal method applied to visual SLAM according to claim 1, characterized in that, Step (5) involves performing histogram statistics on the Euclidean distances of all feature points, specifically including: The minimum value of the histogram is 0 pixels, and the maximum value is... Pixels, group interval of k pixels, it can be known that there are a total of Take a set of data and plot the Euclidean distances of all point pairs in a histogram. If their Euclidean distances are greater than 1, then... If it is found, it is placed in the last group. Specifically, first, it is determined whether the Euclidean distance d of the current matched point pair is greater than 1. If it is larger, then put it in. Group, otherwise put in the first group. Group the results and round them down to the nearest integer.

7. The feature point mismatch removal method applied to visual SLAM according to claim 1, characterized in that, Step (7) involves using the histogram data processing results to remove feature point pairs, specifically including: Iterate through the Euclidean distance set, if If an element is not in set N, then the matching pair is considered an outside point, and the matching relationship is terminated; specifically, the element is deleted from the set. and set The corresponding elements in the set of points are recorded. The point set after removal is dot set The point set after removal is .

Images