A ORB feature extraction method based on straight line guidance and adaptive threshold

Abstract

The present invention discloses an ORB feature extraction method based on straight line guidance and adaptive threshold, which comprises the following steps: I) extracting line segment features in an image; II) dividing the i-th line segment into pre-selected points at equal intervals along the length direction of the i-th line segment; III) calculating the adaptive threshold for ORB feature extraction for each pre-selected point; IV) extracting ORB features for each pre-selected point using the feature point extraction algorithm in ORBSLAM according to the adaptive threshold. The ORB feature extraction method based on straight line guidance and adaptive threshold of the present invention can not only extract more feature points at the end points of line segments, intersections between line segments, and areas with rich textures, but also can extract more feature points in the middle of line segments and areas with low textures compared to the existing ORBSLAM algorithm, so that the number of feature points is larger and the distribution is more reasonable.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method for extracting feature points in an image. Background Art

[0002] Image recognition refers to the technology of using computers to process, analyze and understand images to identify targets and objects of various different patterns. It is a practical application of deep learning algorithms. Image recognition is an important field of artificial intelligence. Feature extraction is an important step in the image recognition process, and corner point features are important features in image recognition.

[0003] Specifically in the field of real-time positioning and map creation technology, ORBSLAM is a commonly used real-time positioning and mapping algorithm, and ORB feature extraction is an important step. However, in existing methods, most of the feature points extracted by the feature point extraction algorithm in ORBSLAM are located at the endpoints of line segments, intersections between line segments, and areas with rich textures, while fewer feature points are extracted in the middle of line segments and low-texture areas. The distribution of feature points is not reasonable, which will lead to large errors in estimating the pose of two adjacent frames of images using the feature point extraction algorithm in ORBSLAM in the application of building three-dimensional maps. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide an ORB feature extraction method based on straight line guidance and adaptive threshold to solve the technical problem that when extracting feature points in an image using the existing ORBSLAM, most of the feature points are located at the endpoints of line segments, intersections between line segments, and areas with rich textures, while fewer feature points are extracted in the middle of the line segments and low-texture areas, and the distribution of feature points is not reasonable.

[0005] The ORB feature extraction method based on straight line guidance and adaptive threshold of the present invention comprises the following steps:

[0006] I) extracting line segment features in the image;

[0007] II) Divide the i-th line segment into pre-selected points at equal intervals along the length direction of the i-th line segment. The formula for determining the division distance is:

[0008]

[0009] Among them, d i is the division distance corresponding to the i-th line segment; n is the number of feature points to be extracted; l i is the length of the i-th line segment, l j is the length of the jth line segment; m is the total number of line segments in this image;

[0010] III) Adaptive threshold calculation for ORB feature extraction is performed on each pre-selected point. The threshold calculation formula is:

[0011]

[0012] Among them, λ is the threshold corresponding to each feature point, λ max is the maximum value of the threshold, λ min is the minimum threshold value set, and x is the distance between the feature point and the left endpoint of the line segment;

[0013] IV) ORB features are extracted from each pre-selected point using the feature point extraction algorithm in ORBSLAM according to the adaptive threshold.

[0014] Furthermore, extracting the line segment features in the image in step I) further comprises the steps of:

[0015] 1) Perform Gaussian filtering on the image to smooth the image;

[0016] 2) Calculate the gradient direction and gradient magnitude of each pixel in the image;

[0017] 3) Traverse each pixel and select the pixel whose gradient value is greater than or equal to the gradient value of the adjacent pixel in the gradient direction, which is defined as the anchor point.

[0018] 4) Connect the anchor points to form an edge pixel chain;

[0019] 5) For each edge pixel chain, the PROSAC method is used to remove noise points and fit the line segments, which includes the following steps:

[0020] a) Select two pixels from the edge pixel chain as the starting point and end point of the line segment, and calculate a straight line based on the selected two pixels;

[0021] b) Calculate the weighted distances of other pixels in the edge pixel chain to the straight line calculated in step a) and use the weighted distances in d H The pixels within the range are regarded as inliers to form an inlier set; where d H is a given threshold;

[0022] d i =αd i1 +βd i2 (3)

[0023]

[0024] d i2 =cos(θ1-θ2) (5)

[0025] Among them, d i is the weighted distance between the pixel and the line, d i1 is the pixel (xi ,y i ) to the straight line y = kx + b, k is the slope of the line, b is the intercept of the line, d i2 is the pixel (x i ,y i ) The unit vector distance between the gradient direction θ1 and the normal direction θ2 of the line y=kx+b, α and β are weight factors;

[0026] c) An iterative process is performed with the maximum number of inliers as the indicator. In each iteration, two pixels used to determine the new straight line are selected from the inlier set obtained in the previous iteration, until the ratio of inliers to all pixels in the pixel chain exceeds the ratio threshold τ or the number of iterations num is reached;

[0027] d) taking the line segment obtained in the last iteration as the final result and outputting the extracted line segment equation;

[0028] 6) Screening and merging the line segments obtained in step 5), which includes the following steps:

[0029] A) Randomly select a line segment in the image as the main line segment, and then expand a rectangle with a length of l+Δl and a width of Δw. The main line segment is located at the center of the rectangle, and the long side of the rectangle is parallel to the main line segment, where l is the length of the main line segment, Δl and Δw are the set thresholds, and the line segments in the rectangle and intersecting with the rectangle are screened by vector similarity. The line segments that meet the vector similarity screening formula are put into the set to obtain a line segment set; the vector similarity screening formula is:

[0030]

[0031]

[0032]

[0033] cosθ ≥ cosθ min (9)

[0034] in is the vector representation of the main line segment l1, is the vector representation of the line segment l2 to be merged; the coordinates of the two endpoints of the main line segment l1 are (x 1a ,y 1a )、(x 1b ,y 1b ), the coordinates of the two endpoints of the line segment l2 to be merged are (x 2a ,y 2a )、(x 2b ,y 2b ), θ is the angle between the main line segment l1 and the line segment l2 to be merged, θ min is the set angle threshold;

[0035] B) Sort the line segments in the line segment set in ascending order according to their distance from the main line segment. The distance measurement formula is:

[0036]

[0037]

[0038]

[0039] in, is the coordinate of the midpoint p of the line segment to be merged, d p is the Euclidean distance from the midpoint p to the main line segment y=k1x+b1, k1 is the slope of the line, and b1 is the intercept of the line;

[0040] C) Select the line segments in ascending order in step B) as the line segments to be merged, project the line segments to be merged onto the straight line containing the main line segment, obtain the projected line segments of the line segments to be merged, and then merge the line segments according to the following situations:

[0041] Case 1: When the two endpoints of the projection segment are outside the main segment and the two endpoints of the projection segment are on the same side of the main segment, the distance between the two adjacent endpoints of the main segment and the projection segment is calculated. If the distance between the two adjacent endpoints is less than the set threshold Then, among the four endpoints, the two endpoints of the main segment and the two endpoints of the projection segment, the segment determined by the two endpoints with the largest distance is taken as the new segment after the main segment and the segment to be merged are merged; if the distance between two adjacent endpoints is greater than the set threshold No line segment merging is performed;

[0042] Case 2: When the two endpoints of the projected line segment are outside the two ends of the main line segment, the projected line segment is taken as the new line segment after the main line segment and the line segment to be merged are merged;

[0043] Case 3: When one endpoint of the projected line segment is on the main line segment, and the other endpoint of the projected line segment is outside the main line segment, then the line segment determined by the two endpoints with the largest distance between the two endpoints of the main line segment and the two endpoints of the projected line segment is taken as the new line segment after the main line segment and the line segment to be merged are merged;

[0044] Case 4: When both endpoints of the projected line segment are on the main line segment, the main line segment is taken as the new line segment after the main line segment and the line segment to be merged are merged;

[0045] D) Repeat steps A) to C) to complete the merging of the line segments in the image.

[0046] Beneficial effects of the present invention:

[0047] 1. The ORB feature extraction method based on straight line guidance and adaptive threshold of the present invention can not only extract more feature points at the endpoints of line segments, intersections between line segments, and areas with rich textures, but also can extract more feature points in the middle of line segments and low-texture areas compared with the feature point extraction algorithm in the existing ORBSLAM, making the distribution of feature points more reasonable.

[0048] 2. The ORB feature extraction method based on straight line guidance and adaptive threshold of the present invention has a disclosed straight line feature extraction method which extracts fewer short line segments at non-feature locations and has a stronger ability to resist noise infection compared with the existing EDLines; and after screening and merging, the broken line segments are merged into longer line segments, and the restored line segments are more consistent with the actual ones, which further improves the accuracy of subsequent ORB feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The effect diagram of dividing the pre-selected points equally along the length of the line segment;

[0050] Figure 2 is the relationship between the adaptive threshold λ and the distance x between the feature point and the left endpoint of the line segment;

[0051] Figure 3 This is an effect diagram of extracting feature points in image A using the feature point extraction algorithm in the existing ORBSLAM;

[0052] Figure 4 This is a diagram showing the effect of extracting feature points in image A using the method in the embodiment;

[0053] Figure 5 This is an effect diagram of extracting feature points in image B using the feature point extraction algorithm in the existing ORBSLAM;

[0054] Figure 6 This is a diagram showing the effect of extracting feature points in image B using the method in the embodiment;

[0055] Figure 7 This is the effect diagram of the straight line features extracted by the original EDLines line feature extraction algorithm;

[0056] Figure 8 This is the effect diagram of extracting straight line features using the improved EDLines straight line extraction method based on PROSAC to remove noise points;

[0057] Fig. 9 It is a schematic diagram of merging the two endpoints of the projection line segment with the line segment outside one end of the main line segment;

[0058] Fig.10 It is a schematic diagram that the two endpoints of the projection line segment are not merged with the line segment outside one end of the main line segment;

[0059] Fig.11 It is a schematic diagram of merging the line segments whose two endpoints of the projection line segment are outside the two ends of the main line segment;

[0060] Fig.12 A schematic diagram of merging a line segment in which one endpoint of a projection line segment is on the main line segment and the other endpoint is outside the main line segment;

[0061] Fig.13 This is a schematic diagram of merging line segments where both endpoints of the projection line segment are on the main line segment;

[0062] Fig.14 This is the effect diagram of the straight line features extracted by the improved EDLines straight line extraction method based on PROSAC and screening;

[0063] Fig.15 Flowchart of the ORB feature extraction method based on straight line guidance and adaptive threshold. DETAILED DESCRIPTION

[0064] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0065] The ORB feature extraction method based on straight line guidance and adaptive threshold in this embodiment includes the following steps:

[0066] I) Extract line segment features from the image.

[0067] II) Divide the i-th line segment into pre-selected points at equal intervals along the length direction of the i-th line segment. The formula for determining the division distance is:

[0068]

[0069] Among them, d i is the division distance corresponding to the i-th line segment; n is the number of feature points to be extracted; l i is the length of the i-th line segment, l j is the length of the jth line segment; m is the total number of line segments in this image.

[0070] III) Adaptive threshold calculation for ORB feature extraction is performed on each pre-selected point. The threshold calculation formula is:

[0071]

[0072] Among them, λ is the threshold corresponding to each feature point, λ max is the maximum value of the threshold, λ min is the minimum threshold value set, and x is the distance between the feature point and the left endpoint of the line segment.

[0073] from Figure 2From the relationship diagram of the adaptive threshold λ and the distance x of the feature point from the left end point of the line segment, we can see that the feature points near the two ends of the line segment are not only few in number but also easily disappear in the next frame as the camera moves, while the feature points near the middle of the line segment are more numerous and relatively stable during camera movement. Therefore, we hope to propose more feature points near the middle of the line segment. The lower the threshold, the looser the feature point extraction conditions, and thus more feature points will be extracted. Therefore, by adaptively lowering the threshold of feature points near the middle of the line segment, we can achieve the purpose of extracting ORB features from a line segment in a more balanced manner.

[0074] IV) ORB features are extracted from each pre-selected point using the feature point extraction algorithm in ORBSLAM according to the adaptive threshold.

[0075] from Figure 4 and Figure 3 The comparison and Figure 6 and Figure 5 It can be seen from the comparison that the ORB feature extraction method based on straight line guidance and adaptive threshold in this embodiment can not only extract more feature points at the endpoints of line segments, intersections between line segments, and areas with rich textures, but also can extract more feature points in the middle of line segments and low-texture areas compared to the feature point extraction algorithm in the existing ORBSLAM, so that the number of feature points is larger and the distribution is more reasonable.

[0076] As an improvement to the above embodiment, the step I) of extracting line segment features in the image further comprises the steps of:

[0077] 1) Perform Gaussian filtering on the image to smooth the image.

[0078] 2) Calculate the gradient direction and gradient magnitude of each pixel in the image.

[0079] 3) Traverse each pixel and select pixels whose gradient values ​​in the gradient direction are greater than or equal to the gradient values ​​of adjacent pixels, and define them as anchor points.

[0080] 4) Connect the anchor points to form edge pixel chains.

[0081] 5) For each edge pixel chain, the PROSAC method is used to remove noise points and fit the line segments, which includes the following steps:

[0082] a) Select any two pixels from the edge pixel chain as the starting point and end point of the line segment, and calculate a straight line based on the selected two pixels.

[0083] b) Calculate the weighted distances of other pixels in the edge pixel chain to the straight line calculated in step a) and use the weighted distances in d H The pixels within the range are regarded as inliers to form an inlier set; where d H is a given threshold;

[0084] d i =αd i1 +βd i2 (1)

[0085]

[0086] d i2 =cos(θ1-θ2) (3)

[0087] Among them, d i is the weighted distance between the pixel and the line, d i1 is the pixel (x i ,y i ) to the straight line y = kx + b, k is the slope of the line, b is the intercept of the line, d i2 is the pixel (x i ,y i ) The unit vector distance between the gradient direction θ1 and the normal direction θ2 of the line y=kx+b, α and β are weight factors;

[0088] c) The iteration process is performed with the maximization of the number of inliers as the indicator. In each iteration, the two pixels used to determine the new straight line are selected from the inlier set obtained in the previous iteration, until the ratio of inliers to all pixels in the pixel chain exceeds the ratio threshold τ or the number of iterations num is reached.

[0089] d) Take the line segment obtained in the last iteration as the final result and output the extracted line segment equation.

[0090] Figure 7 This is the effect diagram of the straight line features extracted by the original EDLines line feature extraction algorithm; Figure 8 This is the effect diagram of extracting straight line features based on the improved EDLines straight line extraction method based on PROSAC to remove noise points; by comparing Figure 8 and Figure 7 It can be seen that the original EDLines algorithm will extract more short straight lines at non-linear features under the interference of noise, while the method in this embodiment will extract very few short straight lines at non-linear features. The method disclosed in this embodiment has stronger anti-noise interference ability.

[0091] 6) Screening and merging the line segments obtained in step 5), which includes the following steps:

[0092] A) Randomly select a line segment in the image as the main line segment, and then expand a rectangle with a length of l+Δl and a width of Δw. The main line segment is located at the center of the rectangle, and the long side of the rectangle is parallel to the main line segment. l is the length of the main line segment, and Δl and Δw are the set thresholds. After testing, Δl is set to 5 pixels for better results, and Δw is set to 3 pixels for better results. Of course, in the specific implementation, Δl and Δw can be adjusted according to actual conditions. Use vector similarity to filter the line segments in the rectangle and those intersecting with the rectangle, and put the line segments that meet the vector similarity screening formula into the set to obtain a line segment set; the vector similarity screening formula is:

[0093]

[0094]

[0095]

[0096] cosθ ≥ cosθ min (7)

[0097] in is the vector representation of the main line segment l1, is the vector representation of the line segment l2 to be merged; the coordinates of the two endpoints of the main line segment l1 are (x 1a ,y 1a )、(x 1b ,y 1b ), the coordinates of the two endpoints of the line segment l2 to be merged are (x 2a ,y 2a )、(x 2b ,y 2b ), θ is the angle between the main line segment l1 and the line segment l2 to be merged, θ min is the set angle threshold;

[0098] B) Sort the line segments in the line segment set in ascending order according to their distance from the main line segment. The distance measurement formula is:

[0099]

[0100]

[0101]

[0102] in, is the coordinate of the midpoint p of the line segment to be merged, d p is the Euclidean distance from the midpoint p to the main line segment y=k1x+b1, k1 is the slope of the line, and b1 is the intercept of the line;

[0103] C) Select the line segments in ascending order in step B) as the line segments to be merged, project the line segments to be merged onto the straight line containing the main line segment, obtain the projected line segments of the line segments to be merged, and then merge the line segments according to the following situations:

[0104] Case 1: When the two endpoints of the projection segment are outside the main segment and the two endpoints of the projection segment are on the same side of the main segment, the distance between the two adjacent endpoints of the main segment and the projection segment is calculated. If the distance between the two adjacent endpoints is less than the set threshold Then, among the four endpoints, the two endpoints of the main segment and the two endpoints of the projection segment, the segment determined by the two endpoints with the largest distance is taken as the new segment after the main segment and the segment to be merged are merged; if the distance between two adjacent endpoints is greater than the set threshold No line segment merging is performed;

[0105] Case 2: When the two endpoints of the projected line segment are outside the two ends of the main line segment, the projected line segment is taken as the new line segment after the main line segment and the line segment to be merged are merged;

[0106] Case 3: When one endpoint of the projected line segment is on the main line segment, and the other endpoint of the projected line segment is outside the main line segment, then the line segment determined by the two endpoints with the largest distance between the two endpoints of the main line segment and the two endpoints of the projected line segment is taken as the new line segment after the main line segment and the line segment to be merged are merged;

[0107] Case 4: When both endpoints of the projected line segment are on the main line segment, the main line segment is taken as the new line segment after the main line segment and the line segment to be merged are merged;

[0108] D) Repeat steps A) to C) to complete the merging of the line segments in the image.

[0109] Fig.14 This is the effect diagram of extracting straight line features by the improved EDLines straight line extraction method based on PROSAC and screening merging. Fig.14 and Figure 8 It can be seen that after the line segment screening and merging process in this embodiment, the broken line segments are merged into longer line segments, and the restored line segments are consistent with the actual ones, which improves the accuracy of line segment extraction. Fig.14 and Figure 7 By comparison, it can be clearly seen that compared with the original EDLines, the straight line features extracted by the method disclosed in this embodiment are more consistent with the actual situation, and the straight line feature extraction accuracy is higher.

[0110] This improved ORB feature extraction method based on straight line guidance and adaptive threshold, compared with the existing EDLines, the straight line feature extraction method disclosed by the invention extracts fewer short line segments at non-feature locations and has a stronger ability to resist noise infection; and after screening and merging, the broken line segments are merged into longer line segments, and the restored line segments have a better consistency with the actual ones, which further improves the accuracy of subsequent ORB feature extraction. Of course, in different embodiments, the line segment feature extraction method for extracting line segments from the image in step I) can also be implemented using existing algorithms such as the original EDLines.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, and should be included in the scope of the claims of the present invention.

Claims

1. An ORB feature extraction method based on straight line guidance and adaptive threshold, characterized in that: Includes steps: I) extracting line segment features in the image; II) Divide the i-th line segment into pre-selected points at equal intervals along the length direction of the i-th line segment. The formula for determining the division distance is: Among them, d i is the division distance corresponding to the i-th line segment; n is the number of feature points to be extracted; l i is the length of the i-th line segment, l j is the length of the jth line segment; m is the total number of line segments in this image; III) Adaptive threshold calculation for ORB feature extraction is performed on each pre-selected point. The threshold calculation formula is: Among them, λ is the threshold corresponding to each feature point, λ max is the maximum value of the threshold, λ min is the minimum threshold value set, and x is the distance between the feature point and the left endpoint of the line segment; IV) ORB features are extracted from each pre-selected point using the feature point extraction algorithm in ORBSLAM according to the adaptive threshold.

2. The ORB feature extraction method based on straight line guidance and adaptive threshold according to claim 1 is characterized in that: The step (1) of extracting line segment features in the image further comprises the steps of: 1) Perform Gaussian filtering on the image to smooth the image; 2) Calculate the gradient direction and gradient magnitude of each pixel in the image; 3) Traverse each pixel and select the pixel whose gradient value is greater than or equal to the gradient value of the adjacent pixel in the gradient direction, and define it as the anchor point; 4) Connect the anchor points to form an edge pixel chain; 5) For each edge pixel chain, the PROSAC method is used to remove noise points and fit the line segments, which includes the following steps: a) Select two pixels from the edge pixel chain as the starting point and end point of the line segment, and calculate a straight line based on the selected two pixels; b) Calculate the weighted distances of other pixels in the edge pixel chain to the straight line calculated in step a) and use the weighted distances in d H The pixels within the range are regarded as inliers to form an inlier set; where d H is a given threshold; d i =αd i1 +βd i2 (3) d i2 =cos(θ1-θ2) (5) Among them, d i is the weighted distance between the pixel and the line, d i1 is the pixel (x i ,y i ) to the straight line y = kx + b, k is the slope of the line, b is the intercept of the line, d i2 is the pixel (x i ,y i ) The unit vector distance between the gradient direction θ1 and the normal direction θ2 of the line y=kx+b, α and β are weight factors; c) An iterative process is performed with the maximum number of inliers as the indicator. In each iteration, two pixels used to determine the new straight line are selected from the inlier set obtained in the previous iteration, until the ratio of inliers to all pixels in the pixel chain exceeds a ratio threshold τ or the number of iterations num is reached; d) taking the line segment obtained in the last iteration as the final result and outputting the extracted line segment equation; 6) Screening and merging the line segments obtained in step 5), which includes the following steps: A) Randomly select a line segment in the image as the main line segment, and then expand a rectangle with a length of l+Δl and a width of Δw. The main line segment is located at the center of the rectangle, and the long side of the rectangle is parallel to the main line segment, where l is the length of the main line segment, Δl and Δw are the set thresholds, and the line segments in the rectangle and intersecting with the rectangle are screened by vector similarity. The line segments that meet the vector similarity screening formula are put into the set to obtain a line segment set; the vector similarity screening formula is: cosθ≥cosθ min (9) in is the vector representation of the main line segment l1, is the vector representation of the line segment l2 to be merged; the coordinates of the two endpoints of the main line segment l1 are (x 1a ,y 1a )、(x 1b ,y 1b ), the coordinates of the two endpoints of the line segment l2 to be merged are (x 2a ,y 2a )、(x 2b ,y 2b ), θ is the angle between the main line segment l1 and the line segment l2 to be merged, θ min is the set angle threshold; B) Sort the line segments in the line segment set in ascending order according to their distance from the main line segment. The distance measurement formula is: in, is the coordinate of the midpoint p of the line segment to be merged, d p is the Euclidean distance from the midpoint p to the main line segment y=k1x+b1, k1 is the slope of the line, and b1 is the intercept of the line; C) Select the line segments in ascending order in step B) as the line segments to be merged, project the line segments to be merged onto the straight line containing the main line segment, obtain the projected line segments of the line segments to be merged, and then merge the line segments according to the following situations: Case 1: When the two endpoints of the projection segment are outside the main segment and the two endpoints of the projection segment are on the same side of the main segment, the distance between the two adjacent endpoints of the main segment and the projection segment is calculated. If the distance between the two adjacent endpoints is less than the set threshold Then, among the four endpoints, the two endpoints of the main segment and the two endpoints of the projection segment, the segment determined by the two endpoints with the largest distance is taken as the new segment after the main segment and the segment to be merged are merged; if the distance between two adjacent endpoints is greater than the set threshold No line segment merging is performed; Case 2: When the two endpoints of the projected line segment are outside the two ends of the main line segment, the projected line segment is taken as the new line segment after the main line segment and the line segment to be merged are merged; Case 3: When one endpoint of the projected line segment is on the main line segment, and the other endpoint of the projected line segment is outside the main line segment, then the line segment determined by the two endpoints with the largest distance between the two endpoints of the main line segment and the two endpoints of the projected line segment is taken as the new line segment after the main line segment and the line segment to be merged are merged; Case 4: When both endpoints of the projected line segment are on the main line segment, the main line segment is taken as the new line segment after the main line segment and the line segment to be merged are merged; D) Repeat steps A) to C) to complete the merging of the line segments in the image.

Images