Vision slam image line segment extraction method based on approximate line segment merging

Abstract

The application is a visual SLAM image line segment extraction method based on approximate line segment merging. Firstly, the CannyLines algorithm is used to perform edge detection on the image to obtain all pixel chains. Then, the RANSAC algorithm is used to perform line fitting on each pixel chain to extract line segments in the image. Finally, approximate line segments in all line segments are judged and merged. For the new line segment after merging, if the angle difference between the new line segment and the long line segment before merging is within a reasonable range, the new line segment after merging is retained; otherwise, the new line segment after merging is not retained, and the two similar line segments before merging are reset to continue traversing the remaining line segments. The method uses the approximate line segment merging strategy to merge the broken short line segments into longer line segments, and merge the approximate line segments into one line segment, thereby reducing the short and redundant line segments, improving the feature matching accuracy in the visual SLAM process, and reducing the time consumption of feature matching.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and analysis in visual SLAM, and particularly relates to a visual SLAM image line segment extraction method based on approximate line segment merging. TECHNICAL BACKGROUND

[0002] Line extraction is an important part of image segmentation task, and has a wide range of applications in multi-target tracking, robot simultaneous localization and mapping (SLAM), road extraction, etc. Line features reflect important geometric information of images, especially in man-made scenes composed of many man-made objects, line features reflect important environmental information, so line feature extraction is also an important feature information for analyzing the environment or the object to be measured. Commonly used line extraction methods, such as CannyLines algorithm, first perform edge detection on the image, then determine the line points of the straight line by judging the relationship between adjacent pixel points of the edge, and finally further optimize the fitting straight line according to the line points. However, these methods have a large number of short lines or redundant lines.

[0003] In the process of visual SLAM, the camera is used to collect environmental images, and appropriate line feature information is extracted from the environmental images for feature matching to obtain the pose of the robot, so as to realize the autonomous positioning and navigation of the robot. Line feature extraction is an important prerequisite for realizing feature matching, and determines the accuracy of pose estimation, while short lines increase the instability of feature extraction, leading to feature matching errors, for example, short lines existing in the current frame image may disappear in the next frame image, causing the extracted line features in each frame image to be unstable and prone to feature matching errors. In addition, line redundancy increases the complexity of feature matching calculation, leading to long feature matching time, and thus affecting the real-time performance of robot simultaneous localization and mapping.

[0004] In order to solve the problems existing in line feature extraction in visual SLAM, the present application provides a visual SLAM image line segment extraction method based on approximate line segment merging, which merges broken line segments or approximate line segments into long line segments through approximate line segment merging, avoiding short and redundant lines. SUMMARY

[0005] In view of the deficiencies of the prior art, the technical problem to be solved by the present application is to provide a visual SLAM image line segment extraction method based on approximate line segment merging.

[0006] The technical solution adopted by the present application to solve the technical problem is as follows:

[0007] A visual SLAM image line segment extraction method based on approximate line segment merging, characterized in that the method comprises the following steps:

[0008] The first step is to perform edge detection on the image by using the CannyLines algorithm to obtain all pixel chains.

[0009] The second step is to perform line fitting on each pixel chain by using the RANSAC algorithm to extract line segments in the image.

[0010] The third step is to judge the approximate line segments in all line segments and perform approximate merging.

[0011] 3.1 Calculate the angle of each line segment.

[0012] 3.2 Calculate the angle difference between any two line segments, if the angle difference is less than or equal to the first angle deviation threshold, then calculate the shortest distance between the two line segments, if the shortest distance is less than or equal to the first distance threshold, consider the two line segments as similar line segments, then execute steps 3.3 and 3.4, otherwise repeat this step to traverse the remaining line segments.

[0013] 3.3 Two approximate line segments are denoted as La and Lb, the two endpoints of line segment La are a1 and a2, the two endpoints of line segment Lb are b1 and b2; assume that line segment La is the shorter line segment and line segment Lb is the longer line segment; calculate the included angle of vectors b1a1 and b1b2 and the included angle of vectors b2a2 and b2b1, if both included angles are acute angles, it indicates that the two endpoints of the shorter line segment La can be projected onto the longer line segment Lb, then the longer line segment Lb is taken as the new line segment after merging, and the shorter line segment La is deleted, completing the merging of line segments La and Lb; if one of the included angles is not an acute angle, then calculate the Euclidean distance between any two endpoints of the two line segments La and Lb, if the shortest Euclidean distance is less than or equal to the second distance threshold, then the two endpoints corresponding to the maximum Euclidean distance are taken as the two endpoints of the new line segment, obtaining the new line segment after merging; if the shortest Euclidean distance is greater than the second distance threshold, then no merging is performed.

[0014] 3.4 Calculate the angle of the new line segment, if the angle difference between the new line segment and the longer line segment before merging is less than or equal to the second angle deviation threshold, consider that the merging deviation is within a reasonable range, then retain the new line segment after merging; otherwise, do not retain the new line segment after merging, reset the line segments La and Lb and return to step 3.2 to continue traversing the remaining line segments.

[0015] Further, in the second step, for the pixel chain, calculate the Manhattan distance of all pixel points to the origin of the image coordinate, and determine the distance threshold t for judging the in-point according to the following formula:

[0016]

[0017] Wherein, a is an adjustable coefficient, M max , M min are the maximum and minimum of Manhattan distance respectively.

[0018] Further, in step 3.2, the distance between the two ends of a line segment to another line segment is calculated, and the smaller distance is the shortest distance between the two line segments.

[0019] Compared with the prior art, the present application has the following beneficial effects:

[0020] 1. The present application combines the broken short line segments into longer line segments based on the approximate line segment merging strategy, and combines the approximate line segments into a line segment, thereby avoiding the short and redundant straight lines extracted by the CannyLines algorithm, improving the accuracy of feature matching in the visual SLAM process, reducing the time consumption of feature matching, and ensuring the real-time performance of the robot simultaneous localization and mapping.

[0021] 2. The RANSAC algorithm is used to replace the least square method of the original CannyLines algorithm for line fitting, thereby reducing the deviation of the fitted straight line under the influence of noise, improving the anti-noise interference ability of the CannyLines algorithm, and improving the accuracy of the straight line feature extraction. BRIEF DESCRIPTION OF DRAWINGS

[0022] Fig. 1 is the overall flowchart of the method of the present application;

[0023] Fig. 2 is the result graph obtained by the original CannyLines algorithm;

[0024] Fig. 3 is the result graph obtained by the method of the present application. DETAILED DESCRIPTION

[0025] The specific embodiments will be described below in conjunction with the drawings, which are only used to specifically introduce the technical scheme of the present application, and do not limit the protection scope of the present application.

[0026] The present application is a visual SLAM image line segment extraction method based on approximate line segment merging (referred to as method, see Figs. 1-3 ), comprising the following steps:

[0027] Firstly, the CannyLines algorithm is used to detect the edges of the image to obtain all pixel chains, and all pixel points of each edge form a pixel chain;

[0028] Secondly, the RANSAC algorithm is used to fit the straight line of each pixel chain to extract the line segments in the image;

[0029] Randomly select two pixels from the pixel chain, calculate the distance between the two pixels, if the distance is less than or equal to the threshold S, it indicates that the distance between the two pixels is close, then reselect two pixels, until the distance between the two pixels is greater than the distance threshold S, obtain the fitting straight line according to the selected two pixels;

[0030] Calculate the Manhattan distance of all pixels in the pixel chain to the origin of the image coordinate, determine the distance threshold t for judging the inlier according to the maximum and minimum values of the Manhattan distance, see formula (1):

[0031]

[0032] In the formula, α is an adjustable coefficient, M max , M min are the maximum and minimum values of the Manhattan distance respectively;

[0033] Traverse all the remaining pixels in the pixel chain, calculate the distance of each pixel to the fitting straight line according to formula (2), if the Euclidean distance is less than the distance threshold t, it is considered that the pixel is an inlier, otherwise it is an outlier, obtain the inlier set, and complete the first iteration;

[0034]

[0035] In the formula, d m represents the distance of pixel m to the fitting straight line y = ax + b, (x m , y m ) is the coordinate of pixel m;

[0036] Randomly select two pixels from the inlier set to obtain a new fitting straight line, calculate the distance of the remaining pixels in the pixel chain to the fitting straight line, and update the inlier set; repeat the iteration N times to maximize the number of inliers, and obtain the final inlier set; the iteration number N is calculated as shown in formula (3):

[0037]

[0038] In the formula, z is the probability of obtaining a high-quality sample, which is set to 99%; w is the proportion of inliers in the pixel chain point set, and n is the minimum number of pixel points required for model parameter estimation;

[0039] Perform straight line fitting on the final inlier set to obtain a line segment; traverse all the pixel chains to obtain all the line segments;

[0040] Calculate the length of each line segment according to formula (4), and filter out short line segments according to formula (5) using the Helmholtz principle, and keep the remaining line segments;

[0041]

[0042]

[0043] wherein, l i is the length of line segment i, (x1, y1) and (x2, y2) are the coordinates of the two end points of line segment i, K is the image size, and p is a constant, generally taking a value of 0.125;

[0044] Step 3, approximate line segment merging;

[0045] 3.1 Calculate the angle of each line segment obtained in Step 2 according to formula (6);

[0046]

[0047] wherein, θ i is the angle of line segment i;

[0048] 3.2 Calculate the angle difference between any two line segments La and Lb. If the angle difference is less than or equal to a first angle deviation threshold (for example, taking a value of π / 12), calculate the shortest distance between the two line segments. According to the relationship between the shortest distance between the two line segments and a first distance threshold (for example, taking a value of 2), determine whether the two line segments are approximate line segments. If the two line segments are approximate line segments, perform Steps 3.3 and 3.4. Otherwise, repeat this step to traverse the remaining line segments;

[0049] Calculate the distance from the two end points of one line segment La to the other line segment Lb according to formula (7). The smaller distance is the shortest distance between the two line segments;

[0050]

[0051] wherein, (x a1 ,y a1 ) is the coordinate of one end point of line segment La, A = y b1 -y b2 , B = x b2 -x b1 , C = x b1 × y b2 -y b1 × x b2 , and (x b1 ,y b1 ) and (x b2 ,y b2 ) are the coordinates of the two end points of the other line segment Lb;

[0052] If the shortest distance between the two line segments La and Lb is less than or equal to the first distance threshold, it is considered that the line segments La and Lb are approximate line segments and can be merged;

[0053] 3.3 Approximate line segment merging

[0054] Let points a1 and a2 be the two endpoints of line segment La, and points b1 and b2 be the two endpoints of line segment Lb. Assume line segment La is the shorter of the two line segments, and line segment Lb is the longer of the two line segments. Calculate the angles between vectors b1a1 and b1b2, and between vectors b2a2 and b2b1. If both angles are acute, it means the two endpoints of the shorter line segment La can be projected onto the longer line segment Lb. Then, take the longer line segment Lb as the new line segment after merging, and delete the shorter line segment La, completing the merging of line segments La and Lb. If one of the angles is not acute... For acute angles, the Euclidean distance between any two endpoints of the two line segments La and Lb is calculated, including the Euclidean distances between a1 and b1, a1 and b2, a2 ​​and b1, and a2 and b2. If the shortest Euclidean distance is less than or equal to the second distance threshold (e.g., a value of 20), it indicates that the two line segments are close enough to meet the merging condition. In this case, the two endpoints corresponding to the maximum Euclidean distance are taken as the two endpoints of the new line segment, resulting in the merged new line segment. If the shortest Euclidean distance is greater than the second distance threshold, it indicates that the two line segments are far enough not to meet the merging condition, and therefore, they are not merged.

[0055] 3.4 Calculate the angle of the new line segment. If the angle difference between the new line segment and the long line segment before merging is less than or equal to the second angle deviation threshold, the deviation caused by the line segment merging is considered to be within a reasonable range, and the merged new line segment is retained. If the angle difference between the new line segment and the long line segment before merging is greater than the second angle deviation threshold (for example, a value of π / 360), the deviation caused by the line segment merging is considered to exceed a reasonable range, and the merged new line segment is not retained. Line segments La and Lb are reset, and the process returns to step 3.2 to continue traversing the remaining line segments.

[0056] Fig. 2 , 3 The images show the results of line segment extraction using the original CannyLines algorithm and the method of this invention, respectively. Compared with the original CannyLines algorithm, the method of this invention significantly reduces redundant and short line segments, lowers the complexity of feature matching, improves the accuracy of feature matching, and ensures the real-time performance of visual SLAM.

[0057] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A method for extracting straight line segments from visual SLAM images based on approximate line segment merging, characterized in that, The method includes the following steps: The first step is to use the CannyLines algorithm to perform edge detection on the image and obtain all pixel chains; The second step is to use the RANSAC algorithm to perform linear fitting on each pixel chain and extract line segments from the image. The third step is to identify approximate line segments among all line segments and then approximate them by merging them. 3.1 Calculate the angles of each line segment; 3.2 Calculate the angle difference between any two line segments. If the angle difference is less than or equal to the first angle deviation threshold, calculate the shortest distance between the two line segments. Calculate the distance from the two endpoints of one line segment to the other line segment. The smaller distance is the shortest distance between the two line segments. If the shortest distance is less than or equal to the first distance threshold, the two line segments are considered similar. Then, execute steps 3.3 and 3.

4. Otherwise, repeat this step to traverse the remaining line segments. 3.3 Two approximate line segments are denoted as La and Lb, with the two endpoints of line segment La being a1 and a2, and the two endpoints of line segment Lb being b1 and b2. Assume that line segment La is the shorter of the two line segments, and line segment Lb is the longer. Calculate the angles between vectors b1a1 and b1b2, and between vectors b2a2 and b2b1. If both angles are acute, it indicates that the two endpoints of the shorter line segment La can be projected onto the longer line segment Lb. In this case, the longer line segment Lb is used as the new line segment after merging, and the shorter line segment La is deleted, thus merging line segments La and Lb. If one of the angles is not acute, calculate the Euclidean distance between any two endpoints of the two line segments La and Lb. If the shortest Euclidean distance is less than or equal to the second distance threshold, the two endpoints corresponding to the maximum Euclidean distance are used as the two endpoints of the new line segment, resulting in the merged new line segment. If the shortest Euclidean distance is greater than the second distance threshold, no merging is performed. 3.4 Calculate the angle of the new line segment. If the angle difference between the new line segment and the long line segment before merging is less than or equal to the second angle deviation threshold, the merging deviation is considered to be within a reasonable range, and the merged new line segment is retained; otherwise, the merged new line segment is not retained, line segments La and Lb are reset and the process returns to step 3.2 to continue traversing the remaining line segments.

2. The method for extracting straight line segments from visual SLAM images based on approximate line segment merging according to claim 1, characterized in that, In the second step, for the pixel chain, calculate the Manhattan distance from all pixels to the image coordinate origin, and determine the distance threshold t for judging the interior points according to the following formula; In the formula, α is an adjustable coefficient, and M max M min These are the maximum and minimum values ​​of the Manhattan distance, respectively.

Images