Binocular vision SLAM method based on improved point-line fusion and key frame selection

Abstract

The invention discloses a binocular vision SLAM (Simultaneous Localization and Mapping) method based on improved point-line fusion and key frame selection, and belongs to the technical field of vision SLAM positioning and mapping. According to the method, a line feature extraction thread is added on the basis of ORB-SLAM3, feature information in a weak texture environment is added, an error function model after point-line feature fusion is optimized, and the participation degree of line features in the adjustment process is balanced by using adaptive factors; in a tracking thread, a geometric mapping model is established, a self-adaptive dynamic threshold is set according to the variable quantity of point features in different frames, and key frames are screened. The pose estimation effect of the method is generally better than that of a visual positioning method in which only point features participate, stable tracking can still be carried out in the environment of insufficient point feature information and exposure scenes, and the method is suitable for weak texture scenes.

Description

Technical Field

[0001] The present invention relates to the field of visual SLAM positioning and mapping technology, and more specifically to a binocular visual SLAM method with improved point-line fusion and key frame selection, which is used for realizing autonomous positioning and navigation of a mobile robot. Background Art

[0002] Visual SLAM is a key technology for achieving high-precision positioning and navigation of moving objects in unknown scenes. This technology is widely used in unmanned driving, augmented reality / virtual reality, automatic guided vehicles, mobile robots and other fields. Visual SLAM can usually be divided into two categories: direct method and feature point method. Compared with the direct method, the feature point method can adapt to the lighting changes of the scene to the greatest extent, is insensitive to the movement of the visual camera, and has high robustness. However, when the texture information in the scene is insufficient, the number of feature points will decrease sharply, which will lead to a decrease in the accuracy of the algorithm, which brings great obstacles to the practical application of SLAM technology. In the process of estimating the camera pose, in order to avoid the cumulative error of pose estimation, the map points and poses of all frames need to be uniformly optimized nonlinearly to improve the positioning accuracy of the SLAM system, but the calculation amount of the system also increases accordingly. In order to achieve the computer's limited data processing capabilities, it can also effectively improve the system positioning accuracy and reduce data redundancy. Optimizing the selection of key frames is one of the effective solutions to this problem.

[0003] At present, there are visual odometry based on semi-direct method using point and line features, and then the binocular point and line feature SLAM method was open sourced, which fully considers and weighs the point and line features in the loop detection module. However, the real-time performance, trajectory accuracy and robustness of line feature extraction of these methods are still poor. In terms of key frame selection, relevant researchers proposed a heuristic threshold key frame screening method, which has higher robustness. Although this method shows high robustness, it is still not enough to accurately select key frames in special environments such as weak texture features, uneven lighting intensity or narrow space, thus affecting the positioning and map construction performance of the entire system. Summary of the invention

[0004] 1. Technical problem to be solved by the invention

[0005] The point-line feature fusion method used in the present invention can make up for the defect that a SLAM system with only point features or only line features cannot obtain sufficient feature information in a complex environment; the error function after improving the point-line fusion can effectively eliminate abnormal values ​​in the system and improve the accuracy of system tracking and matching; the participation degree of line features in the beam adjustment process is balanced by introducing an adaptive factor, the calculation amount of the system is reduced, and the real-time performance of the system is improved; the geometric mapping model designed in the present invention can effectively reduce the generation of redundant key frames and enhance the adaptability of the system to complex environments.

[0006] 2. Technical solution

[0007] To achieve the above-mentioned purpose, the present invention provides a binocular vision SLAM method based on improved point-line fusion and keyframe selection, a line feature extraction module based on ORB-SLAM3, improves the feature richness of the system in complex environments, and utilizes the fused line feature distance error and angle measurement error to effectively remove outliers and improve the accuracy of the system; in the process of beam adjustment, an adaptive factor is set to balance the participation of line features, and the effects of point and line features are dynamically adjusted to improve the flexibility of the system; on the constraint condition of whether the tracking thread decides to insert a keyframe, the best keyframe is selected by setting a dynamic threshold, thereby improving the accuracy of the system and realizing the precise positioning of the SLAM system. The present invention discloses a binocular vision SLAM method based on improved point-line fusion and keyframe selection, comprising the following steps:

[0008] Step 1: Select ORB-SALM3 as the main framework, add line feature extraction threads to make up for the lack of feature information in complex scenes, and use the distance error and angle measurement error of the fused weighted line features to eliminate outliers;

[0009] Step 2: Introduce adaptive factors to balance the participation degree of line features during bundle adjustment;

[0010] Step 3: Improve the key frame selection strategy;

[0011] Step 4: Estimate the pose error.

[0012] Furthermore, the detection and description of the line features described in step 1 use LSD detector and LBD descriptor (linebanddiscriptor). In the bundle adjustment (BA) optimization process, due to feature extraction, matching or illumination changes of the environment scene, the SLAM system has a small amount of outliers. This paper uses two error functions, one is the traditional minimization of the distance from the reprojected endpoint to the observation line segment. On this basis, the angle measurement error function is added to further constrain the SLAM system to eliminate outliers.

[0013] Furthermore, the distance error function described in step 1 uses the Euclidean distance between the endpoints of the reprojected three-dimensional line segment and its corresponding straight line endpoint on the image plane. The distance error function is expressed as:

[0014]

[0015] Furthermore, the angular distance error function described in step 1 uses the cosine value of the angle between the reprojected line segment and the observed line segment, and the angle measurement error function is expressed as:

[0016]

[0017] Furthermore, the bundle adjustment described in step 2 is a motion-only bundle adjustment in the tracking thread, which only optimizes the camera parameters (motion parameters) and keeps the 3D structure (landmarks) unchanged;

[0018] Furthermore, the adaptive factor introduced in step 2 is dynamically adjusted according to the sum of the number of clusters of feature points. In practical applications, by observing the number of associated features involved in matching in the initial observation frame, according to the general performance of feature extraction in the SLAM system, the adaptive factor is expressed as:

[0019]

[0020] Furthermore, the improved keyframe selection strategy described in step 3 can also reduce redundant computations by selecting keyframes that maximize environmental understanding through geometric and temporal constraints.

[0021] Furthermore, the geometric constraints in step 3 establish a geometric mapping model based on the change in the number of feature points between different frames. When the frame moves from the previous key frame to the current frame, the number of feature points corresponding to the vector in each region also changes. Based on the number of these change points, a dynamic threshold is determined to decide whether to insert a new key frame. The dynamic threshold is expressed as:

[0022] T=(1+εθ+α-β)·T initial

[0023] Among them, T initial Represents the initial threshold in the keyframe selection strategy, defined as:

[0024]

[0025] N c Indicates the number of feature points corresponding to the current frame and the previous key frame, N r Indicates the number of feature points corresponding to the reference frame and the previous key frame, B c Indicates the number of all feature points in the current frame, B rrepresents the number of all feature points in the reference frame, C c Indicates the number of feature point changes between the current frame and the previous key frame, C r Indicates the number of feature point changes in the geometric mapping plane between the reference frame and the previous key frame.

[0026] The theta coefficient is defined as:

[0027]

[0028] The ε coefficient is:

[0029]

[0030] Coefficient α:

[0031]

[0032] β is defined as:

[0033]

[0034] Furthermore, in step 4, the present invention uses the Euroc dataset to improve the pose error estimation after point-line fusion and key frame selection. In the test, control experiments are carried out on ORBS-SLAM3 and PL-SLAM respectively, and the superiority of the algorithm is evaluated by measuring the fit between the trajectory and the true value and the ATE value.

[0035] Furthermore, in order to comprehensively evaluate the performance of the present invention in step 4, a representative weak-texture conference room scene is selected as a real test environment, and the distance back to the starting point is used as the error judgment experimental result to test the anti-interference ability of the system.

[0036] 3. Beneficial effects

[0037] Compared with the existing known technologies, the technical solution provided by the present invention has the following significant effects:

[0038] (1) A binocular vision SLAM method based on improved point-line fusion and key frame selection of the present invention, a feature line extraction module is added to the front-end feature extraction module, which effectively makes up for the defect of insufficient feature information extraction in a weak texture environment, and at the same time, according to the motion estimation method, the matching of line features is constrained, and qualified line segments are screened out; in the line segment reprojection error part, the error function of the two methods of Euclidean distance and angular distance is fused, and weighted processing is performed to achieve the elimination of outliers; on this basis, for the point-line feature selection problem in the bundle adjustment method, an adaptive factor is introduced to measure the participation degree of the line segment in the adjustment process. In terms of key frame selection, the present invention designs a key frame selection method based on geometry and photogrammetry principles, and sets a dynamic threshold according to the frame rate of the camera to avoid inserting redundant key frames, thereby reducing the impact on the robustness and real-time performance of the SLAM system.

[0039] (2) Visual SLAM relies on features extracted from images to estimate the position and attitude of the camera. Outliers, such as incorrect feature matching or errors caused by dynamic objects, can lead to misleading pose estimates. Eliminating these outliers helps provide more accurate position data, ensuring that the SLAM system correctly understands and reflects its environment. For line features, the traditional method of determining the distance error of their endpoints and eliminating outliers has limitations. The present invention can solve the problem of instability of line segment endpoints caused by displacement caused by camera motion and occlusion between line segments by adding angle error constraints.

[0040] (3) In the SLAM system, point features have simpler geometric structures, fewer occlusion problems, and faster matching speeds than line features. The present invention sets an adaptive factor to dynamically balance the contributions of point features and line features according to the feature distribution of the current scene (such as the richness of points), thereby minimizing the participation of line features and improving the accuracy of the entire SLAM system.

[0041] (4) The SLAM system needs to run in real time, and the creation and updating of maps usually involve a lot of calculations. The present invention reduces the amount of data that needs to be processed and reduces the computational burden by reasonably selecting key frames. In complex environments, the method of selecting key frames based on geometric constraints can ensure that the selected frames have sufficient changes in perspective and position. Selecting frames with obvious features and stable geometric structures can increase the system's robustness in processing perspective changes and frames with occlusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a process framework diagram of the present invention;

[0043] Figure 2 is the line characteristic function error model;

[0044] Figure 3 It is a trend chart of the number of features involved in tracking and matching when no adaptive factor is added;

[0045] Figure 4 It is a scatter plot of the number of features involved in tracking and matching when no adaptive factor is added;

[0046] Figure 5 It is a trend chart of the number of features involved in tracking and matching when the adaptive factor is added;

[0047] Figure 6 It is a scatter plot of the number of features involved in tracking and matching when the adaptive factor is added;

[0048] Figure 7 APE result diagram of V1_02_medium sequence without adding adaptive factor;

[0049] Figure 8 This is the APE result diagram of the V1_02_medium sequence with the adaptive factor added;

[0050] Fig. 9 It is a geometric mapping model;

[0051] Fig.10 It is a real test scene diagram;

[0052] Fig.11 This is the effect diagram of feature extraction in real test scenarios;

[0053] Fig.12 It is a four-wheel differential car motion model;

[0054] Fig.13 It is the real track of the four-wheel differential car;

[0055] Fig.14 This is a diagram of the positioning trajectory of different algorithms in real test scenarios;

[0056] Fig.15 This is a diagram showing the local positioning effects of different algorithms in real test scenarios; DETAILED DESCRIPTION

[0057] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments.

[0058] Feature point extraction and matching:

[0059] Point feature selection is the same as the ORB-SLAM3 algorithm, based on the FAST key point detector and BRIEF descriptor, using P = (X, Y, Z) to represent the coordinates of a 3D point, and p = (x, y) to represent the coordinates of a 2D point. In order to make the feature descriptor invariant to rotation, the key point direction is assigned to ensure that the same feature point can be matched after the image is rotated. The matching degree is determined by projecting the 3D point onto the image plane and using the Hamming distance between the corresponding descriptors to find the closest observation value.

[0060] Feature line extraction and matching:

[0061] The present invention uses LSD detector and LBD descriptor to detect and describe line features. The endpoints of the three-dimensional map line segment and the two-dimensional observation line segment are respectively P(X P ,Y P ,Z P )、Q(X Q ,Y Q ,Z Q )、p(x p ,y p ) and q(x q ,y q ) indicates that the function expression of the two-dimensional observation line segment is defined as:

[0062]

[0063] In order to determine the match between the 3D map segment and the 2D observation segment, the two endpoints of the 3D map segment are projected separately and matched using the LBD descriptor.

[0064] Establish the line segment error function model:

[0065] In the process of bundle adjustment (BA), there are a small number of outliers due to feature extraction, matching or changes in illumination of the environment scene. If these outliers are not handled, they will have a serious negative impact on the optimization process and reduce the accuracy of positioning. Since the line segment may be displaced from the previous frame to the current frame due to camera movement, occlusion between line segments may occur, resulting in instability of the line segment endpoints. Therefore, for the error model of line features, the present invention uses two error functions to remove outliers. First, the distance from the reprojected endpoint to the observed line segment is constrained by traditional minimization. On this basis, the angle measurement error function is added for further constraints.

[0066] Each line feature has two distance errors and two angle observation errors. The process of establishing the line feature error model is as follows: Figure 2 Assume that the jth line segment in the i-th frame is processed, and the distance error function is:

[0067]

[0068] Among them, the values ​​of the endpoints of the reprojected line segment are and a, b, c are the coefficients of the standard equation of the line segment.

[0069] The angle error function is:

[0070]

[0071] where e 3 i,j (ξ) is the vector V qp' and V qp The cosine value, e 4 i,j (ξ) is the vector V pq' and V qp The cosine value of .

[0072] Therefore, the error function of the j-th line feature of the i-th frame is a 4×1 error vector, which is finally expressed as e i,j (ξ):

[0073]

[0074] H is the weight matrix of the jth observed line segment feature in the i-th frame. Since the dimensions of the two distance error functions and the two angle error functions are different, a weighted method is used for fusion.

[0075]

[0076] According to the difference between the predicted value described by the error function and the actual observed value, if the norm of the error exceeds the chi-square distribution threshold of 5.991 given a degree of freedom of 2 and a confidence level of 95%, they are considered to be outliers caused by mismeasurement, noise or other non-ideal factors and should be eliminated from the system.

[0077] Motion-only bundle adjustment optimizes only the camera parameters (motion parameters) and keeps the 3D structure (landmarks) unchanged. The camera pose can be expressed as:

[0078]

[0079] e i,k (ξ) represents the reprojection error of the kth feature point in the i-th frame; e i,j(ξ) represents the reprojection error of the jth feature line segment in the i-th frame; ω is the pose of the i-th frame (the variable to be optimized); P and I are clusters of point and line matching sets, respectively; ρ(·) represents the robust Huber cost function; α(P) is the sum of the number of elements in the feature point cluster P; F(α(P)) is the adaptive factor, which is defined in general form as:

[0080] F(α(P))=W -α(Ρ)divT

[0081] In practical applications, by observing the number of matching-related features in the initial observation frame and testing the feature extraction of the improved SLAM system, the adaptive factor base is set to 2 and the weight is set to 80. That is:

[0082] F(α(P))=2 -α(Ρ)div80

[0083] F(α(Ρ)) can be dynamically adjusted according to the α(Ρ) value, that is, the system can dynamically balance the contribution of point features and line features according to the feature distribution of the current scene. This method is particularly suitable for scenes with poor texture and insufficient point features, and the lack of point features can be compensated by increasing the participation of line features. On the contrary, in scenes with rich point features, given that point features have simpler geometric structures, fewer occlusion problems, and faster matching speeds than line features, the system will reduce the participation of line features through adaptive factors, thereby improving the robustness and accuracy of the entire SLAM system.

[0084] Figure 3 It shows that without adding adaptive factors, the number of point features and line features changes irregularly, and the number of line features is too large, which increases the computational burden of the SLAM system and affects the robustness of the system. Figure 4 By analyzing the data in, it is found that the correlation coefficient is 0.04. As the number of point features increases, the number of line features also increases accordingly.

[0085] In contrast, by Figure 5 It can be seen that the introduction of the adaptive factor of point and line feature selection in the bundle adjustment process can effectively balance the participation of point features and line features. When the number of point features increases, the system will limit the number of line features and use point features for pose estimation as much as possible. On the contrary, when the number of point features decreases, the system will increase the number of line features extracted and matched to make up for the lack of feature information in the scene. Figure 6 The distribution scatter plot of point features and line features involved in tracking and matching in the scene is displayed. Data analysis shows that the distribution of point features and line features is negatively correlated, and line features decrease as the number of point features increases, with a correlation coefficient of -0.31.

[0086] The system adjusts the application of point and line features through adaptive factors when processing complex scenes. In an environment with rich point features, the system will reduce its reliance on line features accordingly, thereby reducing the amount of calculation and effectively enhancing real-time performance. This adaptive adjustment strategy optimizes resource allocation and ensures efficient and accurate processing speed in complex and changing visual environments. Table 1 shows the tracking and matching speed of each frame of the algorithm of the present invention in the Euroc dataset when the adaptive factor is not added and when the adaptive factor is added.

[0087] Table 1 Real-time analysis

[0088] sequence No adaptive factor added (seconds / frame) Add adaptive factor (seconds / frame) MH_01_easy 0.0915 0.0896 MH_02_easy 0.0859 0.0845 MH_03_medium 0.0823 0.0804 MH_04_difficult 0.0711 0.0698 MH_05_difficult 0.0719 0.0714 V1_01_easy 0.0671 0.0667 V1_02_medium 0.0618 0.0608 V1_03_difficult 0.0558 0.0543 V2_01_easy 0.0666 0.0662 V2_02_medium 0.0690 0.0675 V2_03_difficult 0.0584 0.0563 Average 0.0710 0.0698

[0089] As can be seen from Table 1, the average tracking time of the algorithm without the adaptive factor on the 11 sequences is 0.0710 seconds / frame, and the average tracking time with the adaptive factor is 0.0698 seconds / frame. By introducing the adaptive factor, the average tracking time of the algorithm of the present invention on the 11 sequences of the Euroc dataset is significantly reduced. Figure 5 The V1_02_medium sequence was selected for testing, and the positioning accuracy of the algorithm without and with the adaptive factor was compared.

[0090] like Figure 7 , 8 As shown, rmse means root mean square error, median means median error, mean means mean error, and std means standard deviation. Figure 7 This is the pose estimation error result diagram of the V1_02_medium sequence without adding the adaptive factor. Compared with the true trajectory, the RMSE error is 0.0183m; Figure 8 This is the pose estimation error result diagram with the addition of the adaptive factor, and the rmse error is 0.0161m. Compared with the traditional motion-only bundle adjustment method, the invention improves the accuracy of the system on the V1_02_medium sequence by 12.02% by adding the adaptive factor. This result proves the effectiveness of the adaptive factor in improving the accuracy of the visual SLAM system.

[0091] SLAM systems need to run in real time, and the creation and updating of maps usually involves a lot of calculations. By reasonably selecting key frames, the amount of data that needs to be processed can be reduced, thereby reducing the computational burden. In dynamic or complex environments, the use of geometric constraint methods can ensure that the selected frames have sufficient changes in perspective and position, and have obvious features and stable geometric structures, which can increase the system's robustness in processing perspective changes and frames with occlusion.

[0092] like Fig. 9As shown, the geometric mapping model established by the present invention mainly relies on understanding and calculating the three-dimensional structure of the scene to provide richer information for pose estimation. In the figure, the pentagon represents the landmark point, the square represents the detected feature point, and the circle represents the tracked feature point. The dynamic threshold is determined by counting the total number of feature points in each frame image, the number of tracking points, and the number of points that have changed in the geometric mapping area. First, some feature points in the current frame are matched with the feature points in the previous key frame to calculate the corresponding three-dimensional landmark points. Then, a geometric mapping model is constructed for each landmark point, connecting the landmark point with the plane of the geometric mapping model, whose main axis is perpendicular to the center of the circular geometric mapping model, and the three areas are divided at an angle of 15 degrees with the main axis. A vector is formed by connecting the position of each tracking point in each frame with its corresponding position of the landmark point in the map, and the vector is divided into three predetermined areas according to the angle between the vector and the main axis. When the frame moves from the previous key frame to the current frame, the number of feature points corresponding to the vector in each area also changes. Based on the number of these change points, the dynamic threshold is determined to decide whether to insert a new key frame.

[0093] Under the assumption that the current frame state is similar to the reference frame after system initialization, the initial threshold is set according to the number of corresponding points and the number of points where the line of sight changes, which is used to select the initial key frame. The initial threshold is set as follows:

[0094]

[0095] Where T initial is the initial threshold, N c Indicates the number of feature points corresponding to the current frame and the previous key frame, N r Indicates the number of feature points corresponding to the reference frame and the previous key frame, B c Indicates the number of all feature points in the current frame, B r represents the number of all feature points in the reference frame, C r Indicates the number of feature point changes in the geometric mapping plane between the reference frame and the previous key frame.

[0096] The initial threshold is set based on the assumption of similarity between the current frame and the reference frame. As the camera moves, the distance between the current frame and the previous keyframe gradually increases, resulting in a decrease in the similarity between the state of the current frame and the reference frame. In order to ensure the reliability of keyframe selection, new parameters must be introduced to make the dynamic threshold closer to the actual situation. The dynamic threshold is defined as:

[0097] T=(1+εθ+α-β)·T initial

[0098] Among them, ε is the interval control coefficient, θ is the matching feature quantity adjustment coefficient, α is the reference frame feature change adjustment coefficient, and β is the current frame feature change adjustment coefficient.

[0099] In order to avoid the probability of selecting a key frame in the first two frames after the previous key frame, the present invention stipulates that the next key frame will not be selected until at least K / 3 frames have passed, in view of the limited running speed of the indoor environment robot. This method effectively avoids the selection of redundant key frames. The introduced ε coefficient is:

[0100]

[0101] Among them, K is the camera shooting frame rate, ΔID represents the distance between the current frame and the previous key frame. This coefficient is used to reduce redundant key frames and enhance the stability and accuracy of the system in complex environments.

[0102] In the transition from the previous keyframe to the current frame, the threshold for keyframe selection needs to be highly adaptable to ensure a reasonable distribution of keyframes. By reducing the number of feature points in the current frame and the reference frame, the adjustment of the θ coefficient is proportional to the reduction of matching feature points. The θ coefficient is defined as:

[0103]

[0104] The present invention optimizes the dynamic threshold value by multiplying the coefficient θ by the coefficient ε.

[0105] The ε coefficient sets ideal conditions for selecting keyframes. However, when the robot moves slowly, the feature information between consecutive frames captured by the camera is highly similar, resulting in the number of feature points in the changing area of ​​the geometric mapping model not reaching the preset threshold, thus failing to meet the requirements for selecting keyframes. In this case, the system cannot add new keyframes for a long time, which leads to camera tracking failure, seriously affecting the accuracy of the positioning system. To solve this problem, the β coefficient is introduced to adjust the threshold value of keyframe selection, thereby lowering its standard. The coefficient β needs to be subtracted from the previous coefficient to simplify the initial threshold. β is defined as:

[0106]

[0107] C c Indicates the number of feature point changes in the mapping area between the current frame and the previous key frame.

[0108] To further ensure the reliability of the threshold, the ratio of matching points whose mapping area of ​​the reference frame changes relative to the previous key frame is increased to adjust the threshold, and the coefficient α is introduced:

[0109]

[0110] The algorithm of the present invention proposes a point-line feature fusion scheme based on ORB-SLAM3 and adds a keyframe selection strategy. In order to prove the improvement effect of the two improved methods on the SLAM system, an ablation experiment is designed, and the MH_04_difficult sequence and the V1_03_difficult sequence are used for testing. The scene in the MH_04_difficult sequence has the characteristics of dark sight and poor texture environment; the V1_03_difficult sequence has the characteristics of large white walls, large light exposure, and insufficient point feature information. Using these two sequences for control experiments can fully verify the adaptability of the algorithm of the present invention to weak texture environments. Table 2 is the absolute trajectory error (ATE) between the trajectories of the MH_04_difficult and V1_03_difficult sequences, and the root mean square error (RMSE) is used to measure the deviation between the pose estimate and the true value.

[0111] Table 2 ATE result comparison (unit: m)

[0112]

[0113] It can be clearly seen from Table 2 that the accuracy of the system can be effectively improved by fusing point and line features and adding key frame selection strategies. Especially for the V103 sequence, it can be calculated that the positioning accuracy of the improved algorithm can be improved by 63.00%. The ablation experiment proves the feasibility of the algorithm of the present invention. The following experiment will be conducted on the full sequence of the Euroc dataset to analyze the stability and reliability of the improved system in different difficulty scenarios.

[0114] In order to test the performance of the proposed algorithm and the existing algorithms on public datasets, the absolute trajectory errors (ATE) between the estimated trajectories and the true trajectories of 11 sequences on the Euroc dataset are compared between different methods.

[0115] Table 3 Comparison of absolute trajectory error (ATE) results (unit: m)

[0116] ORB-SLAM3 PL-SLAM PLKF-SLAM MH_01_easy 0.0495 0.0680 0.0283 MH_02_easy 0.0241 0.0325 0.0163 MH_03_median 0.0327 0.0518 0.0280 MH_04_difficult 0.0637 0.0914 0.0612 MH_05_difficult 0.0615 0.0869 0.0489 V1_01_easy 0.0354 0.0305 0.0299 V1_02_median 0.0233 0.0603 0.0186 V1_03_difficult 0.0738 0.0467 0.0273 V2_01_easy 0.0564 0.0542 0.0211 V2_02_median 0.0408 0.0800 0.0197 V2_03_difficult 0.4958 0.1698 0.1539 Average 0.0870 0.0702 0.0412

[0117] As can be seen from Table 3, PLKF-SLAM performs well on sequences of various difficulty levels (simple "easy", medium "median", difficult "difficult"), and the absolute trajectory error values ​​of the 11 sequences are better than ORB-SLAM3 and PL-SLAM, and the average positioning accuracy improvement rate on the 11 sequences reaches 36.00% and 41.31%. Compared with ORB-SLAM3, the PLKF-SLAM algorithm proposed in the present invention performs particularly well in processing V1_03_difficult and V2_03_difficult sequences. There are large areas of white walls in the environment of these two sequences, and conventional point feature extraction methods do not work well in such scenarios. Therefore, by increasing the use of line features, PLKF-SLAM effectively optimizes pose estimation and significantly improves the overall accuracy of the system. In addition, compared with PL-SLAM, PLKF-SLAM also shows significant performance improvements on simple sequences such as MH_01_easy and V1_02_medium. This improvement is attributed to the optimization of the keyframe selection strategy, which reduces the involvement of redundant keyframes and effectively reduces the probability of mismatching in matching tracking, thereby improving the positioning accuracy of the system.

[0118] In order to comprehensively evaluate the performance of the system, a representative weak-texture conference room scene is selected as a real test environment to test the algorithm of the present invention.

[0119] like Fig.10 As shown in the figure, the characteristics of this scene include severe exposure changes and difficulty in feature extraction. The uneven light distribution and lack of obvious texture inside the conference room increase the difficulty of visual navigation, which poses a great challenge to the robustness of the SLAM system. Fig.11 The actual scene extraction effect after adding the line feature extractor to the binocular vision SLAM system of the present invention is demonstrated. By integrating the line feature extractor at the front end, the system can obtain a sufficient number of line features, effectively making up for the problem of insufficient information that the system may encounter in scenes where features are missing or not obvious, thereby enhancing the system's robustness and environmental adaptability.

[0120] Fig.12 In: n1, n2, n3 and n4 are the rotation speeds of the four wheels; v c is the linear velocity of the chassis center; ω c is the angular velocity of the center of the chassis; C is the rotation center of the chassis; l is the wheelbase of the left and right wheels; h is the wheelbase of the front and rear wheels. The speed of the wheels on the same side of the four-wheel differential chassis is equal, so n is used. L Indicates the speed of the left wheel, expressed as n R Represents the speed of the right wheel, and the motion expression of the four-wheel differential chassis is:

[0121]

[0122] The algorithm of the present invention is compared with ORB-SLAM3 and PL-SLAM systems. Fig.13 This is the actual trajectory diagram of the four-wheel differential car. Fig.14 Estimated trajectory diagrams for PLKF-SLAM, ORB-SLAM3, and PL-SLAM.

[0123] from Fig.15 It can be seen that the ORB SLAM3 algorithm has serious trajectory drift due to feature loss when turning. At the same time, both the PL-SLAM algorithm and ORB-SLAM3 did not trigger loop detection at the end point, resulting in positioning failure. In contrast, the improved point-line fusion PLKF-SLAM algorithm of the present invention has a smoother trajectory in the same scenario, with only slight trajectory drift, and successfully returns to the starting point, demonstrating stronger anti-interference ability. In terms of positioning error, taking the distance back to the starting point as the error judgment result, the errors of PLKF-SLAM, ORB-SLAM3 and PL-SLAM are 0.05 meters, 0.39 meters and 0.30 meters respectively. Obviously, the algorithm proposed in the present invention has better positioning effect. The average running frame rate of the algorithm in this paper on a computer host with a CPU of i5-12400F and a system of Ubuntu20.04 reaches 32.06 frames per second, which meets the real-time requirements and has stronger practical performance.

[0124] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

[0125] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A binocular vision SLAM method based on improved point-line fusion and keyframe selection, characterized in that: Based on ORB-SLAM3, the feature fusion and key frame selection strategies are improved; the error function model of line features is optimized, the angle error function is introduced, and the outliers of feature matching are eliminated; In the bundle adjustment part of the tracking thread, an adaptive factor is introduced to dynamically adjust the participation degree of line features according to the richness of scene feature information; In the tracking thread, an improved key frame selection method based on feature points is adopted, and a dynamic threshold is set to reduce the generation of redundant key frames.

2. A binocular vision SLAM method based on improved point-line fusion and key frame selection according to claim 1, characterized in that: The steps include: Step 1: Select ORB-SALM3 as the main framework, add line feature extraction threads to make up for the lack of feature information in complex scenes, and use the distance error and angle measurement error of the fused weighted line features to eliminate outliers; Step 2: Introduce adaptive factors to balance the participation degree of line features during bundle adjustment; Step 3: Establish a geometric mapping model based on feature points and improve the key frame selection strategy; Step 4: Estimate the pose error.

3. A binocular vision SLAM method based on improved point-line fusion and key frame selection according to claim 2, characterized in that: The line features described in step 1 are extracted using an LSD (Line Segment Detector) extractor, and a 256-bit LBD (Line Band Descriptor) descriptor is selected to match line segments between different frames.

4. A binocular vision SLAM method based on improved point-line fusion and key frame selection according to claim 3, characterized in that: The distance error described in step 1 uses the Euclidean distance between the endpoints of the reprojected line segment and its corresponding straight line endpoint on the image plane, and the angle measurement error is represented by the cosine value of the angle error between the observed line segment and the reprojected line segment.

5. A binocular vision SLAM method based on improved point-line fusion and key frame selection according to claim 4, characterized in that: The weighted error function in step 1 is expressed as: Among them, e 1 i,j (ξ), e 2 i,j (ξ) is the distance error, e 3 i,j (ξ) and e 4 i,j (ξ) is the angle measurement error.

6. A binocular vision SLAM method based on improved point-line fusion and key frame selection according to claim 2, characterized in that: The adaptive factor added to the balance line feature during the bundle adjustment process in step 2 is: F(α(P))=2 -α(Ρ)div80 Among them, α(P) is the sum of the number of elements in the feature point cluster P.

7. A binocular vision SLAM method based on improved point-line fusion and key frame selection according to claim 2, characterized in that: In step 3, a geometric mapping model based on the change in the number of feature points is established according to geometric constraints. The dynamic threshold is determined by counting the total number of feature points in each frame image, the number of tracking points, and the number of points that have changed in the geometric mapping area to constrain the selection of key frames.

8. A binocular vision SLAM method based on improved point-line fusion and key frame selection according to claim 2, characterized in that: The pose error estimation experiment in step 4 uses the open source dataset Euroc to test the effectiveness of the method of the present invention. The test includes trajectory positioning effect and pose drift results. The superiority of the algorithm is evaluated by evaluating the generated ATE value.

9. A binocular vision SLAM method based on improved point-line fusion and key frame selection according to claim 2, characterized in that: In step 4, a four-wheel differential car is used to make a conference room weak texture data set, and the distance back to the starting point is used as the error judgment result to evaluate the reliability of the algorithm proposed in the present invention.

Images