Point-line feature slam method fusing depth information in low-texture scene

Abstract

The application discloses a point-line feature SLAM method for fusing depth information in a low-texture scene. Firstly, a color image is inputted, and point features and line features are extracted in the color image. A depth map of the same frame is inputted and aligned with the color image, the extracted line features are mapped to the depth map, and a pseudo-depth gradient of the line features is calculated. According to a quadtree algorithm, the pseudo-depth gradient and the length of the line features, the point features and the line features are screened and homogenized, and then feature matching is performed on the retained features. Then, a point EPNP algorithm is used and is extended to the line features, initial values of camera poses are calculated, an improved re-projection error model is constructed according to the pseudo-depth gradient and the length of the line features, and the camera poses are optimized. Then, local mapping and loop detection are performed, and the improved re-projection error model is also applied to local optimization and global optimization. The application fully utilizes information of the depth map and the color image, overcomes the problem of precision reduction or even system failure of a point-line feature-based visual SLAM in a low-texture scene, and improves system precision and real-time performance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of robot simultaneous localization and mapping, and specifically designs a point-line feature SLAM method fusing depth information in a low-texture scene. BACKGROUND

[0002] The continuous progress of automation, computer and artificial intelligence technology enables the rapid development of the robot industry, and gradually becomes a research hotspot in the field of artificial intelligence. Intelligent robots have been widely used in many fields at home and abroad, and are used to improve production efficiency or replace humans to complete tasks in mine environment detection, accident disaster and other harmful environments. For autonomous mobile robots, the prerequisite for completing tasks in various complex and changeable environments is to perceive the surrounding environment and locate itself through various technologies, and the emergence of SLAM technology solves this problem. Simultaneous Localization and Mapping (SLAM) refers to a subject carrying a specific sensor, which establishes an environment model and estimates its own motion in the process without prior information about the environment.

[0003] According to the sensor type, the SLAM technology can be divided into a camera-based visual SLAM system and a laser radar-based visual SLAM system. Laser SLAM technology started earlier, and the theoretical technology is relatively mature, and the established map has high precision and good robustness of pose estimation. However, due to the high cost of laser radar, people tend to use visual sensors with lower cost and easier installation, such as monocular cameras, binocular cameras, RGB-D cameras, etc.

[0004] Although visual SLAM can perceive rich environmental information, it still faces some challenges in scenes with a lack of texture features. In a low-texture environment, the lack of texture makes it difficult for feature point methods to capture feature points, making it impossible to estimate the pose, resulting in camera tracking failure. In addition, the effective information in a weak-texture scene is severely lacking compared to a texture-rich scene, but the noise sources and scale are consistent, resulting in a significantly reduced signal-to-noise ratio and severely affecting the accuracy of pose estimation. Since SLAM technology is mostly applied in artificial environments, these scenes usually have rich line features, so the solution to the failure of visual SLAM in low-texture scenes is usually to introduce line features. However, the introduction of line features will inevitably increase the complexity of the system, increase the time consumption of each module of the SLAM system, and when the system continues to run in a texture-rich environment, the real-time performance of the SLAM system is more difficult to guarantee, so it is particularly important to improve the real-time performance of the point-line feature-based visual SLAM system. In combination with depth information, the present application proposes a point-line feature SLAM method that fuses depth information in a low-texture scene, improves the real-time performance of the point-line feature-based visual SLAM system in an environment with a lack of texture, and at the same time improves the accuracy of the system. SUMMARY

[0005] In view of the problems of the prior art, the present application provides a point-line feature SLAM method that fuses depth information in a low-texture scene, solves the problem of poor real-time performance of the point-line feature-based visual SLAM system in a scene with a lack of texture, and at the same time improves the accuracy of the system.

[0006] To achieve the above-mentioned purpose, the technical scheme of the present application is:

[0007] A point-line feature SLAM method that fuses depth information in a low-texture scene, comprising the following steps:

[0008] S1, input a color image, and extract point features and line features from the color image;

[0009] S2, input a depth image, align the depth map of the same frame with the pixels of the color image, then map the line features extracted on the color image to the depth map in the form of line segments, and then calculate the gradient change on both sides of the line segment on the depth map, which is referred to as the pseudo-depth gradient of the line feature in the present application;

[0010] S3, if the point feature is higher than , the number of line features is higher than ( , The number of features is too large, and the system has too heavy a calculation burden, so that the point features and the line features are uniformly distributed and screened, so that the system has a proper number of features in a texture-lacking environment and a texture-rich environment; and the descriptors of the screened point features and the line features are calculated and matched;

[0011] S4, estimating a motion pose initial value from the matched point features and the line features, and then establishing a unified re-projection error model containing the point features and the line features to optimize the re-projection error model;

[0012] S5, judging whether the current frame is a key frame, if the current frame is a key frame, inserting the current frame into a local map and updating the local map; and then performing joint optimization, i.e. local optimization, on the current key frame and adjacent key frames according to the re-projection error model established in S4.

[0013] S6, performing loop detection on the key frames, and considering that a real loop occurs when both the point features and the line features detect a loop; and performing loop correction and global optimization when the loop occurs.

[0014] Further, the step S1 specifically comprises: firstly inputting a color image, extracting point features in the color image by using an ORB feature extraction algorithm; and extracting line features in the color image by using an LSD algorithm (the extracted line features contain information such as end points, midpoints and slopes of the line features).

[0015] Further, the step S2 specifically comprises:

[0016] S21, inputting a depth image, firstly aligning the depth image and the color image in the same frame, so that the depth value of a pixel point with a coordinate of in the color image is the depth value of a point with a coordinate of in the corresponding depth image;

[0017] S22, calculating the depth change on both sides of the line feature, i.e. the pseudo-depth gradient of the line feature, specifically comprising: denoting the line feature extracted in the color image, , after the line feature is mapped to the depth image, the coordinate values of the two end points of the line feature in the depth image are still , ; and the line segment in the depth image is expanded by a length along a direction perpendicular to , to form a large rectangular area, which takes the line segment as a central axis and is centered on the line segment The big rectangle is divided into two small rectangles with the same size; the sum of all pixel values in the two small rectangles is calculated respectively, then the two sums are subtracted and the absolute value is obtained, and the absolute value is divided by the number of pixels in the big rectangle to obtain the pseudo-depth gradient ; since the pixel value in the depth map represents the depth of the pixel at that position, when the pseudo-depth gradient is large, it can be considered that the depth at that position changes significantly, such line features contain texture information and depth change information of the environment, and can be considered as features observed from two dimensions, so such line features are more reliable.

[0018] Further, the step S3 is specifically:

[0019] S31, if the number of point features is greater than , then the feature points in the image are screened and uniformized using a quadtree algorithm;

[0020] S32, if the number of line features is greater than , then the line features are screened and uniformized, and the specific steps are: using the midpoint of the line feature to represent the position of the line feature, using a quadtree algorithm to realize uniform distribution of the line features, when some line features need to be removed in places where the line features are dense, the line features with larger pseudo-depth gradient are preferentially retained, and if the pseudo-depth gradient is the same, the line features with longer length are preferentially retained;

[0021] S33, calculating the descriptor after screening and uniformizing and performing feature matching: calculating the BRIEF descriptor of the point features and performing feature matching; using the LBD algorithm to calculate the descriptor of the line features and performing matching.

[0022] Further, the step S4 is specifically:

[0023] S41, calculating the initial value of the pose of the current frame: assuming that there are point features, line features; using the EPNP method to obtain linear equations for the point features: , wherein is a vector composed of the control points in the EPNP method, is a matrix of (every feature point corresponds to two constraint equations); the line feature is composed of two end points, and two linear equations same as the point features can be obtained; let be a residual vector, wherein is a matrix (including the point features and the two end points of the line features) of , and the control point coordinates can be calculated by minimizing the following objective function for iterative optimization: Then the pose of camera is solved according to the ICP algorithm, and finally the pose relationship matrix of the camera coordinate system relative to the world coordinate system is obtained and ;

[0024] S42, after calculating the initial value of the camera pose transformation, a new re-projection error model is constructed for optimization; the improved re-projection error model includes the re-projection error of point features and line features, and different weights are given to each line feature according to the length and pseudo-depth gradient of the line feature, wherein the line re-projection error is defined as the average value of the perpendicular distance from the two end points of the projected line segment to the line feature on the pixel plane.

[0025] Further, the step S42 is specifically:

[0026] S421, the re-projection error calculation method of point features: for a spatial point in the world coordinate system , its projection to the pixel plane is , wherein is a scale factor, is a camera intrinsic matrix, is the camera pose of the frame; and represents the spatial point in the observation value of the pixel plane, then the re-projection error of the point is: ;

[0027] S422, the re-projection error calculation method of line features: for a line segment in space , the coordinates of two end points are , ; represents the line segment in the observation value of the pixel plane (i.e. the line feature corresponding to the spatial line on the pixel plane), using the general method is expressed as , and the end points are , ; similar to step S421, the projection of , to the pixel plane is , , the distance from to the line segment is , wherein represents the coordinate value of , and the distance from to the line segment can be obtained in the same way; therefore, the re-projection error of the line feature is expressed as: , wherein , ;

[0028] S423、According to the pseudo-depth gradient and length of the line feature, the weight of each line feature is adjusted, and the application considers that the line feature with greater pseudo-depth gradient and longer length has higher reliability; the pseudo-depth gradients of all line features are arranged in order of size, and after excluding the largest 15% and the smallest 15%, the average value of the remaining pseudo-depth gradients is calculated ; the lengths of all line features are arranged in order of size, and after excluding the largest 15% and the smallest 15%, the average value of the remaining lengths is calculated ; the initial weight of each line feature is 1, and the adjustment is performed using and : for a certain line feature, the weight adjustment is , wherein represents the pseudo-depth gradient of the line feature, represents the length of the line feature; therefore, the final representation of the line feature re-projection error is: ; assuming that there are map points and spatial lines, the unified re-projection error model containing line features and point features is: .

[0029] Further, the step S5 is specifically:

[0030] S51, according to the quality of the current frame, the time elapsed since the last time a key frame was created, and the working state of the current system, it is determined whether the current frame is created as a key frame, if it is determined as a key frame, it is added to the local map, otherwise it returns to step S1;

[0031] S52, for the newly inserted key frame, the co-view and the generation tree are updated, and the current key frame is added to the map; the point features and line features observed less than 2 times in the continuous 3 key frames since the creation, and the point features and line features with a recall rate less than 0.25 are regarded as bad features, and are removed, wherein the recall rate is defined as: ; The current key frame is matched with the 10 co-view key frames with the highest co-view degree respectively, new map points and spatial lines are created, and are fused with the established map;

[0032] S53, after updating the map, the current key frame, the first-level co-view key frame of the current key frame, and the second-level co-view key frame of the current frame are jointly optimized, and outliers are removed in the optimization to maximize the optimization of the camera pose; the optimization object is the camera pose of the related key frame , and map points and spatial lines; the same re-projection error as step S423 is used in the optimization process .

[0033] Further, the specific condition for judging the key frame in step S51 is:

[0034] (1) the total sum of the point features and line features matched by the current frame cannot be less than 25;

[0035] (2) the distance to the last created key frame has exceeded 12 frames, and the current local mapping thread is in an idle state, then the key frame is judged;

[0036] (3) the distance to the last created key frame has exceeded 25 frames, then the key frame is judged;

[0037] (4) the coincidence degree of the features tracked by the current frame with the nearest key frame is lower than 90%, then the key frame is judged.

[0038] Further, the step S6 is specifically: based on the bag-of-words model, the point feature word vector and the line feature word vector of the current key frame are calculated respectively; the similarity between the word vector of the current key frame and the word vectors of other key frames is calculated to obtain the similarity between frames; when the loop is detected by both the point features and the line features, it is considered that the loop really occurs; a pose graph of all key frames is constructed, global pose graph optimization is performed to reduce the cumulative error, and the optimization process is the same as step S3, and the optimization object is the camera pose of all historical key frames 、 and map points and spatial lines.

[0039] Compared with the prior art, the present application has the following beneficial effects:

[0040] (1) the present application uses the pseudo-depth gradient of the line features calculated by the depth map to uniformly distribute and select the features according to the pseudo-depth gradient and length of the line features, and retains more reliable line features, thereby reducing the time consumption of the line feature matching and subsequent calculation related to the line features, and improving the real-time performance of the SLAM based on point and line features;

[0041] (2) the weight of each feature is calculated according to the length and pseudo-depth gradient of the line feature, and a unified re-projection error model containing point and line features is reconstructed according to the weight, thereby improving the accuracy of the SLAM based on point and line features.

[0042] In summary, the present application uses the RGB-D camera to fuse the depth information into the visual SLAM system based on point and line features, and improves the real-time performance and accuracy of the whole system through the selection and weight calculation of the features. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1A flow chart of the point-line feature SLAM method fusing depth information in a low-texture scene;

[0044] Figure 2 A schematic diagram of the pseudo-depth gradient calculation process of the line feature. DETAILED DESCRIPTION

[0045] The technical solutions of the present application will be further described in detail below with specific examples.

[0046] As shown in the drawings, Figure 1 the present application provides a point-line feature SLAM method fusing depth information in a low-texture scene, which comprises the following steps:

[0047] Step S1: using a depth camera to collect images, each frame of image contains a color (RGB) image and a depth image; inputting the color image, extracting point features in the color image by using an ORB feature extraction algorithm; extracting line features in the color image by using an LSD algorithm (the extracted line features contain their own endpoints, midpoints, slopes, etc.);

[0048] Step S2: inputting the depth image, first aligning the depth image and the color image of the same frame, so that the depth value of the pixel with coordinates in the color image is the depth value of the pixel with coordinates in the corresponding depth image; then mapping the line features extracted in the color image to the depth image in the form of line segments, and then calculating the depth changes on both sides of the line segment in the depth image as the pseudo-depth gradient of the line feature, the calculation steps being specifically as follows: denotes the line feature extracted in the color image, , are its two endpoints, after mapping the line feature to the depth image, a line segment is obtained (as shown in the drawings), Figure 2 the coordinate values of the two endpoints of the line segment in the depth image are still , ; expanding the line segment along the direction perpendicular to by a length to form a rectangular region , the rectangular region has the line segment as the central axis and is divided into two small rectangular regions of the same size by the line segment ; calculating the sum of the pixel values of all pixels in the two small rectangular regions and as , respectively, and the number of pixels in the rectangular region is The pseudo-depth gradient is calculated as: Since the pixel value in the depth map represents the depth of the pixel, when the pseudo-depth gradient is large, it can be considered that the depth changes significantly at this point. Such line features contain texture information and depth change information of the environment, and can be considered as features observed from two dimensions, so such line features are more reliable.

[0049] Step S3, if the number of point features is higher than , and the number of line features is higher than , the number of features is too large, and the system has too heavy a calculation burden, then the point features and line features are uniformly distributed and screened, so that the system has a suitable number of features in environments with lack of texture and rich texture; the descriptors of the point features and line features after calculation, screening and uniformization are calculated and matched;

[0050] Step S31, if the number of point features is higher than , then the feature points in the image are screened and uniformly distributed using the quadtree method;

[0051] Step S32, if the number of line features is higher than , then the line features are screened and uniformly distributed, and the specific steps are as follows: the midpoint of the line feature is used to represent the position of the line feature, the quadtree method is used to uniformly distribute the line features, and when some line features need to be removed in the dense line feature area, the line feature with a larger pseudo-depth gradient is preferentially retained, and if the pseudo-depth gradient is the same, the line feature with a longer length is preferentially retained;

[0052] Step S33, the BRIEF descriptor of the point feature retained after step S31 is calculated and the feature matching is performed; the descriptor of the line feature retained after step S32 is calculated using the LBD algorithm and the feature matching is performed;

[0053] Step S4, the motion pose initial value is estimated from the matched point features and line features, and then a unified reprojection error model containing point features and line features is established to optimize it; the improved reprojection error model includes the reprojection error of point features and line features, and each line feature is given different weights according to the length and pseudo-depth gradient of the line feature, wherein the line reprojection error is defined as the average value of the perpendicular distance from the two end points of the projected line segment to the line feature on the pixel plane;

[0054] Step S41, the pose initial value of the current frame is calculated: assuming that there are point features, line features; the EPNP method is used for point features to obtain a linear equation: wherein is a vector composed of control points in the EPNP method, is matrix (two constraint equations for each feature point); a line feature is composed of two endpoints, and two linear equations can be obtained which are the same as the point feature; let be the residual vector, where is the residual vector, where matrix (including point features and two endpoints of line features), the control point coordinates can be calculated by iterative optimization by minimizing the following objective function: Then, the pose relationship matrix of the camera coordinate system relative to the world coordinate system can be obtained by solving the pose according to the ICP algorithm and ;

[0055] Step S42, after the initial value of the camera pose transformation is calculated, an improved re-projection error model is constructed to optimize it.

[0056] Step S421, the re-projection error calculation method of a point feature: for a spatial point in the world coordinate system, its projection to the pixel plane is , where is a scale factor, is a camera intrinsic matrix, is the camera pose of the frame; the observation value of the spatial point in the pixel plane is represented by , and the re-projection error of the point is: ;

[0057] Step S422, the re-projection error calculation method of a line feature: for a line segment in space, the coordinates of its two endpoints are , ; represents the observation value of the line segment in the pixel plane (i.e., the line feature corresponding to the spatial line in the pixel plane), which is represented by , and the endpoints are , , ; similarly to step S421, the projection of , to the pixel plane is , , the distance from to the line segment is , where represents the coordinate value of , and the distance from to the line segment ; thus the reprojection error of line feature is represented as: , where , ;

[0058] Step S423, adjust the weight of each line feature according to its pseudo-depth gradient and length, the present application considers that the line feature with larger pseudo-depth gradient and longer length is more reliable; arrange the pseudo-depth gradients of all line features in order of size, after excluding the largest 15% and the smallest 15%, calculate the average value of the remaining pseudo-depth gradients ; arrange the lengths of all line features in order of size, after excluding the largest 15% and the smallest 15%, calculate the average value of the remaining lengths ; the initial weight of each line feature is 1, and the adjustment is made using and : for a certain line feature, its weight is adjusted to , where represents the pseudo-depth gradient of the line feature, represents the length of the line feature; thus the final representation of the reprojection error of line feature is: ; assuming that there are map points and spatial lines, the unified reprojection error model containing line features and point features is: ;

[0059] Step S5, judge whether the current frame is a key frame, if so, insert it into the local map and update the local map; then according to the reprojection error model established in S4, jointly optimize the current key frame and the adjacent key frame, i.e. local optimization;

[0060] S51, according to the quality of the current frame, the time elapsed since the last key frame was created, and the working state of the current system, judge whether to create the current frame as a key frame, if so, add it to the local map, otherwise return to step S1; the specific conditions for judging a key frame are: (1) the total number of point features and line features matched by the current frame cannot be less than 25; (2) more than 12 frames have elapsed since the last key frame was created, and the current local mapping thread is in an idle state, then the current frame is judged as a key frame; (3) more than 25 frames have elapsed since the last key frame was created, then the current frame is judged as a key frame; (4) the overlap between the features tracked by the current frame and the nearest key frame is less than 90%, then the current frame is judged as a key frame;

[0061] S52, for the newly inserted key frame, update the co-view and generate tree, add the current key frame to the map; the point features and line features observed less than 2 times in the 3 key frames created continuously and the point features and line features with a recall rate less than 0.25 are regarded as bad features, and are removed, wherein the recall rate is defined as: ; The current key frame is matched with the 10 co-view key frames with the highest co-view degree respectively, new map points and spatial lines are created, and are fused with the established map;

[0062] S53, after updating the map, the current key frame, the first-level co-view key frame of the current key frame and the second-level co-view key frame of the current frame are jointly optimized, and outliers are removed in the optimization to maximize the camera pose; the optimization object is the camera pose of the related key frame 、 and the map points and spatial lines; the same re-projection error as step S423 is used in the optimization process ;

[0063] Step S6, loop detection is performed on the key frame, and when the point features and line features both detect loop, it is considered that the loop really occurs; if the loop occurs, loop correction and global optimization are performed; specifically: based on the bag-of-words model, the point feature word vector and the line feature word vector of the current key frame are calculated respectively; the similarity between the word vectors of the current key frame and other key frames is calculated to obtain the similarity between frames; when the point features and line features both detect loop, it is considered that the loop really occurs; a pose graph of all key frames is constructed, global pose graph optimization is performed to reduce cumulative error, and the optimization process is the same as step S3, and the optimization object is the camera pose of all historical key frames 、 and the map points and spatial lines.

[0064] In summary: the application discloses a point and line feature SLAM method fusing depth information in a low-texture scene, and belongs to the technical field of robot simultaneous localization and map construction. Color images and depth images are obtained by using a depth camera, first, the ORB point feature extraction algorithm and the LSD line segment detection algorithm are used to extract point features and line features in the color images; then, the quadtree algorithm, the pseudo-depth gradient of the line feature and the length of the line feature are used to retain a suitable number of more reliable features, so that the subsequent calculation burden is reduced under the premise of ensuring accuracy; then, the pseudo-depth gradient and length information of the line feature are fused into the re-projection error model containing point and line features, and the improved model is applied to front-end pose optimization, local optimization and global optimization, and the system real-time performance is improved.

Claims

1. A point-line feature SLAM method that fuses depth information in low-texture scenes, characterized in that, Includes the following steps: S1. Input a color image and extract point and line features based on the color image; S2. Input a depth image, align the pixels of the depth map and color image of the same frame, map the line features extracted from the color image to the depth map in the form of line segments, calculate the depth change of the region surrounding the line feature on the depth map, and obtain the pseudo depth gradient of the line feature; the specific calculation step is: expand the line segments in the depth map to both sides by a preset length along a direction perpendicular to the line segment. A rectangular region is formed, which is divided into two smaller rectangular regions of the same size by a line segment as the central axis. The absolute value of the difference between the sum of all pixel values ​​in the two smaller rectangular regions is calculated, and the absolute value is divided by the number of pixels in the rectangular regions to obtain the pseudo depth gradient. S3, if the point feature is higher than The number of line features is higher than Then, the point features and line features are uniformly distributed and filtered, and the descriptors of the filtered point features and line features are calculated and matched. S4. Estimate the initial motion pose from the matched point features and line features, and then optimize it by establishing a unified reprojection error model that includes point features and line features. The reprojection error model integrates the reprojection errors of point features and line features, and assigns different weights to each line feature according to the length of the line feature and the pseudo depth gradient. S5. Determine whether the current frame is a keyframe. If it is a keyframe, insert it into the local map and update it. Use the reprojection error model to perform local optimization on the current keyframe and adjacent keyframes. S6. Perform loop closure detection on keyframes. When a loop closure is detected, perform loop closure correction and global optimization.

2. The point-line feature SLAM method for fusing depth information in low-texture scenes according to claim 1, characterized in that, The homogenization distribution and screening in step S3 are specifically as follows: If the number of point features is greater than The quadtree method is used to filter and homogenize point features in the image; If the number of linear features is greater than The line features are filtered and homogenized using the following strategy: the midpoint of the line feature represents its position, and a quadtree method is used to achieve a uniform distribution of the line features; when some line features need to be removed, line features with larger pseudo-depth gradients are retained first, and if the pseudo-depth gradients are the same, line features with longer lengths are retained first.

3. The point-line feature SLAM method for fusing depth information in low-texture scenes according to claim 1, characterized in that, The calculation descriptor in step S3 is specifically as follows: Calculate the BRIEF descriptor for point features; The LBD algorithm is used to compute descriptors for line features.

4. The point-line feature SLAM method for fusing depth information in low-texture scenes according to claim 1, characterized in that: The specific steps of step S4 are as follows: First, the EPNP method is applied to point features and line features to construct a unified residual vector, calculate the initial value of the camera pose transformation, and then construct an improved reprojection error model to optimize it. The improved reprojection error model includes reprojection errors for both point and line features, and assigns different weights to each line feature based on its length and pseudo-depth gradient. The line reprojection error is defined as the average of the vertical distances from the two endpoints of the projected line segment to the line feature on the pixel plane. The specific construction method is as follows: (1) Method for calculating the reprojection error of point features: For spatial points in the world coordinate system Projecting it onto the pixel plane is ,in As a scale factor, For the camera intrinsic parameter matrix, For the camera pose of this frame; use Representing a spatial point For the observations in the pixel plane, the reprojection error at that point is: ; (2) Calculation method for reprojection error of line features: For a line segment in space The coordinates of its two endpoints are , ; Represents line segment Observations on the pixel plane, i.e., spatial lines Corresponding to the line features on the pixel plane, Represented as The endpoints are respectively , ; Similar to step S421 , Projected onto the pixel plane , , to line segment The distance is ,in express The coordinates can be calculated similarly. to line segment distance Therefore, the reprojection error of the line feature is expressed as: ,in , ; (3) Adjust the weight of each line feature according to the pseudo-depth gradient and length of the line feature, and consider the line feature with larger pseudo-depth gradient and longer length to be more reliable; arrange the pseudo-depth gradients of all line features in order of magnitude, exclude the largest 15% and the smallest 15%, and calculate the average value of the remaining pseudo-depth gradients. Arrange all line features in ascending order of length, exclude the largest and smallest 15%, and calculate the average of the remaining lengths. The initial weight of each line feature is 1, using... and Adjust it: For a certain linear feature, its weight is adjusted to... ,in This represents the pseudo-depth gradient of the line feature. This represents the length of the line feature; therefore, the final expression for the line feature reprojection error is: ; Assuming there is already Map points and Given a spatial line, the unified reprojection error model that includes both line and point features is: .

Images