Edge-line feature joint visual odometry method based on edge probability confidence propagation of pedger++

Abstract

The application discloses an edge-line feature joint visual odometer method based on PEdger++ edge probability confidence propagation, and comprises the following steps: collecting video images and obtaining a current image frame, performing edge feature extraction on the current image frame by using a PEdger++ algorithm to obtain an edge probability image; obtaining the probability confidence of edge feature points in the edge probability image, and obtaining sub-pixel geometric coordinates greater than a preset precision threshold; performing distance transformation based on the sub-pixel geometric coordinates and the probability confidence of the edge feature points to construct an error function; screening local extreme points greater than a preset probability confidence threshold in the edge probability image, and taking the local extreme points as seed points; extracting two-dimensional line features by using the seed points and constructing a projection error function of a three-dimensional straight line; combining the error function and the projection error function of the three-dimensional straight line, and iteratively solving by using a Gauss-Newton method to obtain an iteratively updated pose increment.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, simultaneous localization and mapping (SLAM), and autonomous robot navigation, and in particular to a joint visual odometry method based on edge-line features using PEdger++ edge probability confidence propagation. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) aims to enable robots to achieve accurate pose estimation and environmental modeling in unknown environments using only onboard sensors. It is a fundamental core technology in fields such as autonomous driving, robot navigation, and spatial computing. Among numerous sensor solutions, visual SLAM has become a research hotspot due to its advantages such as rich camera information, low power consumption, and ease of deployment. Visual Odometry (VO), as the front end of a visual SLAM system, continuously tracks camera pose by analyzing the relative motion of adjacent image frames. Its robustness and accuracy provide crucial prior constraints for subsequent back-end optimization and mapping modules.

[0003] In the early stages of visual odometry development, mainstream approaches were divided into feature-point methods and direct methods. Feature-point methods rely on extracting corner points or patches from images for matching, but in weakly textured environments such as corridors and white walls, the reduced number of extracted features often leads to tracking loss. Direct methods skip the feature extraction stage and directly use pixel photometric errors to solve for pose, but are prone to model failure under drastic lighting changes or violent camera movements. In contrast, edges and line segments are insensitive to lighting changes and maintain strong structural invariance even in weakly textured man-made architectural environments. However, existing visual odometry based on edge features or joint point-line features still struggles to distinguish between real physical geometric edges and complex texture noise in real-world complex scenes. Furthermore, existing feature selection strategies based on information entropy only consider the uniformity of point distribution in image space, leading to optimizers easily introducing uniformly distributed but physically unstable noise points. Furthermore, traditional point-line joint VO, when performing back-end nonlinear optimization, usually assumes that all line segment features have equal covariance, or only assigns weights empirically based on the line segment length, ignoring the fact that long line segments may also be strong noise artifacts, which can easily lead to trajectory drift in pose calculation.

[0004] In line segment feature extraction, traditional methods such as LSD or Hough transform, which are based on global search or local pixel gradients, are not only susceptible to interference from lighting and complex textures, resulting in severe fragmentation and artifact line segments, but also fail to utilize high-order semantic probabilities to guide the direction of feature extraction. Furthermore, when feeding the extracted line segments into the backend optimization, there is a lack of rigorous mathematical derivation to transform the uncertainty of the underlying features into the covariance matrix of the line segments. This leads to real and false line segments being assigned the same weight in pose calculation, further exacerbating the degradation problem of the system in complex environments.

[0005] Existing technologies, such as the one disclosed in "Publication No. CN116245899A, entitled 'A Dynamic Visual Odometry Method Based on Canny Edges'", construct the current frame and keyframes from image data; perform instance segmentation on the color image of the keyframes; determine whether there are potential high-dynamic objects, and if so, prevent missed detection of high-dynamic objects; otherwise, extract and classify Canny edges based on image segmentation; update Canny edge weights; and perform 2D-3D registration and pose estimation of the image. Its drawbacks are that the binary hard truncation loses the probability confidence of edges in continuous space, making it prone to false detections and breaks in environments with changing lighting or weak textures. Furthermore, the cascaded architecture of "segmentation first, detection later" results in excessive computational overhead, making it difficult to meet the real-time positioning and tracking requirements of mobile robots, drones, or lightweight embedded terminals. It should be noted that the PEDger++ (full name Practical Edge Detector ++) of this technology is an enhanced and upgraded practical edge detector. It is a new deep learning framework for practical edge detection. Its core is to achieve high accuracy, fast inference and lightweight model by integrating a variety of "cross information".

[0006] Therefore, there is an urgent need to propose a logically simple, accurate and reliable joint visual odometry method based on PEdger++ edge probability confidence propagation of edge-line features. Summary of the Invention

[0007] To address the aforementioned problems, the present invention aims to provide a joint visual odometry method for edge-line features based on PEDger++ edge probability confidence propagation. The technical solution adopted by the present invention is as follows: The edge-line feature joint visual odometry method based on PEDger++ edge probability confidence propagation includes the following steps: Acquire video images and obtain the current image frame. Use the PEdger++ algorithm to extract edge features from the current image frame to obtain an edge probability image. Obtain the probability confidence of edge feature points in the edge probability image, and calculate a precision threshold that is greater than or equal to a preset threshold. The sub-pixel geometric coordinates; the preset precision threshold The range of values ​​is ; An error function is constructed based on the distance transformation using sub-pixel geometric coordinates and the probability confidence of edge feature points. Filter the edge probability image to obtain a probability confidence threshold greater than the preset threshold. The local extreme points are used as seed points; two-dimensional line features are extracted using the seed points, and a projection error function of the three-dimensional line is constructed; the preset probability confidence threshold is used. The range of values ​​is ; By combining the joint error function and the projection error function of the three-dimensional line, and using the Gauss-Newton method for iterative solution, the pose increment for iterative updates is obtained. Compared with existing technologies, this invention has the following advantages: This invention acquires video images and obtains the current image frame. It then uses the PEdger++ algorithm to extract edge features from the current image frame to obtain an edge probability image. This effectively introduces deep high-order semantic priors. Compared with traditional hard threshold edge detection, it fundamentally suppresses texture noise, reflection, and illumination abrupt changes in complex environments, ensuring the integrity of edge structure extraction.

[0008] This invention obtains the probability confidence of edge feature points in the edge probability image and calculates sub-pixel geometric coordinates that are greater than or equal to a preset precision threshold. It achieves efficient and unique feature occupancy in discrete pixel space, which not only effectively avoids redundant and overlapping features on the same edge, but also eliminates complex sub-pixel interpolation calculations, significantly reducing the computational overhead of the algorithm front end.

[0009] This invention constructs an error function based on sub-pixel geometric coordinates and the probability confidence of edge feature points through distance transformation. This enables "soft-weighted" quantization of edge quality at the single-pixel level, allowing the error function to dynamically adjust constraint weights according to semantic confidence, thereby improving the system's tracking performance for weak edges and ambiguous structures.

[0010] This invention filters out local extreme points in the marginal probability image that are greater than a preset probability confidence threshold and uses them as seed points. It uses the seed points to extract two-dimensional line features and constructs a projection error function for three-dimensional straight lines. It uses gridding and non-maximum suppression mechanisms to ensure that the seed points are evenly distributed in the space of the entire image. Combined with Plück coordinate modeling, it provides strong structured spatial constraints at the multi-view geometric level.

[0011] This invention combines the error function and the projection error function of a 3D line, and uses the Gauss-Newton method to iteratively solve for the pose increment. Its advantage is that it perfectly integrates the geometric complementarity of point-level close registration and line-level macrostructure, enabling the state estimator (back-end optimization) to maintain extremely high repeatability and anti-drift capability in high dynamic and weak texture scenes. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope of protection. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a logic flowchart of the present invention.

[0014] Figure 2 This is a schematic diagram of the framework of the present invention.

[0015] Figure 3 This is a diagram of the network model structure of the present invention.

[0016] Figure 4 These are the original images used in the experiments of this invention.

[0017] Figure 5 This is a result image of edge feature detection in an image using the PEdger++ method of the present invention.

[0018] Figure 6 This is a diagram showing the results of detecting line features in an image using the DeepLSD algorithm.

[0019] Figure 7 This is a diagram showing the results of line feature detection in an image using the line feature detection algorithm of this invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the present invention will be further described below with reference to the accompanying drawings and embodiments. The embodiments of the present invention include, but are not limited to, the following embodiments. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0021] In this embodiment, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0022] The terms "first" and "second," etc., used in the specification and claims of this embodiment are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0023] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0024] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0025] like Figures 1 to 7 As shown, this embodiment provides a joint visual odometry method for edge-line features based on PEdger++ edge probability confidence propagation. Its core objective is inter-frame camera pose estimation, and it is strictly constrained by a pre-deployed PEdger++ deep learning edge detection network. The method includes the following steps: The first step is to acquire video images and obtain the current image frame. The PEdger++ algorithm is then used to extract edge features from the current image frame to obtain an edge probability image. Here, the PEdger++ algorithm employs the PEdger++ deep edge detection network; the PEdger++ deep edge detection network includes an encoding module, a recurrent module, a decoding module, a fusion module, and a subpixel thinning module connected in sequence. The encoding module receives the current image frame and uses the recurrent module, decoding module, and fusion module to extract edge features and obtain the edge probability image; the subpixel thinning module solves for the subpixel geometric coordinates from the edge probability image. The encoding module includes a first convolutional layer, a second convolutional layer, and a third convolutional layer connected in sequence; the first convolutional layer is connected to the current image frame. The looping module includes a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer, an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer, an eleventh convolutional layer, a twelfth convolutional layer, a thirteenth convolutional layer, a fourteenth convolutional layer, a max pooling layer, and a channel attention layer connected in sequence; the fourth convolutional layer is connected to the third convolutional layer, and the channel attention layer is connected to the decoding module. The subpixel thinning module includes a peak anchoring layer, a Taylor unrolling layer, and an offset resolution layer connected in sequence; the peak anchoring layer is connected to the fusion module.

[0026] Here, obtaining the edge probability image includes: acquiring the current image frame acquired by the acquisition device at time k. ; Set the current image frame The input is fed into the pre-trained Pedger++ deep edge detection network to obtain the edge probability map matrix. Here, the marginal probability graph matrix The current image frame The mapping in the semantic probability space, in the current image frame Each point originally had only one pure physical attribute—brightness or RGB color. After deep feature extraction and activation by PEDger++, the original color information of each point was stripped away, and instead, a probability value of [0,1] was assigned to it. The probability values ​​of all points were combined to form the edge probability map matrix. .

[0027] The second step is to obtain the probability confidence of the edge feature points in the edge probability image (PEdger++ outputs essentially a "probability map", which naturally contains the confidence of each edge point), and calculate a precision threshold greater than or equal to the preset threshold. The sub-pixel geometric coordinates; the preset precision threshold The range of values ​​is Specifically, this includes: Marginal probability map matrix along gradient direction Perform a local extremum search to obtain several candidate edge pixels; let the integer coordinates of one of the candidate edge pixels be... And the probability confidence of the candidate edge pixels. Where T represents the matrix transpose operation; The selected candidate edge pixels are used as the current center pixel.

[0028] Within the spatial local neighborhood of the current center pixel, an approximate Hessian matrix and Jacobian vector are constructed using adjacent probability scores; the sub-pixel offset is analytically obtained using a one-dimensional second-order Taylor expansion. Its expression is: ; in, This represents the probability confidence level of the current center pixel itself. This represents the probability confidence of the current center pixel in the negative gradient direction of its previous neighbor pixel. This represents the probability confidence of the current center pixel in the direction of the gradient's next neighbor pixel. subpixel offset Truncate to the range [-0.5, 0.5] and obtain sub-pixel geometric coordinates. This improvement enhances the accuracy of the network's output coordinates, thus providing an error base with smooth gradients for subsequent geometric optimization of visual odometry. The third step involves constructing an error function based on sub-pixel geometric coordinates and the probability confidence of edge feature points through distance transformation. Specifically: (31) Obtain the sub-pixel geometric coordinates Feature joint score function Its expression is: ; ; in, Represents the sub-pixel geometric coordinates The probability confidence level at the corresponding nearest neighbor integer pixel location, assigned by the PEDger++ depth edge detection network; Indicates the current image frame The sub-pixel geometric coordinates described in After spatial alignment matching with the edge features of historical reference frames, the current image frame The information entropy reduced by the spatial feature distribution of the edge probability image; This represents the balance coefficient corresponding to the probability confidence level; This represents the balance coefficient corresponding to the decrease in information entropy.

[0029] (32) Based on the feature joint score function Calculate the joint score of any sub-pixel geometric coordinate, sort the joint scores in descending order, and select the top N sub-pixel geometric coordinates to form the target edge set. Where N is a positive integer greater than 1.

[0030] (33) Obtain the target edge point set Any sub-pixel coordinate, and the corresponding depth value .

[0031] (34) Obtain the target edge point set Probability confidence of the true geometric edge at any sub-pixel coordinate .

[0032] (35) Based on Lie group algebra The current image frame subpixel coordinates Projected onto the previous reference frame In the keyframe, the corresponding pixel coordinates are obtained, and their expression is: ;in, Indicates the current image frame subpixel coordinates Projected onto the previous reference frame The corresponding pixel coordinates; This represents a reprojection function based on camera intrinsics and pose transformation, which converts the current frame's integer image coordinates. The depth value is combined with backprojection to restore it to a three-dimensional space point, and then the Lie group pose is used for rigid body transformation. Finally, it is reprojected onto the two-dimensional pixel plane of the historical reference frame to solve the inter-frame matching coordinates.

[0033] (36) Generate a range field image in the reference frame. The distance error after reprojection is obtained. Its expression is: .

[0034] (37) In order to simultaneously resist both “geometric outliers caused by large displacements” and “semantic artifacts caused by complex textures”, the error function of any feature point is obtained by combining the probability confidence of the feedforward input of the Huber kernel function and the PEDger++ deep edge detection network. Its expression is: ; in, This represents the Huber kernel function; represents the probability confidence of the feedforward input to the PEdger++ deep edge detection network; i represents the i-th feature point currently being calculated.

[0035] (38) Construct a nonlinear least squares objective cost function for feature point pose estimation Its expression is: ; in, This represents the function for finding the minimum point.

[0036] The fourth step is to filter out individuals from the edge probability image whose probability confidence level is greater than a preset threshold. The local extreme points are used as seed points; two-dimensional line features are extracted using the seed points, and a projection error function of the three-dimensional line is constructed; the preset probability confidence threshold is used. The range of values ​​is .

[0037] (41) Filter the edge probability image to obtain a probability confidence threshold greater than the preset threshold. The local extrema of the edge detection network are identified and used as seed points; the probability confidence level corresponding to the seed point is obtained by the PEDger++ deep edge detection network. .

[0038] (42) Within the local pixel window where the seed point is located, the probability confidence level corresponding to the seed point is given by the PEdger++ deep edge detection network. The structure tensor matrix of the image gradient is calculated using the weights. Its expression is: ; in, This represents the pixel grayscale gradient of the image in the x-direction; This represents the pixel grayscale gradient of the image in the y-direction; x is the horizontal direction of the image; y is the vertical direction of the image, and the origin (0, 0) is located at the upper left corner of the image.

[0039] (43) The structure tensor matrix of the image gradient Perform eigenvalue decomposition and take the eigenvector corresponding to the smallest eigenvalue as the true edge tangent direction of the seed point.

[0040] (44) Perform bidirectional pixel-level step tracking along the true edge tangent direction of the seed point on the edge probability image; preset the pixel-level step tracking stop condition and obtain the two-dimensional observation line segment.

[0041] (45) Extract the homogeneous coordinates of all pixels on the two-dimensional observation line segment. The probability confidence scores corresponding to the homogeneous coordinates of pixels are used to construct the two-dimensional information moments of the two-dimensional observation line segments. Its expression is: .

[0042] in, This represents the number of probability pixels obtained through pixel-level step tracking.

[0043] (46) Represent the three-dimensional spatial lines in the reference frame using Plück coordinates as follows: ,in, The normal vector representing a line in three-dimensional space. It represents the direction vector of a straight line in three-dimensional space.

[0044] (47) Calculate the algebraic projection residual vector between the three-dimensional spatial line of the reference frame projected onto the current image plane and the two-dimensional observation line segment. Its expression is: ; in, The algebraic projection residual vector representing the feature of the j-th line; Represents the second line segment observed in two dimensions. The algebraic distance error from a pixel to the theoretically projected two-dimensional line. Values ​​1, 2, ... ; Represents the second line segment observed in two dimensions. The x-coordinate of a pixel; Represents the second line segment observed in two dimensions. The ordinate of a pixel; (This represents the dimension coefficient in the x-direction of the general form equation of the theoretically projected two-dimensional line, indicating the horizontal axis direction). The dimension coefficient in the y-direction of the general equation representing the longitudinal direction of a theoretically projected two-dimensional straight line. This represents the constant term intercept coefficient of the theoretically projected two-dimensional straight line. Here, suppose that the j-th two-dimensional observation line segment obtained by pixel-level step tracking contains N l Given a probability pixel, extract the homogeneous coordinates of all pixels on the line segment, represented as a set. , of which The homogeneous coordinates of each pixel are Calculate the algebraic distance error from each pixel on the two-dimensional observation line segment to the theoretically projected two-dimensional straight line l. The calculation formula is as follows: Combine all N on the two-dimensional observation line segment l The algebraic distance error of each pixel is used to construct the algebraic projection residual vector of the j-th line feature. .

[0045] (48) Introducing two-dimensional information moments of two-dimensional observation line segments Construct the Maharanobis distance cost term for line features Its expression is: . The fifth step involves combining the joint error function and the projection error function of the 3D line, and then using the Gauss-Newton method to iteratively solve for the iterative pose update increment, expressed as: ; ; ; ; in, H represents the scalar weights calculated by the Huber kernel for the current residual in the Huber kernel function; H represents the information matrix. Indicates the pose increment; Represents the residual tension vector; The Jacobian matrix representing the distance error of edge features with respect to the pose Lie group algebra; This represents the preset robust kernel function threshold; Indicates the current image frame The total number of valid edge points extracted and used in pose optimization; Let represent the Jacobian matrix of the feature projection error of the j-th line with respect to the Lie algebra of the camera pose.

[0046] Iteratively update pose increment The process continues until convergence, outputting a refined camera relative pose. Finally, a method is introduced to determine whether to add a keyframe for each current frame by evaluating the camera's tracking quality: the edge feature pixels from the previous N frames, and the line feature pixels obtained by equidistant divergence sampling of the line features from the previous N frames, are unified as a historical geometric projection point set and reprojected into the current frame. The frequency of each pixel being covered by the projection in the current frame is counted, with the frequency ranging from 0 to N. A variable W is defined. q Let W0 be the total number of pixels whose reprojection frequency is exactly q (q∈[0,N]), where W0 refers to the number of newly added pixels not covered by any historical features. Finally, this is determined by evaluating the historical co-view pixel set W. q The proportional relationship between the current frame and the uncovered pixel set W0 is used to dynamically determine whether the current frame needs to be inserted as a new keyframe (i.e., a proportional threshold can be set manually).

[0047] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any changes made based on the design principles of the present invention, or any non-creative modifications made thereon, shall fall within the scope of protection of the present invention.

Claims

1. A joint visual odometry method for edge-line features based on Pedger++ edge probability confidence propagation, characterized in that, Includes the following steps: Acquire video images and obtain the current image frame. Use the PEdger++ algorithm to extract edge features from the current image frame to obtain an edge probability image. Obtain the probability confidence of edge feature points in the edge probability image, and calculate a precision threshold that is greater than or equal to a preset threshold. The sub-pixel geometric coordinates; the preset precision threshold The range of values ​​is ; An error function is constructed based on the distance transformation using sub-pixel geometric coordinates and the probability confidence of edge feature points. Filter the edge probability image to obtain a probability confidence threshold greater than the preset threshold. The local extreme points are used as seed points; two-dimensional line features are extracted using the seed points, and a projection error function of the three-dimensional line is constructed; the preset probability confidence threshold is used. The range of values ​​is ; The combined error function and the projection error function of the 3D line are used, and the Gauss-Newton method is employed to iteratively solve for the pose increment of the iterative update.

2. The edge-line feature joint visual odometry method based on PEdger++ edge probability confidence propagation according to claim 1, characterized in that, The PEdger++ algorithm employs the PEdger++ deep edge detection network. The PEdger++ deep edge detection network includes an encoding module, a loop module, a decoding module, a fusion module, and a subpixel thinning module connected in sequence. The encoding module receives the current image frame and uses the loop module, decoding module, and fusion module to extract edge features and obtain an edge probability image. The subpixel thinning module solves the edge probability image to obtain subpixel geometric coordinates.

3. The edge-line feature joint visual odometry method based on PEdger++ edge probability confidence propagation according to claim 2, characterized in that, The encoding module includes a first convolutional layer, a second convolutional layer, and a third convolutional layer connected in sequence; the first convolutional layer is connected to the current image frame; the loop module includes a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer, an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer, an eleventh convolutional layer, a twelfth convolutional layer, a thirteenth convolutional layer, a fourteenth convolutional layer, a max pooling layer, and a channel attention layer connected in sequence; the fourth convolutional layer is connected to the third convolutional layer, and the channel attention layer is connected to the decoding module; the subpixel thinning module includes a peak anchoring layer, a Taylor unrolling layer, and an offset resolution layer connected in sequence; the peak anchoring layer is connected to the fusion module.

4. The edge-line feature joint visual odometry method based on PEdger++ edge probability confidence propagation according to claim 2 or 3, characterized in that, Acquire video images and obtain the current image frame. Use the PEdger++ algorithm to extract edge features from the current image frame to obtain an edge probability image, including: Acquire the current image frame acquired by the acquisition device at time k. ; The current image frame The input is fed into the pre-trained Pedger++ deep edge detection network to obtain the edge probability map matrix. .

5. The edge-line feature joint visual odometry method based on PEdger++ edge probability confidence propagation according to claim 4, characterized in that, Obtain the probability confidence of edge feature points in the edge probability image, and calculate a precision threshold that is greater than or equal to a preset threshold. The sub-pixel geometric coordinates include: Marginal probability map matrix along gradient direction Perform a local extremum search to find several candidate edge pixels; let the integer coordinates of one of the candidate edge pixels be... And the probability confidence of the candidate edge pixels. Where T represents the matrix transpose operation; Select the candidate edge pixels as the current center pixel; Within the spatial local neighborhood of the current center pixel, an approximate Hessian matrix and Jacobian vector are constructed using adjacent probability scores; the sub-pixel offset is analytically obtained using a one-dimensional second-order Taylor expansion. Its expression is: ； in, This represents the probability confidence level of the current center pixel itself. This represents the probability confidence of the current center pixel in the negative gradient direction of its previous neighbor pixel. This represents the probability confidence of the current center pixel in the direction of the gradient's next neighbor pixel. subpixel offset Truncate to the range [-0.5, 0.5] and obtain sub-pixel geometric coordinates. .

6. The edge-line feature joint visual odometry method based on PEdger++ edge probability confidence propagation according to claim 5, characterized in that, An error function is constructed based on sub-pixel geometric coordinates and the probability confidence of edge feature points, including: Obtain sub-pixel geometric coordinates Feature joint score function Its expression is: ； ； in, Represents the sub-pixel geometric coordinates The probability confidence level at the corresponding nearest neighbor integer pixel location, assigned by the PEDger++ depth edge detection network; Indicates the current image frame The sub-pixel geometric coordinates described in After spatial alignment matching with the edge features of historical reference frames, the current image frame The information entropy reduced by the spatial feature distribution of the edge probability image; This represents the balance coefficient corresponding to the probability confidence level; This represents the balance coefficient corresponding to the decrease in information entropy. Based on the joint score function of features Calculate the joint score of any sub-pixel geometric coordinate, sort the joint scores in descending order, and select the top N sub-pixel geometric coordinates to form the target edge set. Where N is a positive integer greater than 1; Obtain the target edge point set Any sub-pixel coordinate, and the corresponding depth value ; Obtain the target edge point set Probability confidence of the true geometric edge at any sub-pixel coordinate ; According to Lie group algebra The current image frame sub-pixel coordinates Projected onto the previous reference frame In the middle, the corresponding pixel coordinates are obtained, and the expression is: ;in, Indicates the current image frame sub-pixel coordinates Projected onto the previous reference frame The corresponding pixel coordinates; This represents the reprojection function based on camera intrinsic parameters and pose transformation; Generate a range field image in the reference frame The distance error after reprojection is obtained. Its expression is: ； By combining the Huber kernel function with the probability confidence score fed forward from the Pedger++ deep edge detection network, the error function for any feature point is obtained. Its expression is: ； in, This represents the Huber kernel function; This represents the probability confidence level of the feedforward input to the PEDger++ deep edge detection network; i represents the i-th feature point currently being computed. Constructing a nonlinear least squares objective cost function for feature point pose estimation Its expression is: ； in, This represents the function for finding the minimum point.

7. The edge-line feature joint visual odometry method based on PEdger++ edge probability confidence propagation according to claim 6, characterized in that, Filter the edge probability image to obtain a probability confidence threshold greater than the preset threshold. The local extreme points are used as seed points; The projection error function for a 3D straight line is constructed by extracting 2D line features using seed points, including: Filter the edge probability image to obtain a probability confidence threshold greater than the preset threshold. The local extrema of the edge detection network are identified and used as seed points; the probability confidence level corresponding to the seed point is obtained by the PEDger++ deep edge detection network. ; Within the local pixel window where the seed point is located, the probability confidence level assigned to the seed point by the PEdger++ deep edge detection network is used. The structure tensor matrix of the image gradient is calculated using the weights. Its expression is: ； in, This represents the pixel grayscale gradient of the image in the x-direction; This represents the pixel grayscale gradient of the image in the y-direction; where x is the horizontal direction of the image; and y is the vertical direction of the image. The structure tensor matrix of image gradient Perform eigenvalue decomposition and take the eigenvector corresponding to the smallest eigenvalue as the true edge tangent direction of the seed point; Bidirectional pixel-level step tracking is performed on the edge probability image along the true edge tangent direction of the seed point; a preset pixel-level step tracking stop condition is set, and a two-dimensional observation line segment is obtained; Extract the homogeneous coordinates of all pixels on the two-dimensional observation line segment. The probability confidence scores corresponding to the homogeneous coordinates of pixels are used to construct the two-dimensional information moments of the two-dimensional observation line segments. Its expression is: ； in, This represents the number of probability pixels obtained from pixel-level step tracking; The three-dimensional spatial lines in the reference frame are represented using Plück coordinates. ,in, The normal vector representing a line in three-dimensional space. Represents the direction vector of a line in three-dimensional space; The algebraic projection residual vector between the three-dimensional spatial line of the reference frame projected onto the current image plane and the two-dimensional observation line segment is calculated as follows: ； in, The algebraic projection residual vector representing the feature of the j-th line; Represents the first two-dimensional observation line segment. The algebraic distance error from a pixel to the theoretically projected two-dimensional line. Values ​​1, 2, ... ; Represents the first two-dimensional observation line segment. The x-coordinate of a pixel; Represents the first two-dimensional observation line segment. The ordinate of a pixel; (This represents the dimension coefficient in the x-direction of the general form equation of the theoretically projected two-dimensional line, indicating the horizontal axis direction). The dimension coefficient in the y-direction of the general equation representing the longitudinal direction of a theoretically projected two-dimensional straight line. The constant term intercept coefficient represents the theoretically projected two-dimensional straight line; Introducing two-dimensional information moments of two-dimensional observation line segments Construct the Maharanobis distance cost term for line features Its expression is: .

8. The edge-line feature joint visual odometry method based on PEdger++ edge probability confidence propagation according to claim 7, characterized in that, The joint error function and the projection error function of the 3D line are used, and the Gauss-Newton method is employed for iterative solution to obtain the iterative pose update increment, the expression of which is: ； ； ； ； in, H represents the scalar weights calculated by the Huber kernel for the current residual in the Huber kernel function; H represents the information matrix. Indicates the pose increment; Represents the residual tension vector; The Jacobian matrix representing the distance error of edge features with respect to the pose Lie group algebra; This represents the preset robust kernel function threshold; Indicates the current image frame The total number of valid edge points extracted and used in pose optimization; Let represent the Jacobian matrix of the feature projection error of the j-th line with respect to the Lie algebra of the camera pose.

Images