A Robust Quadratic Surface Initialization-Based Outdoor Visual Localization and Mapping Method

Abstract

This invention provides an outdoor visual localization and mapping method based on robust quadratic surface initialization, belonging to the field of visual SLAM technology. The invention first initializes the target object as an isometric sphere through two-dimensional observation, and then optimizes the sphere to an ellipsoid using subsequent observations. The system uses a data association algorithm based on semantic features, ellipsoidal projection intersection-over-union ratio, and object motion prediction to ensure accurate association of objects in consecutive frames. Finally, a joint optimization strategy is used to optimize camera pose, quadratic surface parameters, and map points, ultimately constructing a complete object-level semantic map. This invention overcomes interference from object detection noise and dynamic occlusion, and effectively overcomes the dependence of traditional object initialization algorithms on planar assumptions, adapting to complex real-world road conditions and applicable to the fields of visual perception and visual localization in autonomous driving and intelligent robots.

Description

Technical Field

[0001] This invention relates to the field of visual SLAM technology, and in particular to an outdoor visual localization and mapping method based on robust quadratic surface initialization. Background Technology

[0002] Environmental perception is a hot topic in autonomous driving and mobile robotics. In recent years, with the development of deep learning, accurate and efficient image semantic information has been widely used in the field of SLAM. Semantic information can improve the robustness and accuracy of SLAM system localization and plays an important role in complex tasks such as high-level semantic environmental perception and human-computer interaction. Compared with traditional cubes, quadratic surfaces have a complete projective geometric mathematical description and can accurately express the shape, scale, orientation, and position of objects. However, the current advanced quadratic surface initialization methods rely on the detection results of keyframes, and the solution methods rely on closed constraint parameterization. When there is noise in the observation and object occlusion, the parameterization solution values ​​are unstable, leading to initialization failure. Most current quadratic surface SLAM data associations are based on static environment assumptions. When there are dynamic scenes, incorrect object associations lead to deterioration in system localization and object reconstruction performance.

[0003] Introducing quadric surfaces into SLAM systems and enhancing their perception capabilities has become a current research hotspot. The paper "Quadricslam: Dual quadrics from object detections as landmarks in object-oriented SLAM," IEEE Robotics and Automation Letters, vol.4, no.1, pp.1–8, 2018, first introduced quadric surfaces into SLAM systems and proposed a solution method using keyframe object detection observations and multi-view geometry to obtain quadric surface landmarks. However, this algorithm requires sufficient disparity in the keyframes observing the object and is susceptible to observation noise disturbances, leading to instability in the quadric surface solution. The paper "Robust object-based SLAM for high-speed autonomous navigation," in 2019 International Conference on Robotics and Automation (ICRA), IEEE, 2019, pp.669–675, proposes a quadric surface initialization method based on planar constraints, assuming outdoor scenes and object motion in a plane. However, this algorithm relies on a forward-observed vehicle model, and its robustness needs improvement. The paper "Real-time monocular object-model aware sparseslam, in 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 7123–7129" proposes a joint optimization framework that combines multiple constraints of points, surfaces, and quadratic surfaces to optimize quadratic surfaces. However, since the prior shape of the object is estimated by a deep network, the computational cost is high.

[0004] The paper "Object-aware slam based on efficient quadric initialization and joint data association, IEEE Robotics and Automation Letters, pp.1–8, 2022" proposes to integrate object detection and surface element constraints in quadric surface initialization, thereby overcoming the requirements of multi-frame observation and wide-angle observation. However, this initialization method is mainly designed for RGB-D camera design in indoor scenes. The paper "Oa-slam: Leveraging objects for camera relocalization in visual slam, in 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2022, pp. 720–728" proposes a quadric surface initialization method based on a coarse-to-fine strategy to reconstruct objects as spheres. However, this algorithm is mainly suitable for small-scale indoor scenes, and it fails in outdoor scenes due to frequent erroneous data associations. The paper "Accurate and robust object slam with 3d quadric landmark reconstruction in outdoors, IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1534–1541, 2021" proposes a quadric surface initialization algorithm based on parameter separation. This algorithm is based on the road plane assumption, constrains the rotation of the observed vehicle, and then initializes the quadric surface by independently estimating the rotation and translation. However, for complex outdoor scenes, the road plane assumption is often not satisfied, resulting in errors in the rotation of the initialized quadric surface. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an outdoor visual positioning and mapping method based on robust quadratic surface initialization.

[0006] An outdoor visual localization and mapping method based on robust quadratic surface initialization includes the following steps:

[0007] Step 1: Use a SLAM system to receive RGB images captured by a stereo camera;

[0008] Step 2: Perform binocular RGB image feature extraction;

[0009] The left eye image is fed into the deep network Yolact to obtain the object detection box B and semantic mask M, and optical flow extraction is performed on the feature points within the detection box for the previous and next frames.

[0010] Step 3: Perform initial pose estimation for the camera;

[0011] ORB feature points extracted from binocular images are used to initialize point cloud maps through feature matching and feature point triangulation. Then, feature points from subsequent images are used to construct the reprojection error from 3D map points to 2D image features based on ORB features. The camera pose is solved iteratively using the Levenberg–Marquardt algorithm.

[0012] Step 4: Use an object data association algorithm to associate the detection results of the current frame with the objects in the map;

[0013] Step 4.1: Object motion tracking;

[0014] Motion prediction using detection boxes based on Kalman filtering: The motion observation of an object satisfying uniform motion is defined as... Where u and v are the two-dimensional center coordinates of the detection box; h and w are the height and width of the detection box, respectively, and the size of the detection box is s = h·w; the aspect ratio of the detection box is r = h / w, which remains unchanged during the movement. These represent the motion rates of the detection box along the axes of the two-dimensional image; This represents the rate of change of the bounding box size; for object bounding boxes with associated observations, a Kalman filter is used for state updates; if an object bounding box has no associated observations, the current state is not updated; the final updated object bounding box... for:

[0015]

[0016] in, This represents the posterior estimate of the observation, i.e., the value after being updated by Kalman filtering.

[0017] Step 4.2: Object motion detection;

[0018] Optical flow vector boundaries are used to determine the motion attributes of objects; for features within the semantic mask M, Lucas-Carnard optical flow (LK optical flow) is used for tracking, defining the feature coordinates of two consecutive frames as x. i and x i+1 Object feature coordinates x i+1 Use the following judgment:

[0019]

[0020] Among them, S i+1This represents the dynamic detection result of feature points. The binary values ​​0 and 1 represent the static and dynamic attributes of the feature points, respectively. t represents the relative translation between two frames, z represents the pixel depth value obtained through a stereo matching algorithm, and K is the camera intrinsic parameter matrix. min As the lower bound of the characteristic coordinates, x max The upper bound of the characteristic coordinates is defined, and d is the intermediate quantity for judging the optical flow vector boundary; z is defined as... max z min The upper and lower bounds of the feature point depth are calculated by formula (2). When the coordinates of a feature point fall outside the optical flow vector boundary, the feature point is determined to be a dynamic feature. The initial feature points of the object are set to static, and the motion state of the object is updated by the proportion of the dynamic points to which the object belongs.

[0021] Step 4.3: and object association based on assignment;

[0022] Using the Hungarian algorithm on the cost matrix The optimal allocation of M bounding boxes and N ellipsoids in the current frame is calculated; the cost matrix is ​​defined by the following distance error:

[0023]

[0024] Where, θ i Represents the distance weight, a ij Let be the element in the i-th row and j-th column of the cost matrix. Different distances are defined as follows: Define the feature set for LK optical flow tracing as semantic intrapoint distance. The feature set of the current frame is Define size(.) as the number of points in the calculation area. Let `in` be the semantic mask corresponding to the i-th object in the t-th frame of the image; `in(.)` represents the number of statistical feature points. Semantic In-Range Distance for;

[0025]

[0026] To detect IoU distance, IoU(.) is defined as the intersection-union ratio operation for calculating the two-dimensional detection boxes. This represents the i-th object detection box on the t-th image frame. The operation is performed on the bounding rectangle of the ellipse. Let be the parameters of the dual quadratic surface corresponding to the j-th ellipsoid in the map, and P be the camera projection matrix of the ellipsoid onto the image; the intersection-union ratio (IUU) is calculated using the reprojection of the j-th object and the detection box of the i-th object in the current frame:

[0027]

[0028] To predict the IoU distance; define The updated detection bounding box for the j-th object in the t-th frame of the image. Let be the i-th detection box in the t-th frame. The cross-union ratio (CUI) of the object detection box updated based on Kalman filtering in formula (1) and the i-th object detection box in the current frame is used.

[0029]

[0030] Step 5: Initialize the object as an isometric sphere using the 2D object detection boxes of the current frame and neighboring frames. Obtain its center position by triangulating the center point of the detection box. The isometric shape is obtained by the mean of the projections of the detection box at the center position. The axis length is calculated as follows:

[0031]

[0032] Among them, t iz ω represents the depth value of the i-th object in the current frame's camera coordinate system. i and h i f represents the width and height of the detection box for the i-th 2D object; x and f y is the camera intrinsic parameter; n is the number of consecutive frame observations of the associated object.

[0033] After the sphere is initialized, it is updated using subsequent observations from a 2D object detection box. The 9-DOF parameters of the quadratic surface are refined, and the quadratic surface parameter q is iteratively updated using a nonlinear optimization method. The error equation is:

[0034]

[0035] Among them, e(B i ,q) represents the reprojection error of the object detection box and the ellipsoid projected onto the current frame's 2D bounding box, Σ o Let be the covariance matrix of the reprojection error term; A is the estimated axis length of the ellipsoid. Σ is the prior axis length; a is the covariance matrix of the axis length error term; q is the quadratic surface parameter; P i Let be the camera projection matrix for the i-th frame; define This is an operation on the circumscribed rectangle of an ellipse; the prior axis length is defined as...

[0036] Step 6: Optimize the parameters of the quadratic surface already constructed in the map using the object detection bounding boxes of the keyframes within a sliding window composed of keyframes; define the Gaussian description of the reprojection ellipse of the ellipsoid in the image frame as:

[0037] (x-μ) T Σ -1(x-μ)=1 (9)

[0038] Where, μ=(c x ,c y ) represents the coordinates of the center of the ellipse; Σ represents the variance of the ellipse, i.e., the axis length.

[0039] A joint optimization function is formed by combining the reprojection error constructed from the object detection box within the sliding window and the ellipsoid projection, as well as the error constructed from the ellipsoid center point and the object detection box center. The quadratic surface parameter q is iteratively updated through nonlinear optimization. The joint optimization function is defined as follows:

[0040]

[0041] in, Let be the inscribed ellipse of the i-th detection box; π is the camera projection equation; u i Let t be the coordinates of the center point of the i-th object detection box; j Σ represents the center position of the j-th ellipsoid; o Σ is the covariance matrix of the projection error term of the quadratic surface; t T is the covariance matrix of the center distance error term; i Let be the camera pose corresponding to the i-th frame; the sliding window length is n and i∈{1,2,...,n}. ε(.) is the Bhattacharyya distance between the two-dimensional projection of the ellipsoid and the observed object detection box in the current frame, defined as follows:

[0042]

[0043] Where, μ i Σ represents the center coordinates of the projected ellipse; i Represents the variance of an ellipse; `det` is the matrix determinant calculation operation;

[0044] Step 7: The SLAM backend performs joint optimization on the camera pose, quadratic surface parameters, and map points, utilizing the quadratic surface to optimize the camera pose. The map points are defined as P. k The characteristic observation is u k The quadratic surface parameter is q j The joint optimization function is defined as follows:

[0045]

[0046] Where, Σ o and Σ m These are the covariance matrices of the detection box reprojection error and the map point reprojection error, respectively.

[0047] Step 8: Output the camera pose and the object-level map containing quadratic surfaces.

[0048] The beneficial effects of adopting the above technical solution are as follows:

[0049] This invention provides an outdoor visual localization and mapping method based on robust quadratic surface initialization. Utilizing a robust quadratic initialization algorithm, it does not rely on planar assumptions and is robust to observation occlusion and noise interference. Furthermore, this invention proposes an automatic object data association algorithm that can detect object motion while associating objects across frames. To further improve the accuracy of quadratic surface reconstruction, the system uses an additional thread to optimize ellipsoid parameters within a local sliding window composed of keyframes, and employs a joint optimization framework to optimize camera pose, quadratic surface parameters, and point clouds in the local mapping thread to achieve global optimization, ultimately constructing an object-level map. Attached Figure Description

[0050] Figure 1 This is an architecture diagram of the outdoor visual positioning and mapping method in an embodiment of the present invention;

[0051] Figure 2 This is a comparison diagram of the initialization effects of the method in this embodiment of the invention and existing comparison algorithms;

[0052] Figure 3 This is the final object-level map constructed in this embodiment of the invention. Detailed Implementation

[0053] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0054] An outdoor visual localization and mapping method based on robust quadratic surface initialization, such as... Figure 1 As shown, it includes the following steps:

[0055] Step 1: Use a SLAM system to receive RGB images captured by a stereo camera;

[0056] Step 2: Perform binocular RGB image feature extraction;

[0057] The left eye image is fed into the deep network Yolact to obtain the object detection box B and semantic mask M, and optical flow extraction is performed on the feature points within the detection box for the previous and next frames.

[0058] Step 3: Perform initial pose estimation for the camera;

[0059] ORB feature points extracted from stereo images are used to initialize point cloud maps through feature matching and feature point triangulation. Then, feature points from subsequent images are used to construct the reprojection error from 3D map points to 2D image features based on ORB features. Subsequent images refer to images received after stereo images for feature extraction. The camera pose is solved iteratively using the Levenberg-Marquardt algorithm.

[0060] Step 4: To ensure correct object initialization and subsequent observation-based optimization, an object data association algorithm is used to associate the detection results of the current frame with the objects in the map.

[0061] Step 4.1: Object motion tracking;

[0062] To achieve accurate association of moving objects, a detection box motion prediction method based on Kalman filtering is used: The motion observation of an object satisfying uniform motion is defined as... Where u and v are the two-dimensional center coordinates of the detection box, h and w are the height and width of the detection box, respectively, and the size of the detection box is s = h·w; the aspect ratio of the detection box is r = h / w, which remains unchanged during the movement. These represent the motion rates of the detection box along the axes of the two-dimensional image; This represents the rate of change of the bounding box size; for object bounding boxes with associated observations, a Kalman filter is used for state updates; if an object bounding box has no associated observations, the current state is not updated; the final updated object bounding box... for:

[0063]

[0064] in, This represents the posterior estimate of the observation, i.e., the value after being updated by Kalman filtering.

[0065] Step 4.2: Object motion detection;

[0066] To overcome the limitation of epipolar constraints in determining the motion state of objects moving at the same speed relative to the camera, an optical flow vector bound (FVB) is used to determine the object's motion attributes. For features within the semantic mask M, the Lucas-Kanade (LK) method is used for optical flow tracking, defining the feature coordinates of two consecutive frames as x. i and x i+1 Object feature coordinates x i+1 Use the following judgment:

[0067]

[0068] Among them, S i+1This represents the dynamic detection result of feature points. The binary values ​​0 and 1 represent the static and dynamic attributes of the feature points, respectively. t represents the relative translation between two frames, z represents the pixel depth value obtained through a stereo matching algorithm, and K is the camera intrinsic parameter matrix. min , where x is the lower bound of the characteristic coordinates. max The upper bound of the characteristic coordinates is defined, and d is the intermediate quantity for judging the optical flow vector boundary; z is defined as... max z min As the upper and lower bounds of the feature point depth, z in this embodiment max =∞, z min =0.5 meters, the optical flow vector boundary is calculated by formula (2). When the coordinates of the feature point fall outside the optical flow vector boundary, the feature point is determined to be a dynamic feature. The initial feature point of the object is set to static, and the motion state of the object is updated by the proportion of the dynamic point to which the object belongs.

[0069] Step 4.3: and object association based on assignment;

[0070] Using the Hungarian algorithm on the cost matrix The optimal allocation of M bounding boxes and N ellipsoids in the current frame is calculated; the cost matrix is ​​defined by the following distance error:

[0071]

[0072] Where, θ i Represents the distance weight, a ij Let be the element in the i-th row and j-th column of the cost matrix. Different distances are defined as follows: Define the feature set for LK optical flow tracing as semantic intrapoint distance. The feature set of the current frame is Define size(.) as the number of points in the calculation area. Let `in` be the semantic mask corresponding to the i-th object in the t-th frame of the image; `in(.)` represents the number of statistical feature points. Semantic In-Range Distance for;

[0073]

[0074] To detect IoU distance, IoU(.) is defined as the intersection-union ratio operation for calculating the two-dimensional detection boxes. This represents the i-th object detection box on the t-th image frame. The operation is performed on the bounding rectangle of the ellipse. Let be the parameters of the dual quadratic surface corresponding to the j-th ellipsoid in the map, and P be the camera projection matrix of the ellipsoid onto the image. The intersection-union ratio (IUU) is calculated using the reprojection of the j-th object and the detection box of the i-th object in the current frame:

[0075]

[0076] To predict the IoU distance; define The updated detection bounding box for the j-th object in the t-th frame of the image. Let be the i-th detection box in the t-th frame. The cross-union ratio (CUI) of the object detection box updated based on Kalman filtering in formula (1) and the i-th object detection box in the current frame is used.

[0077]

[0078] Step 5: Initialize the object as an isometric sphere using the 2D object detection boxes of the current frame and neighboring frames. Obtain its center position by triangulating the center point of the detection box. The isometric shape is obtained by the mean of the projections of the detection box at the center position. The axis length is calculated as follows:

[0079]

[0080] Among them, t iz ω represents the depth value of the i-th object in the current frame's camera coordinate system. i and h i f represents the width and height of the detection box for the i-th 2D object; x and f y is the camera intrinsic parameter; n is the number of consecutive frame observations of the associated object.

[0081] After the sphere is initialized, it is updated using subsequent observations from a 2D object detection box. The 9-DOF parameters of the quadratic surface are refined, and the quadratic surface parameter q is iteratively updated using a nonlinear optimization method. The error equation is:

[0082]

[0083] Among them, e(B i ,q) represents the reprojection error of the object detection box and the ellipsoid projected onto the current frame's 2D bounding box, Σ o Let be the covariance matrix of the reprojection error term; and let A be the estimated axis length of the ellipsoid. Σ is the prior axis length; a is the covariance matrix of the axis length error term; q is the quadratic surface parameter; P i Let be the camera projection matrix for the i-th frame; define To obtain the circumscribed 2D bounding box of the ellipse; the prior object axis length is mainly used to limit the size of the ellipsoid, and the prior axis length is defined as...

[0084] Step 6: Optimize the parameters of the quadratic surface already constructed in the map using the object detection bounding boxes of the keyframes within a sliding window composed of keyframes; define the Gaussian description of the reprojection ellipse of the ellipsoid in the image frame as:

[0085] (x-μ) T Σ -1 (x-μ)=1 (9)

[0086] Where, μ=(c x ,c y ) represents the coordinates of the center of the ellipse; Σ represents the variance of the ellipse, i.e., the axis length.

[0087] A joint optimization function is formed by combining the reprojection error constructed from the object detection box within the sliding window and the ellipsoid projection, as well as the error constructed from the ellipsoid center point and the object detection box center. The quadratic surface parameter q is iteratively updated through nonlinear optimization. The joint optimization function is defined as follows:

[0088]

[0089] in, Let be the inscribed ellipse of the i-th detection box; π is the camera projection equation; u i Let t be the coordinates of the center point of the i-th object detection box; j Let ∑ be the center position of the j-th ellipsoid; o Let ∑ be the covariance matrix of the projection error term of the quadratic surface; t T is the covariance matrix of the center distance error term; i Let be the camera pose corresponding to the i-th frame; the sliding window length is n and i∈{1,2,...,n}. ε(.) is the Bhattacharyya distance between the two-dimensional projection of the ellipsoid and the observed object detection box in the current frame, defined as follows:

[0090]

[0091] Where, μ i Σ represents the center coordinates of the projected ellipse; i Represents the variance of an ellipse; `det` is the matrix determinant calculation operation;

[0092] Step 7: The SLAM backend performs joint optimization on the camera pose, quadratic surface parameters, and map points, utilizing the quadratic surface to optimize the camera pose. The map points are defined as P. k The characteristic observation is u k The quadratic surface parameter is q j The joint optimization function is defined as follows:

[0093]

[0094] Where, Σo and Σ m These are the covariance matrices of the detection box reprojection error and the map point reprojection error, respectively. This is an operation on the circumscribed rectangle of an ellipse.

[0095] Step 8: Output the camera pose and the object-level map containing quadratic surfaces.

[0096] In this embodiment Figure 2 The example demonstrates a comparison of the initialization effects of the proposed method and existing comparison algorithms. Figure 3 The final object-level map is shown.

[0097] To verify the accuracy of quadratic surface modeling, camera pose estimation, and robustness of the SLAM system, and to demonstrate its ability to handle dynamic object occlusion and observation error perturbations in outdoor scenes, this invention was tested on the KITTI tracking dataset, KITTI raw data dataset, and KITTI odometry dataset. On the KITTI tracking dataset, the system's object tracking and association performance was verified. In the eight test sequences, the average Moment of Response (MOTA) was 43.24%, and the average Motion Response Time (MOTP) was 74.78%, representing improvements of 12.22% and 101.87% in MOTA and 4.28% and 6.38% in MOTP, respectively, compared to the comparative algorithm. This verifies the stability and accuracy of the algorithm in object tracking and object data association. On the KITTI raw data dataset, the reconstruction success rate, 2DIOU error, center translation error, and axis length error of the reconstructed quadratic surface were verified, achieving averages of 85.47%, 0.7976, 0.8184m, and 0.5355m, respectively, on the test dataset. This demonstrates the algorithm's superior accuracy in quadratic surface reconstruction. Our algorithm can achieve accurate quadratic surface reconstruction in dynamic outdoor scenes. On the KITTI odometry dataset, our algorithm achieves mean relative translation error (RPE), mean relative rotation error (RPE), and mean absolute translation error (RPE) of 0.70%, 0.22° / 100m, and 2.27m, respectively, across 10 test sequences. This outperforms the comparison algorithms, proving the accuracy of our algorithm in localization. In real-time system testing, our average tracking time is 96.08ms, which meets the real-time localization requirements of a 10Hz camera.

[0098] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. An outdoor visual localization and mapping method based on robust quadratic surface initialization, characterized in that, Includes the following steps: Step 1: Use a SLAM system to receive RGB images captured by a stereo camera; Step 2: Perform binocular RGB image feature extraction; Step 3: Perform initial pose estimation for the camera; Step 4: Use an object data association algorithm to associate the detection results of the current frame with the objects in the map; Step 5: Initialize the isometric sphere of the object using the 2D object detection bounding boxes of the current frame and neighboring frames; Step 5 specifically involves: obtaining the object's center position using the triangulation of the detection box's center point; obtaining the isoaxial shape using the average projection of the detection box at the center position; and calculating the axis length as follows: (7)； in, This represents the depth value of the i-th object in the current frame's camera coordinate system. and Let be the width and height of the detection box for the i-th two-dimensional object; and is the camera intrinsic parameter; n is the number of consecutive frame observations of the associated object; After the sphere is initialized, it is updated using subsequent observations from a 2D object detection box. The 9-DOF parameters of the quadratic surface are refined, and the quadratic surface parameters are iteratively updated using a nonlinear optimization approach. The error equation is: (8)； in, This refers to the reprojection error of the object detection bounding box and the ellipsoid projected onto the current frame's 2D bounding box. Let be the covariance matrix of the reprojection error term; A is the estimated axis length of the ellipsoid. The a priori axis length; P is the covariance matrix of the axis length error term; q is the quadratic surface parameter, P i Let be the camera projection matrix for the i-th frame; define (.) represents the circumscribed rectangle operation of the ellipse; the prior axis length is defined as... ; Step 6: Optimize the parameters of the quadratic surface already constructed in the map using the object detection bounding boxes of the keyframes within a sliding window composed of keyframes; Step 7: The SLAM backend performs joint optimization of camera pose, quadratic surface parameters, and map points to optimize camera pose using the quadratic surface. Step 8: Output the camera pose and the object-level map containing quadratic surfaces.

2. The outdoor visual localization and mapping method based on robust quadratic surface initialization according to claim 1, characterized in that, Step 2 specifically involves: feeding the left-eye image into the deep network Yolact to obtain the object's target detection box. and semantic mask And perform optical flow extraction on the feature points within the detection box for the preceding and following frames.

3. The outdoor visual localization and mapping method based on robust quadratic surface initialization according to claim 1, characterized in that, Step 3 specifically involves: initializing the point cloud map by performing feature matching and feature point triangulation on the ORB feature points extracted from the binocular image, and constructing the reprojection error from 3D map points to 2D image features based on ORB features using the feature points of subsequent images, and iteratively solving the camera pose using the Levenberg-Marquardt algorithm.

4. The outdoor visual localization and mapping method based on robust quadratic surface initialization according to claim 1, characterized in that, Step 4 specifically includes the following steps: Step 4.1: Object motion tracking; Motion prediction using detection boxes based on Kalman filtering: The motion observation of an object satisfying uniform motion is defined as... ,in, , These are the two-dimensional center coordinates of the detection box; , These represent the height and width of the detection box, and the size of the detection box. The aspect ratio of the detection frame is It remains unchanged during the motion; , These represent the motion rates of the detection box along the axes of the two-dimensional image; This represents the rate of change of the bounding box size; for object bounding boxes with associated observations, a Kalman filter is used for state updates; if an object bounding box has no associated observations, the current state is not updated; the final updated object bounding box... for: (1)； in, , , This represents the posterior estimate of the observed value, i.e., the value after being updated by Kalman filtering; Step 4.2: Object motion detection; Use optical flow vector boundaries to determine the motion properties of objects; apply semantic masks. The features within are tracked using the Lucas-Cornard method optical flow, also known as LK optical flow, defining the feature coordinates for two consecutive frames as follows: and Object feature coordinates Use the following judgment: (2)； Among them, S i+1 This is the dynamic detection result of feature points. The binary values ​​0 and 1 represent the static and dynamic attributes of the feature points, respectively. This indicates the relative translation between two frames. The pixel depth value is obtained through a stereo matching algorithm, where K is the camera intrinsic parameter matrix; x min As the lower bound of the characteristic coordinates, x max The upper bound of the characteristic coordinates is defined as d, and the intermediate quantity for judging the optical flow vector boundary is defined as follows: , The upper and lower bounds of the feature point depth are calculated by formula (2). When the coordinates of the feature point fall outside the optical flow vector boundary, the feature point is determined to be a dynamic feature. The initial feature points of the object are set to static, and the motion state of the object is updated by the proportion of the dynamic points to which the object belongs. Step 4.3: and object association based on assignment; Using the Hungarian algorithm on the cost matrix Calculate the current frame A detection box and The optimal allocation of ellipsoids; the cost matrix is ​​defined by the following distance error: (3)； in, Indicates distance weight, Let be the element in the i-th row and j-th column of the cost matrix. Different distances are defined as follows: Define the feature set for LK optical flow tracing as semantic intrapoint distance. The feature set of the current frame is Define size(.) as the number of points in the calculation area. This is the semantic mask corresponding to the i-th object in the t-th frame of the image; In() represents the number of statistical feature points and the semantic intra-point distance. for; (4)； To detect IoU distance, IoU(.) is defined as the intersection-union ratio operation for calculating the two-dimensional detection boxes. This represents the i-th object detection box on the t-th image frame. The operation is performed on the bounding rectangle of the ellipse. Let be the parameters of the dual quadratic surface corresponding to the j-th ellipsoid in the map, and P be the camera projection matrix of the ellipsoid onto the image; the intersection-union ratio (IUU) is calculated using the reprojection of the j-th object and the detection box of the i-th object in the current frame: (5)； To predict the IoU distance; define The updated detection bounding box for the j-th object in the t-th frame of the image. For the i-th detection box in the t-th frame image, use the intersection-union ratio of the object detection box updated based on Kalman filtering in formula (1) and the i-th object detection box in the current frame: (6)。 5. The outdoor visual localization and mapping method based on robust quadratic surface initialization according to claim 1, characterized in that, Step 6 specifically involves defining the Gaussian description of the reprojection ellipse of the ellipsoid onto the image frame as follows: (9)； in, Indicates the coordinates of the ellipse's center; This represents the variance of the ellipse, i.e., the axis length. A joint optimization function is formed by combining the reprojection error constructed from the object detection box within the sliding window and the ellipsoid projection, as well as the error constructed from the ellipsoid center point and the object detection box center. The quadratic surface parameters are iteratively updated through nonlinear optimization. The joint optimization function is defined as follows: (10)； in, Let be the inscribed ellipse of the i-th detection box; The equation is the camera projection equation; Let be the coordinates of the center point of the i-th object detection box; The location of the center of the j-th ellipsoid; Let be the covariance matrix of the projection error term of the quadratic surface; T is the covariance matrix of the center distance error term; i Let n be the camera pose corresponding to the i-th frame image; the sliding window length is n and ; The Bhattacharyya distance between the 2D projection of the ellipsoid and the Bhattacharyya distance between the observed object detection box in the current frame is defined as follows: (11)； in, The coordinates of the center of the projected ellipse; This represents the variance of an ellipse; `det` is the matrix determinant calculation operation.

6. The outdoor visual localization and mapping method based on robust quadratic surface initialization according to claim 1, characterized in that, Step 7 specifically involves defining map points as follows: Feature observation is The parameters of the quadratic surface are The joint optimization function is defined as follows: （12）； in, and These are the covariance matrices of the detection box reprojection error and the map point reprojection error, respectively.

Images