Visual inertial odometer method of inertial measurement unit combined with random sampling algorithm
By combining visual and inertial information with a random sampling algorithm using an inertial measurement unit, the common in-field point set is optimized, which solves the positioning accuracy and robustness problems of the vision-inertial joint estimation algorithm in low-texture and dynamic scenes, and achieves higher feature point matching accuracy and system stability.
Patent Information
- Application Number
- CN202511017642.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-23
- Publication Date
- 2025-11-21
AI Technical Summary
Existing vision-inertial joint estimation algorithms have poor localization accuracy in low-texture environments, and traditional methods are not robust enough in complex dynamic scenes, especially in the case of large moving objects and degenerate motion, where they are prone to mismatch problems.
By employing an inertial measurement unit (IMU) combined with a random sampling algorithm, and by comprehensively utilizing visual and inertial information, abnormal feature points are filtered out, the common in-line point set is optimized, and the pose of the current frame is solved using the PNP algorithm. Combined with the pre-integration and time synchronization of the IMU sensor, the accuracy of feature point matching and tracking robustness are improved.
It significantly improves the positioning accuracy and stability of the vision-inertial joint estimation system in complex dynamic scenes, enhances the accuracy of feature point matching and the robustness of tracking, and performs particularly well in low-texture and dynamic environments.
Smart Images
Figure CN120991899A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of navigation technology, and in particular to a visual inertial odometry method using an inertial measurement unit combined with a random sampling algorithm. Background Technology
[0002] In recent years, with the rapid development of sensor technology and the improvement of computing power, multi-sensor SLAM, or vision-inertial joint estimation system, has shown broad application prospects in fields such as autonomous driving, robot navigation, and augmented reality. As an important branch of SLAM technology, multi-sensor SLAM has evolved into a high-precision localization and mapping solution integrating multiple advanced technologies. It fuses data from different sensors, including but not limited to LiDAR, vision sensors, inertial measurement units (IMUs), wheel encoders, and sonar. The vision sensors, such as monocular, stereo, and RGBD cameras, through complex data fusion algorithms and advanced computing platforms, achieve robust localization and map building in complex environments.
[0003] Compared to visual odometry, visual-inertial joint estimation systems can fuse visual and inertial information to achieve better robustness in complex environments, such as textureless scenes. However, traditional visual-inertial joint estimation systems still suffer from robustness issues when large moving objects and degenerate motions, such as prolonged stationary periods or pure rotation, are present. While semantic information can be used to improve tracking robustness in dynamic environments, the computational complexity conflicts with the real-time requirements of mobile devices.
[0004] Current popular vision-inertial joint estimation algorithms primarily rely on visual algorithms to calculate pose, while using only the gyroscopes and accelerometers of the inertial measurement unit (IMU) for pose-assisted calculation. This method of pose calculation performs well under high-texture conditions or with stereo and RGBD cameras, but its high cost limits its application scenarios. Although monocular cameras are relatively inexpensive, they require increasing ISO or reducing shutter speed to capture light in low-texture environments, introducing additional noise and disrupting feature extraction. Furthermore, monocular cameras lack depth information and rely on motion estimation from consecutive frames; poor lighting reduces the accuracy of depth estimation. Additionally, low pixel counts further limit image resolution, causing additional loss of detail. Therefore, the accuracy of monocular cameras in vision-inertial joint estimation algorithm applications is not ideal.
[0005] Therefore, low positioning accuracy has become a technical problem that urgently needs to be solved. Summary of the Invention
[0006] This invention provides a visual inertial odometry method using an inertial measurement unit combined with a random sampling algorithm, which solves the technical problem of low positioning accuracy.
[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:
[0008] A visual inertial odometry method using an inertial measurement unit combined with a random sampling algorithm includes the following steps:
[0009] Step S1: Obtain environmental images and attitude data of the inertial measurement unit, and obtain a grayscale image of the environmental images;
[0010] Step S2: Extract feature points from the obtained grayscale image and obtain pose changes based on pose data;
[0011] Step S3: Obtain interior points based on feature points of the grayscale image and generate a map. The interior points are the set of points on the 3D map.
[0012] Step S4: Obtain the predicted pose of the current frame based on the pose data and pose changes of the previous frame in the pose data.
[0013] Step S5: Extract the 3D map point set of the current frame from the 3D map point set. Obtain the 2D matching point corresponding to the 3D map point in the current frame. The predicted 3D point coordinates are calculated and projected onto the current grayscale image. Reprojection error Common view set of inertial measurement unit (IMU) data The predicted 3D point coordinates are projected onto the previous frame's grayscale image. Back projection error Co-visual set with vision Calculate the shared visual set Common view set with IMU data intersection The optimized set of common interior points is obtained by calculation. ;
[0014] Step S6: Optimized set of common interior points The pose of the current frame is obtained by solving the PNP algorithm. .
[0015] A further technical solution is that: in step S1, the step of obtaining a grayscale image of the environment image includes grayscale processing of the environment image to obtain a grayscale image.
[0016] A further technical solution is as follows: In step S2, the step of extracting feature points of the grayscale image includes performing FAST corner detection on the multi-scale pyramid of the grayscale image to extract feature points of the grayscale image; the step of obtaining pose change based on pose data includes synchronizing the pose data of the inertial measurement unit with the grayscale image in time, performing pre-integration of the inertial measurement unit, and calculating the pose change during the time interval between two adjacent grayscale images using the accelerometer and gyroscope data of the inertial measurement unit.
[0017] A further technical solution is as follows: In step S3, the step of obtaining interior points and generating a map based on the feature points of the grayscale image includes performing feature matching on adjacent grayscale images based on the feature points of the grayscale image to obtain matching points, triangulating the matching points to obtain interior points and generating a map, wherein the interior points in the map are the three-dimensional map point set.
[0018] A further technical solution is that: in step S4, the pose data of the previous frame in the attitude data of the inertial measurement unit obtained in step S1 is used. The predicted pose of the current frame is obtained by combining the pose change prediction obtained in step S2. .
[0019] A further technical solution is that, in step S5, the predicted three-dimensional point projection onto the current grayscale image coordinates is calculated according to equation (1). The reprojection error is calculated according to equation (2). The common view set of the inertial measurement unit (IMU) data is calculated according to equation (3). The predicted 3D point coordinates projected onto the previous grayscale image are calculated according to equation (4). The back projection error is calculated according to equation (5). The co-visual set of vision is calculated according to equation (6). The co-visual set of vision is calculated according to equation (7). Common view set with IMU data intersection The optimized set of common interior points is obtained by calculating according to equations (10) and (11). ;
[0020] (1)
[0021] In equation (1), To project the predicted 3D points onto the current grayscale image coordinates, K is the camera intrinsic parameter matrix. The set of 3D map points for the current frame. For the predicted position of the current frame, The predicted pose for the current frame;
[0022] (2)
[0023] In equation (2), This refers to the reprojection error between the actual and predicted observations. The two-dimensional matching point corresponding to the 3D map point in the current frame;
[0024] (3)
[0025] In equation (3), This is the common view set of inertial measurement unit (IMU) data. Forward threshold;
[0026] (4)
[0027] In equation (4), To project the predicted 3D points onto the coordinates of the previous grayscale image, This is the position of the previous frame. The pose of the previous frame;
[0028] (5)
[0029] In equation (5), This is the back projection error;
[0030] (6)
[0031] In equation (6), For the shared visual set;
[0032] (7)
[0033] In equation (7), For visual shared sets Common view set with IMU data The intersection;
[0034] Intersection of shared view sets This serves as the final shared view set for this iteration;
[0035] (8)
[0036] In equation (8), For the distribution of interior points, For visual shared sets Common view set with IMU data The intersection, The proportions of the n inliers in the bin are used as the confidence weights. For k frames of the 3D map points, The coordinates of the 2D center of the camera;
[0037] (9)
[0038] In equation (9), The confidence weights are the proportions of the n inliers in the bin. These are static coefficients. It is the previous confidence weight. The proportion of interior points in the bin is used as the confidence weight;
[0039] (10)
[0040] In equation (10), This represents the weights of the added and prior time factors. The dynamic coefficient is t, which represents the average observation time of all landmarks in bin. These are static coefficients. It is the previous confidence weight. The proportion of interior points in the bin is used as the confidence weight;
[0041] (11)
[0042] In equation (11), For the optimized set of common interior points, For visual shared sets Common view set with IMU data The intersection, The highest number of interior points The selected inlier set is the set of inlier points for each keypoint in an image frame. This represents the weights of the added and prior time factors. This represents the determinant of the variance matrix. Let be the distribution covariance matrix of the interior points.
[0043] The beneficial effects of adopting the above technical solution are as follows:
[0044] A visual inertial odometry method based on an inertial measurement unit (IMU) and a random sampling algorithm includes the following steps: Step S1: Obtain an environmental image and attitude data from the IMU, and obtain a grayscale image of the environmental image; Step S2: Extract feature points from the obtained grayscale image, and obtain pose changes based on the attitude data; Step S3: Obtain inliers based on the feature points of the grayscale image and generate a map, where the inliers are a set of 3D map points; Step S4: Obtain the predicted pose of the current frame based on the pose data and pose changes of the previous frame in the attitude data; Step S5: Extract the pose from the 3D map point set... The process involves extracting the set of 3D map points for the current frame, obtaining the corresponding 2D matching points, calculating the predicted 3D point projection onto the current grayscale image coordinates, reprojection error, common view set of IMU data, predicted 3D point projection onto the previous frame grayscale image coordinates, backprojection error, and the common view set of vision, calculating the intersection of the visual common view set and the common view set of IMU data, and calculating the optimized common view in-place point set. Step S6: The optimized common view in-place point set is used to solve for the pose of the current frame using the PNP algorithm. In contrast, this application proposes a fusion inertial measurement unit joint random sampling algorithm based on existing random sampling algorithms and combined with inertial measurement unit sensor data. This algorithm effectively filters out abnormal feature points by comprehensively utilizing visual and inertial information, thereby significantly improving the accuracy of feature point matching and the robustness of tracking, and ultimately improving the positioning accuracy and stability of vision-inertial joint estimation in complex dynamic scenes. Attached Figure Description
[0045] Figure 1 This is a flowchart of Embodiment 1 of the present invention;
[0046] Figure 2 This is a comparison chart of feature matching success rates on the V102 dataset. Detailed Implementation
[0047] To address the issue of abnormal feature points caused by mismatches during tracking, this application proposes a fusion inertial measurement unit (IMU) joint random sampling algorithm based on existing random sampling algorithms and combined with IMU sensors. This algorithm effectively solves the problem of abnormal feature point matching by comprehensively utilizing visual and inertial information, effectively filtering out abnormal feature points, thereby significantly improving the accuracy of feature point matching and the robustness of tracking, and ultimately enhancing the positioning accuracy and stability of the vision-inertial joint estimation system in complex dynamic scenes.
[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this application or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0049] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.
[0050] Example 1:
[0051] This invention discloses a visual inertial odometry method based on an inertial measurement unit and a joint random sampling algorithm, comprising the following steps:
[0052] like Figure 1 As shown, this invention discloses a visual inertial odometry method based on an inertial measurement unit and a joint random sampling algorithm, comprising the following steps:
[0053] Step S1: Obtain environmental images and attitude data of the inertial measurement unit, and perform grayscale processing on the environmental images to obtain grayscale images.
[0054] This step involves inputting images and inertial measurement unit data.
[0055] Input data name: environmental images and attitude data of the inertial measurement unit.
[0056] Processing algorithm name: Store the environmental image and perform grayscale processing; store the attitude data of the inertial measurement unit into the object.
[0057] Output data names: grayscale image and attitude data object of the stored inertial measurement unit.
[0058] Step S2: Perform FAST corner detection on the multi-scale pyramid of the grayscale image to extract feature points of the grayscale image and calculate the direction of the grayscale centroid around the feature points; The processor of the inertial measurement unit synchronizes the attitude data of the inertial measurement unit with the grayscale image in time, performs inertial measurement unit pre-integration, and calculates the pose change, velocity change and zero bias error during the time period between two adjacent grayscale images using the accelerometer and gyroscope data of the inertial measurement unit.
[0059] In this step, features are extracted from the image and inertial measurement unit data preprocessing is performed.
[0060] The processing algorithm involves performing FAST corner detection on a multi-scale pyramid on a grayscale image to extract key feature points and calculate the orientation of the grayscale centroids around these feature points. The attitude data object of the inertial measurement unit (IMU) is input into the IMU processor for time synchronization of IMU data and image data. The system performs IMU pre-integration, calculating the pose change, velocity change, and zero-bias error between two adjacent frames using IMU accelerometer and gyroscope data. Furthermore, IMU measurement data typically contains random noise and bias errors.
[0061] Output data names: feature points of grayscale images and velocity changes, pose changes, and zero bias errors calculated from inertial measurements.
[0062] Step S3: Based on the feature points of the grayscale images, perform feature matching on adjacent grayscale images to obtain matching points, triangulate the matching points to obtain interior points, and generate a map. The interior points in the map are the set of three-dimensional map points.
[0063] Input data name: Feature points of the current grayscale image and feature points of the previous grayscale image.
[0064] Processing algorithm name: Based on the feature points of the current grayscale image extracted in step S2, feature matching is performed with the feature points of the previous grayscale image to form a set of matching points between the two grayscale images. These matching points are triangulated to obtain interior points and generate a map. The interior points in the map are the set of three-dimensional map points.
[0065] Output data names: the set of matching points for the current grayscale image frame and the set of points for the 3D map.
[0066] Step S4: Based on the pose data of the previous frame in the attitude data of the inertial measurement unit obtained in step S1. The predicted pose of the current frame is obtained by combining the pose change prediction obtained in step S2. .
[0067] Input data name: Pose data of the previous frame The pose change calculated by the inertial measurement unit (IMU) in step S2.
[0068] Processing Algorithm Name: Pose data from the previous frame in the attitude data of the inertial measurement unit obtained in step S1 The pose of the current frame is predicted by the pose change calculated by the inertial measurement unit (IMU) in step S2 based on the pose changes in the two grayscale images. .
[0069] Output data name: Predicted pose of the current frame .
[0070] Step S5: Extract the set of 3D map points for the current frame from the set of 3D map points obtained in step S3. Obtain the 2D matching point corresponding to the 3D map point in the current frame. The predicted coordinates of the three-dimensional points projected onto the current grayscale image are calculated according to equation (1). The reprojection error is calculated according to equation (2). The common view set of the inertial measurement unit (IMU) data is calculated according to equation (3). The predicted 3D point coordinates projected onto the previous grayscale image are calculated according to equation (4). The back projection error is calculated according to equation (5). The co-visual set of vision is calculated according to equation (6). The co-visual set of vision is calculated according to equation (7). Common view set with IMU data intersection The optimized set of common interior points is obtained by calculating according to equations (10) and (11). .
[0071] Input data name: Predicted pose data for the current frame The pose data calculated in the previous frame The set of 3D map points in the current frame 2D matching points corresponding to the 3D map points in the current frame And the camera intrinsic parameter matrix K, which is provided by the camera manufacturer.
[0072] Processing algorithm name: First, the set of 3D map points in the current frame. Using the predicted pose data of the current frame Perform forward projection to obtain the predicted 3D point coordinates projected onto the current grayscale image. :
[0073] (1)
[0074] In equation (1), To project the predicted 3D points onto the current grayscale image coordinates, K is the camera intrinsic parameter matrix. The set of 3D map points for the current frame. For the predicted position of the current frame, The predicted pose for the current frame.
[0075] Calculate the reprojection error between the actual and predicted observations. ,
[0076] (2)
[0077] In equation (2), This refers to the reprojection error between the actual and predicted observations. It is the 2D matching point corresponding to the 3D map point in the current frame.
[0078] If the error is less than the forward threshold threshold If the prediction is true, then the point is considered to be stable under the prediction of the current frame.
[0079] (3)
[0080] In equation (3), This is the common view set of inertial measurement unit (IMU) data. This is the forward threshold.
[0081] Next, the back projection error of the previous frame is calculated simultaneously.
[0082] Use the pose calculated in the previous frame. Projecting 3D points To the image plane:
[0083] (4)
[0084] In equation (4), To project the predicted 3D points onto the coordinates of the previous grayscale image, This is the position of the previous frame. This refers to the pose of the previous frame.
[0085] Calculate the back projection error ,
[0086] (5)
[0087] In equation (5), This represents the back projection error.
[0088] If the error is less than the forward threshold threshold If the value is stable, then it is assumed that the point was also stable in the previous frame.
[0089] (6)
[0090] In equation (6), It is a shared set of vision.
[0091] Take the intersection of the point sets that are consistent in the forward and reverse directions to obtain the point set that is consistent in the reprojection of both frames.
[0092] (7)
[0093] In equation (7), For visual shared sets Common view set with IMU data The intersection of.
[0094] Intersection of shared view sets This serves as the final shared view set for this iteration.
[0095] because The subset remains unchanged during the iteration process of the IMU-RANSAC (Inertial Measurement Unit Joint Random Sampling Algorithm), therefore, it can be computed preferentially. .
[0096] At the same time, weighted covariance is used to assess the distribution of interior points:
[0097] (8)
[0098] In equation (8), For the distribution of interior points, For visual shared sets Common view set with IMU data The intersection, The proportions of the n inliers in the bin are used as the confidence weights. For k frames of the 3D map points, The coordinates of the 2D center of the camera are given.
[0099] (9)
[0100] In equation (9), The confidence weights are the proportions of the n inliers in the bin. These are static coefficients. It is the previous confidence weight. The proportion of interior points in the bin is used as the confidence weight.
[0101] In some highly dynamic scenes, identifying static landmarks based solely on information from a single frame may be insufficient. In such cases, historical information is needed. Because static feature points in a scene can be triangulated and stably tracked over a longer period, the continuous observation time of each landmark is collected, and the camera's motion state is measured using the average time of all observations in each bin. This is similar to an algorithm based on improved RANSAC, which uses the proportion of inliers in each bin as confidence weights. These weights determine the impact of each bin on the current model evaluation. To improve the accuracy of the weights, a prior time factor is added to the redefined weights.
[0102] (10)
[0103] In equation (10), This represents the weights of the added and prior time factors. The dynamic coefficient is t, which represents the average observation time of all landmarks in bin. These are static coefficients. It is the previous confidence weight. The proportion of interior points in the bin is used as the confidence weight.
[0104] The observation time is defined as the number of consecutive frames in which a landmark can be observed. To measure the dynamics of the scene, a dynamic coefficient p ∈ [0, 1] is introduced. This represents the static coefficient. This coefficient can be adjusted to suit specific scenarios. A larger t-value indicates that the landmark is more likely to be static and should be given greater weight when evaluating hypotheses compared to newly generated landmarks.
[0105] Therefore, the quality of a consensus set can be defined as:
[0106] (11)
[0107] In equation (11), For the optimized set of common interior points, For visual shared sets Common view set with IMU data The intersection, The highest number of interior points The selected inlier set is the set of inlier points for each keypoint in an image frame. This represents the weights of the added and prior time factors. This represents the determinant of the variance matrix. Let be the distribution covariance matrix of the interior points.
[0108] Output data name: Optimized set of common interior points .
[0109] Step S6: Based on the optimized set of common interior points obtained in step S5 The pose of the current frame is obtained by solving the PNP algorithm. .
[0110] Input data name: Set of common interior points .
[0111] Processing Algorithm Name: By inputting a set of 3D-to-2D point pairs into the PNP algorithm:
[0112] (12)
[0113] Output data: pose of the current frame .
[0114] Based on the above description, this application proposes an improved visual inertial odometry (VIO) system—SIR-SLAM—based on a monocular camera and an inertial measurement unit (IMU) sensor. By introducing the IMU-RANSAC algorithm in conjunction with the IMU, the system significantly improves its localization accuracy, real-time performance, and feature point matching success rate in low-texture environments, complex dynamic scenes, and under varying lighting conditions. Experimental results show that on the EuRoC dataset, SIR-SLAM achieves an average absolute trajectory error (ATE) of 0.031m, a 26.19% reduction compared to ORBSLAM3's 0.042m, significantly improving localization accuracy. The feature point matching success rate in dynamic scenes reaches a maximum of 97%, outperforming other VIO systems. Furthermore, SIR-SLAM's average processing time per frame is 55.693ms, showing varying degrees of reduction compared to other VIO systems. Particularly in scenes with drastic lighting changes or dynamic conditions, SIR-SLAM demonstrates excellent robustness and generalization ability. The research results in this paper not only verify the effectiveness of SIR-SLAM in complex environments through quantitative indicators, but also provide new technical ideas and practical references for the further development of real-time visual SLAM systems. At the same time, they lay the foundation for future extended research in deep learning, multimodal fusion and hardware acceleration.
[0115] Example 2:
[0116] This invention discloses a visual inertial odometry method based on an inertial measurement unit and a joint random sampling algorithm, comprising the following steps:
[0117] Acquire environmental images and attitude data of the inertial measurement unit (IMU), perform grayscale preprocessing on the images, and construct an image-IMU multimodal perception dataset;
[0118] Key feature points of grayscale images are extracted using the multi-scale FAST feature extraction algorithm, and pre-integration is performed using IMU accelerometer and gyroscope data to estimate the relative pose changes between frames.
[0119] An improved IMU-RANSAC algorithm is used to filter interior points of the 3D map based on forward and backward projection errors, and a high-confidence co-viewpoint set is constructed through the visual-inertial dual consistency principle.
[0120] A static scoring mechanism was constructed using weighted covariance analysis and continuous observation time of landmarks to evaluate and select the optimal consensus set.
[0121] The consensus point set is input into the PNP algorithm to solve for the optimal pose estimation of the current frame;
[0122] Finally, a stable and robust vision-inertial joint estimation framework is constructed, which is suitable for continuous attitude tracking and map building tasks in dynamic environments.
[0123] Based on Embodiment 2, a further technical solution is as follows:
[0124] The input observation space includes grayscale information of image frames, image feature point sets, IMU attitude angles, accelerometer data, gyroscope data, and timestamps between image frames;
[0125] The visual-inertial joint estimation method includes forward projection error and backward projection error in the reprojection error calculation process. The forward error is used to detect the geometric consistency between the current frame and the predicted frame, and the backward error is used to constrain the effectiveness of historical frames in the current view.
[0126] In the process of selecting inliers, the IMU-RANSAC algorithm integrates the visual common set and the residual consistency set formed by the prediction path of IMU data to construct a cross-validation mechanism between vision and IMU, which improves the robustness of outlier removal.
[0127] The common viewpoint set evaluation mechanism introduces the landmark observation time and spatial distribution covariance to construct a weighting function, and adjusts the confidence weight by combining static and dynamic coefficients to achieve the judgment of landmark stability in highly dynamic scenes;
[0128] The PNP solving module takes the visual-inertial consensus point as input and combines it with the camera intrinsic parameter matrix to realize 3D-2D point-to-point pose backpropagation and complete the spatial positioning of the camera in the current frame.
[0129] The beneficial effects are detailed below.
[0130] High accuracy: The IMU-RANSAC algorithm is based on a bidirectional projection error joint screening mechanism, which effectively removes dynamic object interference and erroneous feature points in image feature matching, ensuring the purity of the input point set and thus improving the accuracy of pose estimation.
[0131] Strong robustness: Traditional methods are prone to failure in highly dynamic or low-texture environments, while this application significantly enhances the robustness of the system in extreme environments by combining IMU pre-integration with a static evaluation weighting mechanism.
[0132] Efficiency Improvement: Consensus point selection adopts weighted covariance analysis and landmark observation time prior, which makes interior point evaluation more efficient and accurate, reduces optimization overhead, and improves the overall estimation speed.
[0133] High adaptability: The vision-inertial joint estimation mechanism proposed in this application has good adaptability to image sequences with different camera models and different frame rates, and can be flexibly deployed in mobile robots, drones and autonomous driving platforms.
[0134] Example 3:
[0135] This invention discloses a visual inertial odometry method based on an inertial measurement unit and a joint random sampling algorithm, comprising the following steps:
[0136] All experiments were conducted on a Lenovo Y9000P laptop with an Intel(R) Core(TM) i7-12700H processor (2.30 GHz) and 16GB of RAM. The experimental platform was Ubuntu 20.04, compiled using C++ 7.5.
[0137] The experiments utilized the publicly available EuRoC dataset. The EuRoC MAV Dataset is a comprehensive dataset specifically designed for testing and evaluating various vision and visual-inertial systems, collected by the Robotics and Perception Research Group at ETH Zurich. This dataset uses miniature aerial vehicles (MAVs) equipped with multiple sensors to collect data in three different indoor environments: a hangar, an industrial environment, and an office. These environments progressively increase in difficulty, designed to test the algorithm's performance under varying levels of complexity. Regarding sensor configuration, the stereo images in the EuRoC dataset were acquired by two global shutter cameras at a resolution of 752x480 pixels, providing rich visual information for the algorithm. Furthermore, the dataset includes high-frequency 200Hz triaxial acceleration and triaxial angular velocity data acquired from an inertial measurement unit (IMU), crucial for performing vision-inertial fusion. The EuRoC dataset contains a total of 11 sequences, categorized into easy, medium, and hard levels based on the complexity of the visual conditions, covering various challenging scenarios including motion blur and changes in lighting, providing rich visual and motion data for evaluating the algorithm's performance under varying conditions.
[0138] A comparison of trajectory positioning accuracy is detailed below.
[0139] For example, to comprehensively evaluate the performance of SIR-SLAM on the EuRoC dataset, a detailed comparison was performed with ORBSLAM3. Furthermore, comparisons were made with open-source VIO frameworks such as VINS-Mono, ORBSLAM-VI, and VI-DSO. For accuracy evaluation, the EVO tool was used to evaluate the trajectory output by the SLAM algorithm against the actual trajectory, calculating its absolute trajectory error ATE and quantizing the accuracy using the root mean square error RMSE.
[0140] (12)
[0141] In equation (12), This represents the coordinates of the position estimated. This represents the coordinates of the actual trajectory. This is the number of coordinate points that need to be compared.
[0142] The primary evaluation criterion is the absolute trajectory error (ATE) of the estimated pose at all time points. The root mean square error (RMSE) is used to evaluate the parameters of ATE.
[0143] As shown in Table 1, to ensure the reliability of the experimental results, each SLAM system was repeated 10 times, and the mean of the absolute trajectory error RMSE ATE was calculated based on the complete trajectory.
[0144] The EuRoC dataset is divided into two scenes: Machine Hall and Vicon Room. Machine Hall (MH) and Vicon Room (V1 and V2) represent different environmental characteristics and motion patterns, respectively. The MH series (MH01-MH05) was captured in a machine shop, featuring complex flight trajectories, wide-range movements, and significant lighting changes. The Vicon Room sequences were recorded in a laboratory setting. These sequences, including V101, V102, V201, and V202, were recorded using a Vicon motion capture system, exhibiting smooth flight trajectories and uniform lighting conditions.
[0145] Table 1: Comparison of trajectory accuracy on the Euroc dataset, unit: m
[0146] Dataset MH01 MH02 MH03 MH04 MH05 V101 V102 V103 V201 V202 V203 Average ORB-SLAMVI 0.075 0.084 0.087 0.217 0.082 0.027 0.028 - 0.032 0.041 0.074 0.075 VINS-Mono 0.084 0.105 0.074 0.122 0.147 0.047 0.066 0.180 0.056 0.090 0.244 0.110 VI-DSO 0.062 0.044 0.117 0.132 0.121 0.059 0.067 0.096 0.040 0.062 0.174 0.089 ORBSLAM3 0.032 0.053 0.033 0.099 0.071 0.043 0.016 0.025 0.041 0.015 0.037 0.042 SIR-SLAM 0.019 0.036 0.035 0.053 0.057 0.031 0.011 0.022 0.052 0.011 0.015 0.031
[0147] As shown in Table 1, SIR-SLAM consistently demonstrates lower absolute trajectory error (ATE) compared to other comparative methods across multiple datasets, particularly in extremely dark environments (MH04 dataset) and fast-moving scenes (V102 dataset). For instance, in the MH04 dataset, ORBSLAM3's absolute trajectory error is 0.099m, while the other three VIO algorithms all have absolute errors greater than 0.1m. In contrast, SIR-SLAM's error is 0.053m, a 46.46% reduction compared to the more accurate ORBSLAM3 system. This performance improvement is primarily attributed to the synergistic effect of SIR-SLAM's feature enhancement strategy under low-light conditions and its nonlinear optimization algorithm. In the V102 dataset, SIR-SLAM's absolute trajectory error is 0.011m, representing a 31.25%–83.58% reduction compared to the other four comparative algorithms, validating the applicability of the improved method to fast-moving scenes.
[0148] In scenarios with complex motion trajectories, such as the V202 dataset, SIR-SLAM also performs exceptionally well, with a trajectory error of 0.011m. Compared to the ORBSLAM3 system, which performs relatively well in the comparison algorithms, the trajectory error of the proposed method, SIR-SLAM, is reduced by 0.015m in the bottom row of Table 1.
[0149] The average trajectory error of SIR-SLAM across various datasets is 0.031m. In fast-moving indoor scenes, the improved algorithm demonstrates even more significant improvements in trajectory accuracy. Compared to the ORBSLAM-VI algorithm, the average trajectory accuracy is improved by 58.67%; compared to VINS-Mono, by 71.82%; compared to VI-DSO, by 65.17%; and compared to ORBSLAM3, by 41.26%. These results fully demonstrate the improved trajectory accuracy of SIR-SLAM and its adaptability to complex scenes.
[0150] The real-time analysis is detailed below.
[0151] For example, this application performs real-time performance analysis of the code by comparing the average processing time of image frames. On the EuRoC dataset, the ORBSLAM3, ORBSLAM-VI algorithm frameworks are compared with SIR-SLAM. A quantitative metric—the average processing time per frame—is obtained by dividing the total runtime by the number of image frames to be processed in the corresponding dataset. Each dataset is tested five times, and the average value is taken.
[0152] As shown in Table 2, the data in the rightmost column represents the best results. The average processing time of the SIR-SLAM method in this application is shorter than that of other comparative algorithms on different datasets, demonstrating high real-time performance.
[0153] Compared to ORB-SLAMVI, the method in this application can reduce the processing time per frame by an average of 1-3 ms. Although ORB-SLAM3's real-time performance on half of the experimental datasets is close to that of SIR-SLAM, in scenes with changing lighting and fast motion, such as MH05 and V101, the method in this application can maintain its real-time performance more stably, keeping the average processing time within 55-56 ms.
[0154] Overall, the SIR-SLAM system's ability to process changing scenes quickly is due to the efficient design of its nonlinear optimization and visual feature processing algorithms. This ensures both accuracy and robustness while improving the system's computational efficiency, demonstrating the potential of the improved algorithm in real-time applications.
[0155] Table 2: Real-time performance comparison on the Euroc dataset, unit: ms
[0156] Data Frame ORB-SLAMVI ORBSLAM3 SIR-SLAM MH01 3682 57.939 55.807 55.693 MH02 3040 57.959 57.142 54.078 MH03 2700 58.077 56.524 56.484 MH04 2033 58.018 56.854 56.370 MH05 2273 57.811 58.151 56.052 V101 2912 57.825 62.981 55.799 V102 1710 58.709 56.584 55.588 V103 2149 57.449 55.886 55.715 V201 2280 58.274 57.33 56.083 V202 2348 57.964 56.911 55.897 V203 1992 66.285 64.656 64.653
[0157] The average success rate of feature point matching is detailed below.
[0158] like Figure 2 As shown, exemplarily, to verify the performance advantages of the algorithm in the tracking stage. OURS in the figure refers to SIR-SLAM, and ORB0SLAMVI refers to ORB-SLAMVI. This application compares and analyzes the success rates of SIR-SLAM with ORB-SLAMVI, ORBSLAM3, VI-DSO, and VINS-Mono systems in successfully matching feature points. In the MH01 dataset, the algorithm using inertial measurement unit (IMU) combined with random sampling optimizes interior points using IMU and vision, filtering out abnormally extracted feature points during extraction. This reduces mismatches in the next frame, achieving a feature point matching success rate of 98%. Meanwhile, ORBSLAM3 and VINS-Mono also maintain success rates above 90%, showing relatively ideal performance. However, in scenarios with changing lighting, the matching success rates of all compared algorithms drop below 90%. In this case, the success rate of this application is 91%, still showing an advantage over other algorithms. In fast-moving scenarios, VINS-Mono's matching success rate is relatively outstanding, at 90% and 95% respectively. At this point, SIR-SLAM still performs best among all algorithms, with success rates of 92% and 97%, respectively.
Claims
1. A visual inertial odometry method using an inertial measurement unit combined with a random sampling algorithm, characterized in that: Includes the following steps, Step S1: Obtain environmental images and attitude data of the inertial measurement unit, and obtain a grayscale image of the environmental images; Step S2: Extract feature points from the obtained grayscale image and obtain pose changes based on pose data; Step S3: Obtain interior points based on feature points of the grayscale image and generate a map. The interior points are the set of points on the 3D map. Step S4: Obtain the predicted pose of the current frame based on the pose data and pose changes of the previous frame in the pose data. Step S5: Extract the 3D map point set of the current frame from the 3D map point set. Obtain the 2D matching point corresponding to the 3D map point in the current frame. The predicted 3D point coordinates are calculated and projected onto the current grayscale image. Reprojection error Common view set of inertial measurement unit (IMU) data The predicted 3D point coordinates are projected onto the previous frame's grayscale image. Back projection error Co-visual set with vision Calculate the shared visual set Common view set with IMU data intersection The optimized set of common interior points is obtained by calculation. ; Step S6: Optimized set of common interior points The pose of the current frame is obtained by solving the PNP algorithm. .
2. The visual inertial odometry method based on the inertial measurement unit and random sampling algorithm according to claim 1, characterized in that: In step S1, the step of obtaining a grayscale image of the environment image includes grayscale processing of the environment image to obtain a grayscale image.
3. The visual inertial odometry method based on the inertial measurement unit and random sampling algorithm according to claim 1, characterized in that: In step S2, the step of extracting feature points of the grayscale image includes performing FAST corner detection on the multi-scale pyramid of the grayscale image to extract feature points of the grayscale image; the step of obtaining pose change based on pose data includes synchronizing the pose data of the inertial measurement unit with the grayscale image in time, performing inertial measurement unit pre-integration, and calculating the pose change during the time period between two adjacent grayscale images using the accelerometer and gyroscope data of the inertial measurement unit.
4. The visual inertial odometry method based on the inertial measurement unit and random sampling algorithm according to claim 1, characterized in that: In step S3, the step of obtaining interior points and generating a map based on the feature points of grayscale images includes performing feature matching on adjacent grayscale images based on the feature points of grayscale images to obtain matching points, triangulating the matching points to obtain interior points and generating a map, wherein the interior points in the map are the set of three-dimensional map points.
5. The visual inertial odometry method based on the inertial measurement unit and random sampling algorithm according to claim 1, characterized in that: In step S4, the pose data of the previous frame in the attitude data of the inertial measurement unit obtained in step S1 is used. The predicted pose of the current frame is obtained by combining the pose change prediction obtained in step S2. .
6. The visual inertial odometry method based on the inertial measurement unit and random sampling algorithm according to claim 1, characterized in that: In step S5, the predicted three-dimensional point projection onto the current grayscale image coordinates is calculated according to equation (1). The reprojection error is calculated according to equation (2). The common view set of the inertial measurement unit (IMU) data is calculated according to equation (3). The predicted 3D point coordinates projected onto the previous grayscale image are calculated according to equation (4). The back projection error is calculated according to equation (5). The co-visual set of vision is calculated according to equation (6). The co-visual set of vision is calculated according to equation (7). Common view set with IMU data intersection The optimized set of common interior points is obtained by calculating according to equations (10) and (11). ; （1） In equation (1), To project the predicted 3D points onto the current grayscale image coordinates, K is the camera intrinsic parameter matrix. The set of 3D map points for the current frame. For the predicted position of the current frame, The predicted pose for the current frame; （2） In equation (2), This refers to the reprojection error between the actual and predicted observations. The two-dimensional matching point corresponding to the 3D map point in the current frame; （3） In equation (3), This is the common view set of inertial measurement unit (IMU) data. Forward threshold; （4） In equation (4), To project the predicted 3D points onto the coordinates of the previous grayscale image, This is the position of the previous frame. The pose of the previous frame; （5） In equation (5), This is the back projection error; （6） In equation (6), For the shared visual set; （7） In equation (7), For visual shared sets Common view set with IMU data The intersection; Intersection of shared view sets This serves as the final shared view set for this iteration; （8） In equation (8), For the distribution of interior points, For visual shared sets Common view set with IMU data The intersection, The proportions of the n inliers in the bin are used as the confidence weights. For k frames of the 3D map points, The coordinates of the 2D center of the camera; （9） In equation (9), The confidence weights are the proportions of the n inliers in the bin. These are static coefficients. It is the previous confidence weight. The proportion of interior points in the bin is used as the confidence weight; （10） In equation (10), This represents the weights of the added and prior time factors. The dynamic coefficient is t, which represents the average observation time of all landmarks in bin. These are static coefficients. It is the previous confidence weight. The proportion of interior points in the bin is used as the confidence weight; （11） In equation (11), For the optimized set of common interior points, For visual shared sets Common view set with IMU data The intersection, The highest number of interior points The selected inlier set is the set of inlier points for each keypoint in an image frame. This represents the weights of the added and prior time factors. This represents the determinant of the variance matrix. Let be the distribution covariance matrix of the interior points.