Dynamic environment adaptive visual inertial odometer method with zero-speed updating capability

Abstract

The invention belongs to the technical field of visual inertial odometers in the fields of robot navigation, automatic driving, virtual reality and augmented reality, and discloses a dynamic environment adaptive visual inertial odometer method with zero-speed updating capability. Performing multi-resolution static state detection analysis on the inertial data; in a motion state, executing pre-integration calculation based on the inertial data; a pre-integration result of inertial data is used as pose prior information between adjacent frames, epipolar constraints are constructed, and a dynamic feature set and an original static feature set in a dynamic environment are obtained; based on the dynamic feature set, clustering calculation is carried out to obtain a plurality of clustering centers as cue words, and instance segmentation of a plurality of dynamic objects in the scene is guided to be completed; after a dynamic object instance segmentation mask is generated, static information is rechecked, and a selected static feature set is obtained based on the mask; in a static state, executing a zero-speed updating strategy; and in the motion state, executing an iterative extended Kalman filtering updating strategy.

Description

Technical Field

[0001] The present invention relates to the field of visual inertial odometry technology in the fields of robot navigation, autonomous driving, virtual reality, and augmented reality, and in particular to a dynamic environment adaptive visual inertial odometry method with zero-speed update capability. Background Art

[0002] Traditional visual-inertial odometry (VIO) uses a state estimation algorithm to estimate the current device pose by fusing visual and inertial sensors based on a static environment assumption. However, the random and unpredictable dynamic disturbances of a dynamic environment do not conform to the static environment assumption, rendering traditional VIO inoperable. Furthermore, the complex and diverse motion patterns of sensor-equipped devices significantly impact the reliability of inertial data. Existing VIO algorithms still have multiple limitations in addressing these challenges.

[0003] 1. Inadequate static state detection methods and weak zero-speed system update capabilities

[0004] Zero-speed static state is a common operating mode for mobile devices, but previous research has not paid enough attention to it. As the cost-effectiveness of monocular VIO increases, the drift characteristics of inertial sensors in the static state have an increasingly significant impact on the system positioning accuracy. When the document "IEEE Sensors Journal, 5113-5121, 2021" relied entirely on inertial data to perform static state detection, related research in the field of VIO was still in its infancy. Among them, zero-speed detection methods and effective zero-speed update methods are currently severely lacking.

[0005] 2. Insufficient dynamic target detection capabilities

[0006] Dynamic target detection is a key part of VIO in dynamic environments. Methods that rely on geometric constraints (Pattern Recognition Letters, 191-201, 2019) often assume that the number of static elements in the image exceeds the dynamic elements, which limits the adaptability of the algorithm to scenes with rich dynamic features. At the same time, detection methods based on deep learning (IEEE Transactions on Instrumentation and Measurement, 2025) face obstacles in practical application deployment. For example, preset category labels will lead to limited dynamic object detection and often face the risk of feature misclassification. When trying to integrate geometric constraints with deep learning methods, the document "International Conference on Indoor Positioning and Indoor Navigation (IPIN). IEEE, 2021" did not fully explore the complementary capabilities of the two technologies.

[0007] 3. Insufficient dynamic environment processing strategies

[0008] Researchers divide the application of dynamic features in the state estimation process into two categories. The first category (literature "IEEE robotics and automation letters, 4076-4083, 2018") eliminates dynamic features and highlights static environmental elements to infer pose. However, this method may cause the algorithm to fail due to insufficient static constraints. The second category (literature "IEEE Transactions on Instrumentation and Measurement, 2025") chooses to use dynamic probability to avoid explicit identification and removal of environmental elements, and solves the dynamic detection problem and pose estimation problem by jointly modeling them as a maximum likelihood model. Although these methods perform well in specific environments, they are affected by errors in dynamic feature detection.

[0009] 4. State estimation strategies for dynamic environments have flaws

[0010] In extensive research, VIO state estimation strategies are divided into two categories, filtering-based and optimization-based, based on the decomposition method of the least squares model. Although their calculations differ significantly, both methods follow the forward projection-based residual generation principle established by the first estimated Jacobian (the document "The Eleventh International Symposium. Springer, 373-382, 2009") (FEJ). This means that the visual residual is always constructed around all projection relationships that meet the minimum tracking length from each frame to the initial observation frame. However, the misclassification of static feature vectors caused by dynamic interference in dynamic scenes will lead to errors in the construction of traditional visual residuals, which in turn leads to the failure of the algorithm calculation. Summary of the Invention

[0011] The purpose of this invention is to propose a dynamic environment-adaptive visual-inertial odometry method with zero-speed update capability to address the operational issues of existing VIO algorithms in dynamic scenarios with diverse motion patterns. The proposed method is a visual-inertial odometry method that can achieve stable and accurate positioning in dynamic environments with complex and changing motion patterns.

[0012] The technical solution of the present invention is as follows: a dynamic environment adaptive visual inertial odometry method with zero-speed update capability, which performs grid uniform feature extraction and optical flow tracking on the original image sequence; performs multi-resolution static state detection analysis on the inertial data;

[0013] When the static state detection analysis concludes that the state is moving, pre-integration calculation and inertial state propagation calculation are performed based on inertial data; the pre-integration results of inertial data between adjacent frames are cached using adjacent image timestamps as intervals;

[0014] When the extracted image features are used for dynamic feature detection, the pre-integration results of the inertial data are used as the prior information of the pose between adjacent frames to construct epipolar constraints. The dynamic feature set and the original static feature set in the dynamic environment are obtained based on the epipolar constraints.

[0015] Based on the dynamic feature set, the multiple cluster centers obtained by clustering calculation are used as prompt words to guide the unsupervised instance segmentation algorithm to complete the instance segmentation of multiple dynamic objects in the scene. After generating the dynamic object instance segmentation mask, the static information is rechecked and the static feature set is finally obtained based on the mask for the iterative extended Kalman filter update process.

[0016] When stationary: execute the zero-speed update strategy; when in motion, according to the pixel ratio of the segmentation mask of the dynamic object instance in the current frame, the effective tracking length of the static feature used is constructed by the pixel ratio of the mask and the visual residual, and the iterative extended Kalman filter update strategy is executed in a dynamic adjustment manner.

[0017] The multi-resolution stationary state detection analysis is specifically as follows:

[0018] The discrete inertial data is decomposed into the following after wavelet transform:

[0019]

[0020] Where f(t) represents a discrete inertial data sequence, k represents the data timing offset, and j represents the multi-resolution analysis level. represents the Haar wavelet transform parent function, ψ(x) represents the Haar wavelet transform mother function, d k and c jk Indicates the detailed values ​​obtained by parsing inertial data at different resolutions;

[0021] The parent and mother functions of the application are as follows:

[0022]

[0023] Given a set of inertial data from a three-axis accelerometer and a three-axis gyroscope with aligned timestamps from an inertial measurement unit (IMU), where any axis of the three-axis accelerometer or the three-axis gyroscope contains a sequence of N inertial data IMU fixed sampling frequency f IMU ; Camera fixed sampling frequency f Cam ; Data analysis level L = 4; Wavelet transform basic sampling interval T corresponding to the original sequence of inertial datadi ; Data analysis level l corresponding analysis results [Sum l ,Del l ]; The first layer static state judgment conclusion DVG l ; scalar coefficient S; adjacent inertial data static state basic judgment threshold thres; different analysis levels correspond to static judgment flags F1, F2, F3, F4; multi-resolution static detection process is as follows:

[0024] (1) Initial sequence

[0025] (2) When the data analysis level L = 4, the wavelet transform is performed with a sampling interval of 2T di Analysis of Sum5:

[0026] Sum4(i / 2)←(Sum5(i)+Sum5(iT di )) / twenty three)

[0027] Del4(i / 2)←|Sun5(i)-Sum5(iT di )| / 2 (4)

[0028] According to [Sum4, Del4], perform static judgment of the current level and operate the flag bit:

[0029] S←T di ;DVG4←max(Del4)-min(Del4);F4←DVG4≤S×thres (5)

[0030] l=L-1 (6)

[0031] (3) When the data analysis level L=3, the wavelet transform sampling interval starts from the second data in the positive order of Sum4 and is traversed in a loop with an interval of 1 until the traversal is completed. The traversal analysis process is:

[0032] Sum3(i-1)←(Sum4(i)+Sum4(i-1)) / 2 (7)

[0033] Del3(i-1)←|Sum4(i)-Sum4(i-1)| / 2 (8)

[0034] According to [Sum3, Del3], perform static judgment of the current level and operate the flag bit:

[0035] S←2T di ;DVG3←max(Del3)-min(Del3);F3←DVG3≤S×thres (9)

[0036] l=L-1 (10)

[0037] (4) When the data analysis level L = 2, the wavelet transform sampling interval starts from the first data in the positive order of Sum3 and is traversed in a loop with an interval of 2 until the traversal is completed. The traversal analysis process is:

[0038] Sum2(i)←(Sum3(i)+Sum3(i+2)) / 2 (11)

[0039] Del2(i)←|Sum3(i)-Sum3(i+2)| / 2 (12)

[0040] According to [Sum2, Del2], perform static judgment of the current level and operate the flag bit:

[0041] S←6T di ;DVG2←max(Del2)-min(Del2);F2←DVG2≤S×thres (13)

[0042] l=L-1 (14)

[0043] (5) When the data analysis level L = 1, the wavelet transform converts the two data of Sum2:

[0044] Sum1(1)←(Sum2(1)+Sum2(2)) / 2 (15)

[0045] Del1(1)←|Sum2(1)-Sum2(2)| / 2 (16)

[0046] According to [Sum1, Del1], perform static judgment of the current level and operate the flag bit:

[0047] S←12T di ;DVG1←max(Del1)-min(Del1);F1←DVG1≤S×thres (17)

[0048] / =L-1 (18)

[0049] (6) Integrate the static analysis conclusions of each resolution level to obtain the final judgment result:

[0050] Flag=F1∩F2∩F3∩F4 (19)

[0051] According to this process, the IMU three-axis accelerometer and three-axis gyroscope are analyzed separately. As long as the static flag of any axis is not true, the current motion state is not static.

[0052] The pre-integration calculation process is as follows:

[0053] Visual inertial odometry state variable definition:

[0054]

[0055] Represents the state variable expression in the global coordinate system; specify R k Represents the origin of the IMU coordinate system at time k; the origin of the global coordinate system is represented by G, which is equivalent to R0; is a 4×1 unit quaternion that conforms to JPL convention, representing the number from {G} to {R k}'s rotation; Is {G} in {R k} in the position; {R k} represents gravity; τ is the timestamp between k and k+1, Description from {R k} to the current IMU coordinate origin {I τ} movement; and is the relative rotation and translation; It is {I τ} represents the local velocity; and They represent the latest deviations of the three-axis gyroscope and the three-axis accelerometer at time τ respectively; It consists of real-time calibration parameters; and C p I represents the rotation and translation between the camera frame and the IMU frame, and t d represents the difference between the reception timestamps of the camera and IMU measurements at time step k; Provide the relative IMU pose in the N most recent timestamps before timestamp k;

[0056] Assume ω m and a m Indicates the three-axis gyroscope and three-axis accelerometer measurements provided by the IMU; at the same time, the symbol denotes the estimate associated with the variables in formula (20); and On this basis, the robot-centered error state model in the continuous time domain is expressed as follows:

[0057]

[0058] in, is the IMU input noise; in formula (20) and It does not participate in the state propagation process controlled by motion data, so the state transfer matrix F and the noise Jacobian matrix G are only related to Building relationships;

[0059] In the time interval [t τ ,t τ+1 ) performs time domain discretization and integral calculation on formula (21), and obtains δt=t τ+1 -t τ The discrete time error state transfer matrix Φ(t τ+1 ,t τ ):

[0060]

[0061] The entire propagation process evaluates the motion of the IMU in real time and introduces covariance expansion to guide the pre-integration process; the relationship between the diagonal field of view of the visual component and the motion measurement of the inertial component is established within the defined sampling interval, and the covariance transfer mechanism of error propagation is reconstructed; covariance propagation starts from time step k:

[0062]

[0063] in, represents the continuous-time input noise covariance matrix, Γ is the expansion coefficient; given the diagonal field of view Ψ of the visual sensor and the camera frequency f C , when the inter-frame uniform motion assumption is adopted, the threshold ε for determining the motion state from the three-axis gyroscope is derived:

[0064] ε=Ψ*f C (twenty four)

[0065] The average accelerometer measurement value between frames is specified as a v , the expansion coefficient Γ is as follows:

[0066]

[0067] Due to the enhanced state It has nothing to do with the propagation process. and Packaged as This gives the covariance of the transfer at time τ+1:

[0068]

[0069] in The error state transfer matrix is ​​calculated as:

[0070]

[0071] Formula (22) to Formula (27) are executed in sequence according to the time step to obtain the pre-integration result.

[0072] The process of acquiring the dynamic feature set in the dynamic environment is as follows:

[0073] During the feature extraction and optical flow tracking phase, Shi-Tomasi corner points in the image are identified and the KLT algorithm is used to correlate data between frames. A grid-based uniform feature extraction is performed across the entire image area, and the distance between feature point pixels is calculated. During feature point screening, the nearest neighbor radius is set to merge and screen adjacent feature points.

[0074] While obtaining the visual feature points obtained in the feature extraction process, a pre-integration operation is performed on the inter-frame inertial data in the motion state. The pre-integration results are used to establish the pose relationship between the inertial coordinate systems associated with each frame image; through the camera intrinsic parameter matrix K and the spatial relationship between the camera and IMU With the pre-integration result The pose transformation between the corresponding camera coordinate systems is It can be further decomposed into rotation and pan After that, the initial basic matrix between consecutive frames is obtained

[0075]

[0076] Where (·×) represents the antisymmetric matrix of a vector;

[0077] Integrating the initial fundamental matrix with the data association established during feature extraction, the distance from each visual feature point to its corresponding epipolar line is expressed as:

[0078]

[0079] Among them, p k,i and p k+1,i are the normalized coordinates of the i-th visual feature point in the k-frame coordinate system and the k+1-frame coordinate system respectively;

[0080] Following the nearest neighbor distinction radius used for visual feature point screening during feature extraction, a neighborhood screening range of the vertical distance from the visual feature point to the epipolar line is established. The complementary logarithmic model is used to determine the static feature probability distribution interval around the epipolar line, and the regression evaluation standard of the judgment process is obtained:

[0081]

[0082] After regression judgment of the complementary logarithmic model, the feature points obtained in the feature extraction process are divided into two parts: dynamic feature set and original static feature set.

[0083] The instance segmentation of the dynamic object is as follows:

[0084] Extract multiple cluster centers obtained by clustering calculation; input multiple cluster centers as prompt words into the prompt-guided unsupervised instance segmentation algorithm model, map the two-dimensional positions of the cluster centers to the image region through the prompt encoder, and generate a dedicated expression format for processing; relying on the dedicated expression, the prompt-based unsupervised instance segmentation model uses an internal encoding memory mechanism to retain and modify the feature expression of each dynamic object; on this basis, the built-in self-attention module of the prompt-based instance segmentation model uses the modified feature expression to further review the spatial relationship between different dynamic areas in the image, and separate the screened spatial relationship from the background information; the mask decoder captures and enhances the spatial correlation and generates the instance segmentation mask result corresponding to each cluster center.

[0085] The review of the static information is specifically as follows:

[0086] The original static feature set extracted is re-evaluated based on the instance segmentation mask;

[0087] Specifically, the visual feature points outside the instance segmentation mask are designated as static feature points for the subsequent iterative extended Kalman filter update process. When the number does not meet the minimum number requirement for the subsequent iterative extended Kalman filter update process, supplementary features are extracted outside the mask pixel range and the corresponding new data associations are re-established, finally obtaining a selected static feature set within the frame.

[0088] Dynamic environment adaptive iterative extended Kalman filter update is specifically:

[0089] In motion, the dynamic object instance segmentation mask corresponding to each frame image and the selected static feature set corresponding to the frame are obtained based on inertial data detection;

[0090] When the update traverses to the current frame, the selected static features of the current frame are used as the first frame observation of the feature projection, and the reverse projection method is used to project to the historical frame; if the number of dynamic object instance segmentation mask pixels in the current frame is less than 25%, the iterative extended Kalman filter update limits the maximum tracking length of each feature point in the selected static feature set to 20 frames; for every 10% increase in the number of dynamic object instance segmentation mask pixels in the current frame, the corresponding maximum tracking length of the features in the selected static feature set is reduced by 5 frames.

[0091] The zero-speed update process is as follows:

[0092] Under zero-speed conditions, the ideal values ​​of each state variable are compared with the actual predicted values ​​to obtain the residuals of the velocity, displacement, and rotation state parameters. The residual construction results are as follows:

[0093]

[0094] Following the symbolic conventions of formula (20) and formula (21), the zero-speed update operation is constructed based on the robot's moving coordinate system; the update process focuses on the relative state variables of the current position driven by inertial data Modifications are made without correcting global variables during zero speed updates and sensor deviation During the update process, as new image frames are recorded, the relative positions between the frames in the sliding window are dynamically adjusted accordingly. The zero-rate update process replaces the traditional sliding window update method of removing the oldest state with eliminating the second-to-last latest relative state in the current sliding window.

[0095] The beneficial effects of the present invention are as follows: a stationary state detection and analysis method for inertial data is proposed, and a corresponding zero-speed update calculation strategy suitable for visual inertial odometry is given, enhancing the adaptability of the visual inertial odometry method to motion patterns. A new static feature screening strategy is proposed, and a corresponding unsupervised dynamic object instance segmentation algorithm guided by geometric information is provided, improving the algorithm's ability to identify and distinguish dynamic interference in dynamic environments. The method of the present invention implements an adaptive weighted monocular visual inertial odometry method in a dynamic environment that allows for indirect participation of dynamic interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 Flowchart of a dynamic environment adaptive visual-inertial odometry method with zero-speed update capability. DETAILED DESCRIPTION

[0097] Figure 1 This is the main flow chart of the technical solution of the present invention. Figure 1 The figure shows a dynamic environment adaptive visual inertial odometry method with zero-speed update capability proposed by the present invention, which mainly includes the following steps:

[0098] 1. Multi-resolution stationary detection of inertial data

[0099] A wavelet-based signal decomposition method is used to analyze high-frequency inertial measurement unit (IMU) data in real time. Accelerometer and gyroscope signals are integrated and converted at different resolutions to determine whether the device is stationary or in motion. The entire calculation process utilizes only CPU resources.

[0100] 2. Dynamic feature detection based on inertial data

[0101] When the visual features in the image complete the inter-frame data association, the epipolar constraints are constructed by using the pre-integration results of the inertial data as the prior information of the pose between adjacent frames. After further constructing the logistic regression model using the distance from the matched feature points to the epipolar lines, the outlier judgment threshold is set to realize dynamic feature detection based on inertial data.

[0102] 3. Geometric Feature-Guided Dynamic Object Instance Segmentation

[0103] By clustering the dynamic features detected based on inertial data, the resulting cluster centers serve as 2D positional cues to guide the instance segmentation algorithm. By using the neighborhood distance of feature point data as a range parameter for clustering, the instance segmentation algorithm can simultaneously generate segmentation masks for multiple objects experiencing real-world motion within the same scene.

[0104] 4. Composite State Update Strategy

[0105] Based on the detection conclusions obtained by the multi-resolution stationary detector of inertial data, different state update strategies are configured for stationary and moving states. When the state detection conclusion is stationary, the system will implement the zero-speed update strategy; when the state detection conclusion is moving, the system will perform an iterative extended Kalman filter update operation. The simultaneous integration of these two update strategies enables the system to adapt to various motion modes.

[0106] Furthermore, according to the method of this technical solution, a dynamic environment adaptive visual inertial odometry system architecture with zero-speed update capability is established, as follows:

[0107] Preprocessing module: Responsible for feature extraction and optical flow tracking of the original image sequence, and at the same time, performing multi-resolution stationary state detection analysis on the inertial data.

[0108] Inertial Data-Based Visual Information Processing Module: After the preprocessing module extracts image features, it performs a pre-integration operation on the inertial data. Using the inter-frame pose information derived from the inertial data as prior information, the visual information processing module establishes geometric constraints to identify dynamic feature sets in dynamic environments. Clustering results from the dynamic feature set serve as prompts to guide the unsupervised instance segmentation algorithm to segment dynamic objects in real time. Once the dynamic object mask generation process is complete, the module rechecks the static information and, based on the mask, organizes the static features for the final state estimation process.

[0109] State Update Module: This module customizes different update strategies for the multi-resolution stillness detector's conclusions. If the current state is still, the system will execute a zero-speed update strategy. If the current state is moving, the system will execute an iterative extended Kalman filter weighted update strategy based on the pixel count of the segmentation mask for the dynamic object instance in the current frame.

[0110] The execution steps under the corresponding module are described as follows:

[0111] 1. Preprocessing module

[0112] Step 1: Image feature extraction

[0113] After acquiring the original image sequence, the image undergoes visual tracking. This stage identifies Shi-Tomasi corner points and applies the KLT algorithm to correlate data between frames. Uniform grid-based feature extraction is performed across the entire image area. To avoid clumping of feature points, pixel distance is used as the nearest neighbor radius for feature point screening. This step provides sufficient foundational data for subsequent steps.

[0114] Step 2: Multi-resolution static state detection of inertial data

[0115] The discrete inertial data can be decomposed into the following after wavelet transform:

[0116]

[0117] Where f(t) represents a discrete inertial data sequence, k represents the data timing offset, and j represents the multi-resolution analysis level. represents the Haar wavelet transform parent function, ψ(x) represents the Haar wavelet transform mother function, d k and c jk Indicates the detailed values ​​obtained by parsing inertial data at different resolutions. Specifically, the parent function and mother function applied here are as follows:

[0118]

[0119] The core of multi-resolution data analysis lies in determining whether the signal detail of inertial data in different segments falls below a preset threshold. Given that inertial data under static conditions exhibits zero-mean Gaussian noise, the static detector designed in this invention extracts and evaluates inertial data with varying temporal correlations during multiple operating phases. The pseudocode for processing a single dimension of acquired 6-axis IMU data is shown in Algorithm 1.

[0120] To establish a reliable stationary detection threshold, we collected pure stationary sequences and rapid startup sequences of inertial data to simulate the data scenarios that may occur during a stationary state. These sequences were analyzed using Algorithm 1 to generate judgment conclusions at different resolution levels.

[0121] Given a set of inertial data from a three-axis accelerometer and a three-axis gyroscope with aligned timestamps from an inertial measurement unit (IMU), where any axis of the three-axis accelerometer or the three-axis gyroscope contains a sequence of N inertial data IMU fixed sampling frequency f IMU ; Camera fixed sampling frequency f Cam ; Data analysis level L = 4; Wavelet transform basic sampling interval T corresponding to the original sequence of inertial data di ; Data analysis level l corresponding analysis results [Sum l ,Del l ]; The first layer static state judgment conclusion DVG l ; scalar coefficient S; adjacent inertial data static state basic judgment threshold thres; different analysis levels correspond to static judgment flags F1, F2, F3, F4; multi-resolution static detection process is as follows:

[0122] (1) Initial sequence

[0123] (2) When the data analysis level L = 4, the wavelet transform is performed with a sampling interval of 2T di Analysis of Sum5:

[0124] Sum4(i / 2)←(Sum5(i)+Sum5(iT di )) / 2

[0125] Del4(i / 2)←|Sum5(i)-Sum5(iT di )| / 2

[0126] According to [Sum4, Del4], perform static judgment of the current level and operate the flag bit:

[0127] S←T di ;DVG4←max(Del4)-min(Del4);F4←DVG4≤S×thres

[0128] l=L-1

[0129] (3) When the data analysis level L=3, the wavelet transform sampling interval starts from the second data in the positive order of Sum4 and is traversed in a loop with an interval of 1 until the traversal is completed. The traversal analysis process is:

[0130] Sum3(i-1)←(Sum4(i)+Sum4(i-1)) / 2

[0131] Del3(i-1)←|Sum4(i)-Sum4(i-1)| / 2

[0132] According to [Sum3, Del3], perform static judgment of the current level and operate the flag bit:

[0133] S←2T di ;DVG3←max(Del3)-min(Del3);F3←DVG3≤S×thres

[0134] l=L-1

[0135] (4) When the data analysis level L = 2, the wavelet transform sampling interval starts from the first data in the positive order of Sum3 and is traversed in a loop with an interval of 2 until the traversal is completed. The traversal analysis process is:

[0136] Sum2(i)←(Sum3(i)+Sum3(i+2)) / 2

[0137] Del2(i)←|Sum3(i)-Sum3(i+2)| / 2

[0138] According to [Sum2, Del2], perform static judgment of the current level and operate the flag bit:

[0139] S←6T di ;DVG2←max(Del2)-min(Del2);F2←DVG2≤S×thres

[0140] l=L-1

[0141] (5) When the data analysis level L = 1, the wavelet transform converts the two data of Sum2:

[0142] Sum1(1)←(Sum2(1)+Sum2(2)) / 2

[0143] Del1(1)←|Sum2(1)-Sum2(2)| / 2

[0144] According to [Sum1, Del1], perform static judgment of the current level and operate the flag bit:

[0145] S←12T di ;DVG1←max(Del1)-min(Del1);F1←DVG1≤S×thres

[0146] l=L-1

[0147] (6) Integrate the static analysis conclusions of each resolution level to obtain the final judgment result:

[0148] Flag=F1∩F2∩F3∩F4

[0149] According to this process, the IMU three-axis accelerometer and three-axis gyroscope are analyzed separately. As long as the static flag of any axis is not true, the current motion state is not static.

[0150]

[0151]

[0152] To better address situations where the sensor is subject to objective force but no actual movement occurs before a quick start, the present invention also incorporates the most recent sensor deviation calibration results into the static detector. After analyzing the six-dimensional inertial data, the corresponding six-axis judgment conclusions for the accelerometer and gyroscope are finally obtained.

[0153] The inherent downsampling effect of the Haar wavelet decomposition allows the algorithm to quickly capture correlations between nonlocal sequences in the inertial data. In fact, in Algorithm 1, as long as any one-dimensional inactivity flag is invalid, the device can be determined to be stationary. However, to enhance the rationality and effectiveness of this determination, the present invention performs a logical AND operation on the inertial data's inactivity flags in all six dimensions, ensuring that the inactivity detector does not misjudge.

[0154] 2. Visual information processing module based on inertial data

[0155] Step 1: Dynamic Feature Detector

[0156] After receiving the visual feature points obtained by the feature extraction process, the algorithm performs a pre-integration operation on the inertial data between frames. The pre-integration results are used to establish the pose relationship between the inertial coordinate systems associated with each frame image. By using the camera intrinsic parameter matrix and the spatial relationship between the camera and the IMU, the initial basic matrix between consecutive frames can be obtained:

[0157]

[0158] Where (·×) represents the antisymmetric matrix of a vector.

[0159] By integrating the fundamental matrix with the data association established during feature extraction, the distance from each feature point to its corresponding epipolar line can be expressed as:

[0160]

[0161] Among them, p k,i and p k+1,i are the normalized coordinates of the i-th feature point in the k and k+1 frame coordinate systems respectively.

[0162] Since the basic matrix converted from the pre-integration results of inertial data is still affected by the accumulated errors from the inertial data, the feature screening process quantifies the distance from each feature point to its corresponding epipolar line and establishes a complementary logarithmic model. This can transform the identification process of the dynamic attributes of visual features into a binary logistic regression problem, while effectively being compatible with the observation noise of the image data itself. By continuing to use the nearest neighbor distinction radius used for feature point screening during the feature extraction process, a neighborhood screening range for the vertical distance from the feature point to the epipolar line is established, and the complementary logarithmic model is used to determine the probability distribution interval of static features around the epipolar line. Therefore, the regression evaluation criteria for the judgment process can be obtained:

[0163]

[0164] This transformation of calculation method realizes the probabilistic inference of dynamic and static features, effectively solves the problem of fuzzy attribute of feature points at the junction of dynamic and static areas in the image, and lays the foundation for subsequent processing.

[0165] Step 2: Dynamic feature two-dimensional clustering

[0166] To further solve the problem of confusing attribute of feature points at the junction of dynamic and static areas in image data, the present invention performs DBSCAN clustering on the dynamic feature set initially identified in step 2. The clustering process takes advantage of the fact that feature points with the same motion trend tend to maintain consistent distribution patterns of optical flow vectors and epipolar distances. Specifically, for two feature points p in adjacent frames associated by optical flow, k,i (u1,v1) and p k+1,i (u2,v2), the geometric distance between their optical flow vectors can be expressed as

[0167] The optical flow vectors and epipolar distances of identified dynamic feature points are used as the x- and y-axes, respectively, for two-dimensional DBSCAN clustering. Unlike the traditional DBSCAN algorithm, which only concentrates on a single cluster center, the clustering process retains the neighborhood radius used during feature point screening and uses it to determine whether the clustering calculation forms multiple cluster centers. This design allows the method to adaptively identify a variety of dynamic objects in an image based on the visual scene.

[0168] Step 3: Dynamic feature-guided instance segmentation

[0169] Unlike using strong priors to directly segment all predefined objects with dynamic attributes in a single step, this method focuses on objects that are actually moving in the image. This approach enables the system to fully utilize all possible static attribute information in the current scene, thereby constructing visual constraints that are beneficial for state estimation. Inspired by this idea, the algorithm customizes a cue-based unsupervised instance segmentation algorithm based on the SAM2 work (arxiv, 2024).

[0170] The algorithm extracts the cluster centers obtained in step 3, representing the locations of various dynamic objects in the current image. When these sparsely distributed cluster centers are input as prompt words into a prompt-guided unsupervised instance segmentation algorithm model (such as the SAM2 model), the SAM2 model uses a prompt-word encoder to map the cluster center locations to image regions, generating a specialized expression format that can be used for processing. Relying on the specialized expression, the SAM2 model uses a memory method to retain and modify the feature expression of each dynamic object. On this basis, the self-attention module of the SAM2 model uses these modified features to further review the spatial relationships between different dynamic regions in the image and separate the selected spatial relationships from background information. After capturing and enhancing these spatial relationships, the mask decoder generates an instance segmentation mask result corresponding to each cluster center.

[0171] Step 4: Static feature review

[0172] Currently, the actual moving objects within a single image are captured by the instance segmentation mask obtained in step 3. To ensure accurate visual constraints for the subsequent state estimation process, the static feature set extracted in step 1 is re-evaluated based on the instance segmentation mask. This is done for two main reasons: first, a well-defined mask can resolve the issue of uncertain feature attributes at the interface between dynamic and static regions in step 2; second, the instance segmentation mask helps the algorithm assess whether the remaining visual information outside the masked area satisfies the requirements of the subsequent state estimation process.

[0173] Specifically, the review algorithm designates feature points outside the instance segmentation mask as static feature points for the iterative extended Kalman filter update process. If the number of these static feature points does not meet the minimum number required for the subsequent iterative extended Kalman filter update process, supplementary features are extracted outside the mask pixel range and new data associations are re-established. At the same time, the proportion of dynamic object pixels in the corresponding image is determined from the total number of mask pixels in each image, and this is used as prior information in the residual construction operation of the subsequent state estimation process, laying the foundation for establishing an adaptive state estimation process.

[0174] After reviewing the visual information, dynamic interference is further removed and the available static environment information is optimized. This refined visual information filtering method ensures that the visual data captured in the dynamic environment meets the static environment assumptions required by the odometry method, thereby achieving more accurate static environment pose estimation.

[0175] 3. Adaptive visual-inertial odometry centered on the robot coordinate system

[0176] Step 1: State variable definition and model parameterization

[0177]

[0178] Represents the state variable expression in the global coordinate system. The present invention specifies R k Represents the origin of the IMU coordinate system at time k. The origin of the global coordinate system is represented by G, which is equivalent to R0. is a 4×1 unit quaternion that conforms to JPL convention, representing the number from {G} to {R k} rotation. Is {G} in {R k} in the position. {R k} represents gravity. Assume that τ is the timestamp between k and k+1, Describes the k} to the current IMU coordinate origin {I τ} movement. and are relative rotations and translations. It is {I τ} represents the local velocity. and are the latest deviations of the gyroscope and accelerometer at time τ, respectively. Consists of real-time calibration parameters. and c p I represents the rotation and translation between the camera frame and the IMU frame, and t d Represents the difference between the received timestamps of the camera and IMU measurements at time step k (e.g., the IMU measurement associated with image k should correspond to timestamp tk+td). Provides the relative IMU pose in the N most recent timestamps before timestamp k.

[0179] Step 2: Inertial state propagation with covariance expansion

[0180] Assume ω m and a m Indicates the gyroscope and accelerometer measurements provided by the IMU. Represents the estimate related to the variables in formula (9). In order to enhance the recognition of the following symbols, the present invention expresses and On this basis, the present invention can express the robot-centered error state model in the continuous time domain as follows:

[0181]

[0182] in, is the IMU input noise. Because in formula (15) and It does not participate in the state propagation process controlled by motion data, so it only needs to combine the state transfer matrix F and the noise Jacobian matrix G with Build relationships.

[0183] By τ ,t τ+1 ) and perform time domain discretization and integration formula (16) to obtain δt=t τ+1 -t τ The discrete time error state transfer matrix Φ(t τ+1 ,t τ ), as shown below:

[0184]

[0185] During the entire propagation process, this step evaluates the IMU's motion in real time and introduces a covariance expansion technique to guide the entire pre-integration process. This step reconstructs the covariance transfer mechanism of error propagation by establishing the correlation between the diagonal field of view (DFV) of the visual component and the motion measurement of the inertial component within a defined sampling interval. Covariance propagation starts at time step k:

[0186]

[0187] in, Denotes the continuous-time input noise covariance matrix, and Γ is the expansion factor. During the covariance propagation process, the expansion factor Γ modifies the total covariance by enhancing the noise covariance. When the inertial sensor is fixed to a fast-moving object, the spike noise and other interruption levels in the accelerometer and gyroscope measurements can be several times higher than during standard operation, resulting in a significant decrease in measurement reliability. Therefore, this step aims to improve the constraint effect of the visual information in VIO by indirectly utilizing the image constraints of the visual sensor to calculate the expansion factor derived from the inertial measurement data. Given the diagonal field of view angle Ψ of the visual sensor and the camera frequency f C , when the inter-frame uniform motion assumption is adopted, the threshold ε for determining the motion state from the gyroscope can be derived:

[0188] ε=Ψ*f C (19)

[0189] The advantage of this operation is that the gyroscope movement can be used to ensure a common observation area between frames. Based on this premise, if the average accelerometer measurement value between frames is a v , then the final calculation of the expansion coefficient Γ is as follows:

[0190]

[0191] Real-time evaluation of IMU data via covariance inflation can produce more timely and accurate uncertainty estimates than traditional covariance propagation methods.

[0192] Assuming enhanced state It has nothing to do with the propagation process. and Packaged as The covariance of the transmission at time τ+1 can be derived as:

[0193]

[0194] in The error state transfer matrix can be calculated as:

[0195]

[0196] Step 3: Adaptive state iterative update with zero-speed update

[0197] (1) Weighted Iterative Extended Kalman Filter Update

[0198] Considering the random and variable dynamic interference in dynamic environments, and to ensure that visual information applies the most up-to-date constraints to the current frame, this paper constructs visual residuals by tracking the history of features using reverse indexing. As an alternative to traditional forward indexing that relies on initial frame observations, reverse indexing emphasizes reverse data association based on the current frame.

[0199] Through the inertial data-driven visual feature optimization module, the algorithm maintains the tracking history of all static features. As dynamic disturbances increase, the tracking length of static features in the image generally decreases. The larger the proportion of image pixels corresponding to the dynamic object instance segmentation mask, the shorter the effective tracking length of the static features used for filter updates. This ensures more reliable correlation of static data between frames used for updates. Therefore, the algorithm provides the IEKF update with a dynamically adjustable feature tracking length. This allows dynamic disturbances in the environment to have a positive impact on the state estimate without harming the state by affecting the participation of static features.

[0200] Specifically, if the percentage of dynamic object mask pixels in the current frame is less than 25%, the algorithm limits the static feature tracking length to 20 frames during the update phase. For every 10% increase in the percentage of dynamic object pixels, the static feature tracking length decreases by 5 frames. Furthermore, by combining an adjustable static feature tracking history with the last frame's inverse residual construction, the algorithm achieves the ability to closely follow sensor observations as the environment changes during the update calculation phase.

[0201] When the image sequence observes the landmark L, it is in the camera frame C i The position in can be expressed as The algorithm constructs the residual by reverse projection, so C1 represents the last frame of the current camera sequence. At this time, the relative position within the sliding window needs to be readjusted. If you know {R1} and {R i} between the rotation and translation (ie and ),but It can be expressed as:

[0202]

[0203] The online calibration strategy uses t d To accurately associate each camera frame observation with the motion process. This can be modeled using the inverse depth form:

[0204]

[0205] At the same time, this step also defines λ:=[φ,ψ,ρ] T , and Formula (23) can be expressed as:

[0206]

[0207] At this time, the corresponding measurement model is:

[0208]

[0209] in, is the image noise. After linearizing the variables in formula (15) and λ using this new model, the result can be approximated as:

[0210]

[0211] Based on the deviation construction, the update step will iterate at different linearization points; this step defines the state variable of the jth iteration as x j , the incremental component is defined as δx j, the initial point is defined as x0. Considering that the linearization point changes with each iteration, it is necessary to project the obtained Gaussian estimate from the initial point x0 to x j , which requires an additional Jacobian matrix:

[0212]

[0213] in, and Denote the summation and difference operations associated with each parameter in the state variables, respectively. Concerning the construction of linear transmission projection relationships between iterative states, this step can immediately transfer the error state and covariance from the initial state to other stages of the iteration according to formula (29):

[0214]

[0215] The update calculation for the jth iteration can be expressed as follows:

[0216]

[0217] After obtaining the latest value of the current state, the current step will have to modify the current state accordingly:

[0218]

[0219] Each iteration is performed in this way. Soon after the iteration converges, x j+1 It will be transmitted to the subsequent moments as the final posterior mean. At the same time, the final posterior covariance will be determined as follows:

[0220]

[0221]

[0222]

[0223] (2) Zero-speed update

[0224] The present invention constructs a unique closed-loop zero-speed update method for the system. Although the polar coordinate inverse depth feature parameterization method allows the algorithm to maintain normal filter updates in a static state, the increasing inertial sensor drift and interference from the dynamic environment that accompanies static conditions still make it impossible for visual information to suppress the increase in cumulative errors. The present invention believes that a reasonable state estimation algorithm in a dynamic environment should ensure that the zero-speed update process relies only on sensor data that is not affected by the dynamic environment. Therefore, the algorithm treats zero-speed updates and traditional filter updates as equals and incorporates them into the state estimation update module.

[0225] The algorithm compares the ideal value of each state variable with the actual predicted value at zero speed and derives the residuals of the velocity, displacement and rotation state parameters. The residual construction results are as follows:

[0226]

[0227] The zero-speed update operation of the present invention is constructed based on the robot's mobile coordinate system. The algorithm mainly focuses on modifying the relative state variables driven by inertial data based on the current position, without correcting global variables and sensor deviations during the zero-speed update. During the update process, the algorithm dynamically adjusts the relative posture between each frame in the sliding window in response to the entry of new image frames. In addition, as image data is continuously acquired in a static state, the algorithm replaces the traditional sliding window update method of removing the oldest state with eliminating the second-to-last latest relative state in the current sliding window. The advantage of this is that as the static state continues, the visual information with increasingly weaker constraints in the update phase can maintain the visual information association consistent with the historical frame observations before the static state while maintaining the sliding window posture in the inertial data. Such an operation is conducive to the system achieving a smooth transition of the sliding window constraint from inertial data to visual information after the device resumes the motion state, avoiding the cumulative error caused by the reconstruction of the sliding window.

[0228] After completing the correction of the state variables in the system update phase, the present invention uses the delayed state expansion and state combination process described in the document "The International Journal of Robotics Research, 667-689, 2022" to finally realize the state conversion from the dynamic coordinate system to the global coordinate system posture.

[0229] To verify the actual working effect of the algorithm proposed in this paper, the present invention was tested on multiple datasets such as EuRoC, ADVIO, VIODE and WHUVID. The experimental results show that the method proposed in this paper can maintain high accuracy and robustness in different dynamic environments with different motion modes, and is significantly better than the existing VIO algorithm.

Claims

1. A dynamic environment adaptive visual inertial odometry method with zero-speed update capability, characterized by: Perform grid uniform feature extraction and optical flow tracking on the original image sequence; perform multi-resolution stationary state detection analysis on inertial data; When the static state detection analysis concludes that the state is moving, pre-integration calculation and inertial state propagation calculation are performed based on inertial data; the pre-integration results of inertial data between adjacent frames are cached using adjacent image timestamps as intervals; When the extracted image features are used for dynamic feature detection, the pre-integration results of the inertial data are used as the prior information of the pose between adjacent frames to construct epipolar constraints. The dynamic feature set and the original static feature set in the dynamic environment are obtained based on the epipolar constraints. Based on the dynamic feature set, multiple cluster centers obtained by clustering calculation are used as prompt words to guide the unsupervised instance segmentation algorithm to complete the instance segmentation of multiple dynamic objects in the scene; After generating the dynamic object instance segmentation mask, the static information is rechecked and the static feature set is finally obtained based on the mask for the iterative extended Kalman filter update process. When stationary: execute zero-speed update strategy; In the motion state, the iterative extended Kalman filter update strategy is executed in a dynamically adjusted manner based on the proportion of pixels in the segmentation mask of the dynamic object instance in the current frame, the effective tracking length of the static features used is constructed by the proportion of mask pixels and the visual residual.

2. The method for dynamic environment adaptive visual inertial odometry with zero-speed update capability according to claim 1, characterized in that: The multi-resolution stationary state detection analysis is specifically as follows: The discrete inertial data is decomposed into the following after wavelet transform: Where f(t) represents a discrete inertial data sequence, k represents the data timing offset, and j represents the multi-resolution analysis level. represents the Haar wavelet transform parent function, ψ(x) represents the Haar wavelet transform mother function, d k and c jk Indicates the detailed values ​​obtained by parsing inertial data at different resolutions; The parent and mother functions of the application are as follows: Given a set of inertial data from a three-axis accelerometer and a three-axis gyroscope with aligned timestamps from an inertial measurement unit (IMU), where any axis of the three-axis accelerometer or the three-axis gyroscope contains a sequence of N inertial data IMU fixed sampling frequency f IMU ; Camera fixed sampling frequency f Cam ; Data analysis level L = 4; Wavelet transform basic sampling interval T corresponding to the original sequence of inertial data di ; Data analysis second level corresponding analysis results [Sum l , Del l ]; The second layer static state judgment conclusion DVG l ; scalar coefficient S; adjacent inertial data static state basic judgment threshold thres; different analysis levels correspond to static judgment flags F1, F2, F3, F4; multi-resolution static detection process is as follows: (1) Initial sequence T di ←f IMU / (10*f Cam ); (2) When the data analysis level L = 4, the wavelet transform is performed with a sampling interval of 2T di Analysis of Sum5: Sum4(i / 2)←(Sum5(i)+Sum5(i-T di )) / 2 (3) Del4(i / 2)←|Sum5(i)-Sum5(i-T di )| / 2 (4) According to [Sum4, Del4], perform static judgment of the current level and operate the flag bit: S←T di ;DVG4←max(Del4)-min(Del4);F4←DVG4≤S×thres (5) l＝L-1 (6) (3) When the data analysis level L=3, the wavelet transform sampling interval starts from the second data in the positive order of Sum4 and is traversed in a loop with an interval of 1 until the traversal is completed. The traversal analysis process is: Sum3(i-1)←(Sum4(i)+Sum4(i-1)) / 2 (7) Del3(i-1)←|Sum4(i)-Sum4(i-1)| / 2 (8) According to [Sum3, Del3], perform static judgment of the current level and operate the flag bit: <h2 style=";text-align:left;direction:ltr">S←2T<h2 style=";text-align:left;direction:ltr"> di <h2 style=";text-align:left;direction:ltr"> ;DVG3←max(Del3)-min(Del3);F3←DVG3≤S×thres (9) l＝L-1 (10) (4) When the data analysis level L = 2, the wavelet transform sampling interval starts from the first data in the positive order of Sum3 and is traversed in a loop with an interval of 2 until the traversal is completed. The traversal analysis process is: Sum2(i)←(Sum3(i)+Sum3(i+2)) / 2 (11) Del2(i)←|Sum3(i)-Sum3(i+2)| / 2 (12) According to [Sum2, Del2], perform static judgment of the current level and operate the flag bit: S←6T di ;DVG2←max(Del2)-min(Del2);F2←DVG2≤S×thres (13) l＝L-1 (14) (5) When the data analysis level L = 1, the wavelet transform converts the two data of Sum2: Sum1(1)←(Sum2(1)+Sum2(2)) / 2 (15) Del1(1)←|Sum2(1)-Sum2(2)| / 2 (16) According to [Sum1, Del1], perform static judgment of the current level and operate the flag bit: S←12T di ;DVG1←max(Del1)-min(Del1);F1←DVG1≤S×thres (17) l＝L-1 (18) (6) Integrate the static analysis conclusions of each resolution level to obtain the final judgment result: Flag=F1∩F2∩F3∩F4 (19) According to this process, the IMU three-axis accelerometer and three-axis gyroscope are analyzed separately. As long as the static flag of any axis is not true, the current motion state is not static.

3. The method of dynamic environment adaptive visual inertial odometry with zero-speed update capability according to claim 1, characterized in that: The pre-integration calculation process is as follows: Visual inertial odometry state variable definition: Represents the state variable expression in the global coordinate system; specify R k Represents the origin of the IMU coordinate system at time k; the origin of the global coordinate system is represented by G, which is equivalent to R0; is a 4×1 unit quaternion that conforms to JPL convention, representing the number from {G} to {R k }'s rotation; Is {G} in {R k } in the position; {R k } represents gravity; τ is the timestamp between k and k+1, Description from {R k } to the current IMU coordinate origin {I τ } movement; and is the relative rotation and translation; It is {I τ } represents the local velocity; and They represent the latest deviations of the three-axis gyroscope and the three-axis accelerometer at time τ respectively; It consists of real-time calibration parameters; and C p I represents the rotation and translation between the camera frame and the IMU frame, and t d represents the difference between the reception timestamps of the camera and IMU measurements at time step k; Provide the relative IMU pose in the N most recent timestamps before timestamp k; Assume ω m and a m Indicates the three-axis gyroscope and three-axis accelerometer measurements provided by the IMU; at the same time, the symbol denotes the estimate associated with the variables in formula (20); and On this basis, the robot-centered error state model in the continuous time domain is expressed as follows: in, is the IMU input noise; in formula (20) and It does not participate in the state propagation process controlled by motion data, so the state transfer matrix F and the noise Jacobian matrix G are only related to Building relationships; In the time interval [t τ ,t τ+1 ) performs time domain discretization and integral calculation on formula (21), and obtains δt=t τ+1 -t τ The discrete time error state transfer matrix Φ(t τ+1 ,t τ ): The entire propagation process evaluates the motion of the IMU in real time and introduces covariance expansion to guide the pre-integration process; the relationship between the diagonal field of view of the visual component and the motion measurement of the inertial component is established within the defined sampling interval, and the covariance transfer mechanism of error propagation is reconstructed; covariance propagation starts from time step k: in, represents the continuous-time input noise covariance matrix, Γ is the expansion coefficient; given the diagonal field of view Ψ of the visual sensor and the camera frequency f C , when the inter-frame uniform motion assumption is adopted, the threshold ε for determining the motion state from the three-axis gyroscope is derived: e=Ψ*f C (24) The average accelerometer measurement value between frames is specified as a v , the expansion coefficient Γ is as follows: Due to the enhanced state It has nothing to do with the propagation process. and Packaged as This gives the covariance of the transfer at time τ+1: in The error state transfer matrix is ​​calculated as: Formula (22) to Formula (27) are executed cyclically in sequence according to the time step to obtain the pre-integration result.

4. The method of dynamic environment adaptive visual inertial odometry with zero-speed update capability according to claim 1, characterized in that: The process of acquiring the dynamic feature set in the dynamic environment is as follows: During the feature extraction and optical flow tracking phase, Shi-Tomasi corner points in the image are identified and the KLT algorithm is used to correlate data between frames. A grid-based uniform feature extraction is performed across the entire image area, and the distance between feature point pixels is calculated. During feature point screening, the nearest neighbor radius is set to merge and screen adjacent feature points. While obtaining the visual feature points obtained in the feature extraction process, a pre-integration operation is performed on the inter-frame inertial data in the motion state. The pre-integration results are used to establish the pose relationship between the inertial coordinate systems associated with each frame image; through the camera intrinsic parameter matrix K and the spatial relationship between the camera and IMU With the pre-integration result The pose transformation between the corresponding camera coordinate systems is It can be further decomposed into rotation and pan After that, the initial basic matrix between consecutive frames is obtained Where (·×) represents the antisymmetric matrix of a vector; Integrating the initial fundamental matrix with the data association established during feature extraction, the distance from each visual feature point to its corresponding epipolar line is expressed as: Among them, p k,i and p k+1,i are the normalized coordinates of the i-th visual feature point in the k-frame coordinate system and the k+1-frame coordinate system respectively; Following the nearest neighbor distinction radius used for visual feature point screening during feature extraction, a neighborhood screening range of the vertical distance from the visual feature point to the epipolar line is established. The complementary logarithmic model is used to determine the static feature probability distribution interval around the epipolar line, and the regression evaluation standard of the judgment process is obtained: After regression judgment of the complementary logarithm model, the feature points obtained in the feature extraction process are divided into two parts: dynamic feature set and original static feature set.

5. The method of dynamic environment adaptive visual inertial odometry with zero-speed update capability according to claim 1, characterized in that: The instance segmentation of the dynamic object is as follows: Extract multiple cluster centers obtained by clustering calculations; input the multiple cluster centers as prompt words into a prompt-guided unsupervised instance segmentation algorithm model; map the two-dimensional locations of the cluster centers to image regions through a prompt-guided encoder, and generate a specialized expression format for processing; relying on the specialized expression, the prompt-based unsupervised instance segmentation model uses an internal encoding memory mechanism to retain and modify the feature expression form of each dynamic object; On this basis, the self-attention module built into the cue-word-based instance segmentation model uses the modified feature expression to further review the spatial relationship between different dynamic regions in the image and separate the screened spatial relationship from the background information; the mask decoder captures and enhances the spatial correlation and generates the instance segmentation mask result corresponding to each cluster center.

6. The method of dynamic environment adaptive visual inertial odometry with zero-speed update capability according to claim 1, characterized in that: The review of the static information is specifically as follows: The original static feature set extracted is re-evaluated based on the instance segmentation mask; Specifically, the visual feature points outside the instance segmentation mask are designated as static feature points for the subsequent iterative extended Kalman filter update process. When the number does not meet the minimum number requirement for the subsequent iterative extended Kalman filter update process, supplementary features are extracted outside the mask pixel range and the corresponding new data associations are re-established, finally obtaining a selected static feature set within the frame.

7. The method of dynamic environment adaptive visual inertial odometry with zero-speed update capability according to claim 6, characterized in that: The dynamic environment adaptive iterative extended Kalman filter update is specifically as follows: In motion, the dynamic object instance segmentation mask corresponding to each frame image and the selected static feature set corresponding to the frame are obtained based on inertial data detection; When the update traverses to the current frame, the selected static features of the current frame are used as the first frame observation of the feature projection, and the reverse projection method is used to project to the historical frame; if the number of pixels in the segmentation mask of the dynamic object instance in the current frame accounts for less than 25%, the iterative extended Kalman filter update limits the maximum tracking length of each feature point in the selected static feature set to 20 frames; For every 10% increase in the number of pixels in the segmentation mask of dynamic object instances in the current frame, the longest tracking length of the corresponding features in the selected static feature set is reduced by 5 frames.

8. The method of dynamic environment adaptive visual inertial odometry with zero-speed update capability according to claim 1, characterized in that: The zero-speed update process is as follows: Under zero-speed conditions, the ideal values ​​of each state variable are compared with the actual predicted values ​​to obtain the residuals of the velocity, displacement, and rotation state parameters. The residual construction results are as follows: Following the symbolic conventions of formula (20) and formula (21), the zero-speed update operation is constructed based on the robot's moving coordinate system; the update process focuses on the relative state variables of the current position driven by inertial data Modifications are made without correcting global variables during zero speed updates and sensor deviation During the update process, as new image frames are recorded, the relative positions between the frames in the sliding window are dynamically adjusted accordingly. The zero-rate update process replaces the traditional sliding window update method of removing the oldest state with eliminating the second-to-last latest relative state in the current sliding window.

Images