An autonomous positioning method based on multi-source fusion of visual-inertial feature maps

Abstract

This application belongs to the field of guidance technology. This application provides an autonomous positioning method based on multi-source fusion of visual inertial feature maps. It includes: performing visual inertial fusion positioning on IMU information and visual image information to obtain a visual inertial fusion result; performing feature map matching positioning on the visual inertial fusion result to obtain a feature map matching result; and performing dynamic correction on the visual inertial fusion result and the feature map matching result to obtain a multi-source fusion positioning result of the visual inertial feature map. In the embodiments of the present disclosure, through error state Kalman filtering, the feature map matching result and the estimation result of visual inertial fusion are dynamically corrected to achieve collaborative optimization of global and local information. The drift of visual inertial positioning is effectively corrected using the global constraint of the feature map; in dynamic or unknown scenarios, the continuity and real-time performance of positioning are maintained through visual inertial fusion, thereby enhancing the adaptability and stability of the system in complex dynamic environments.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of guidance technology, and in particular, to an autonomous positioning method based on multi-source fusion of visual inertial feature maps. Background Art

[0002] Simultaneous Localization and Mapping (SLAM) technology is the core technology for intelligent autonomous robots to execute task objectives. In recent years, through extensive research and application, this technology has been successfully applied to many fields such as micro unmanned aerial vehicles, intelligent driving, virtual reality, and augmented reality.

[0003] By capturing and processing rich environmental information, visual sensors can achieve feature extraction, map construction, and high-precision positioning, significantly improving navigation performance. Visual SLAM does not rely on external navigation signals and can provide reliable positioning information even when GNSS signals are weak or unavailable, greatly enhancing the flexibility and safety of tasks. However, visual SLAM is vulnerable to factors such as environmental changes, unstable lighting, and sparse feature points during long-term operation, resulting in error accumulation. Especially in dynamic scenarios, its positioning accuracy and stability are significantly challenged.

[0004] In contrast, the visual positioning method based on a prior map can efficiently and accurately achieve autonomous positioning by matching the visually acquired information in real time with a pre-constructed feature map, and performs well in known scenarios. However, in unknown or dynamically changing scenarios, the pre-constructed map may cause positioning interruption due to insufficient information, thereby affecting the stability and accuracy of the system.

[0005] Therefore, it is necessary to improve one or more problems existing in the above-mentioned related technical solutions.

[0006] It should be noted that this part is intended to provide background or context for the technical solutions of the present disclosure stated in the claims. The description herein is not admitted to be prior art merely because it is included in this part. Summary of the Invention

[0007] The purpose of the embodiments of the present disclosure is to provide an autonomous positioning method based on multi-source fusion of visual inertial feature maps, thereby at least to some extent overcoming one or more problems caused by the limitations and defects of the related technologies.

[0008] According to the embodiments of the present disclosure, there is provided an autonomous positioning method based on multi-source fusion of visual inertial feature maps, the method comprising:

[0009] Performing visual inertial fusion positioning on IMU information and visual image information to obtain a visual inertial fusion result;

[0010] Perform feature map matching and positioning on the visual-inertial fusion result to obtain the feature map matching result;

[0011] Perform dynamic correction on the visual-inertial fusion result and the feature map matching result to obtain the multi-source fusion positioning result of the visual-inertial feature map.

[0012] Further, in the step of performing visual-inertial fusion positioning on the IMU information and the visual image information to obtain the visual-inertial fusion result, it includes:

[0013] Perform pre-integration processing on the IMU information to obtain the IMU pre-integration;

[0014] Establish an IMU pre-integration residual based on the IMU pre-integration;

[0015] Extract image features from the visual image information of the current image frame to obtain the image feature points of the current image frame;

[0016] Match the image feature points to establish a visual reprojection residual;

[0017] Establish a tightly coupled optimization model based on the IMU pre-integration residual and the visual reprojection residual;

[0018] Based on the tightly coupled optimization model, use the sliding window method to perform non-linear optimization on the position and attitude of the carrier to obtain the visual-inertial fusion result.

[0019] Further, the expression of the optimization variable in the non-linear optimization is:

[0020]

[0021] Among them, is the IMU pre-integration residual, is the visual reprojection residual.

[0022] Further, the optimization variables include the position, velocity, attitude, accelerometer bias, gyroscope bias, and inverse depth of the key points of all image frames within the sliding window.

[0023] Further, in the step of performing feature map matching and positioning on the visual-inertial fusion result to obtain the feature map matching result, it includes:

[0024] Calculate the Euclidean distance between the current image frame and each key map frame in the feature map library to obtain the rough retrieval similarity metric;

[0025] Calculate the Hamming distance between the current image frame and each key map frame in the feature map library to obtain the fine retrieval similarity metric;

[0026] Using a coarse-to-fine hierarchical retrieval strategy, associate the current image frame with each key map frame in the feature map library, and through geometric constraints and optimization algorithms, obtain the feature map matching result.

[0027] Furthermore, the expression for the similarity metric of the coarse retrieval is:

[0028]

[0029] where is the -th component of the global descriptor of the current image frame , is the -th component of the global descriptor of the -th key map frame in the feature map library;

[0030] The expression for the similarity metric of the fine retrieval is:

[0031]

[0032] where is the -th local descriptor of the current image frame i , is the -th i -th component of the t -th local descriptor of the current image frame, is the -th local descriptor of the j -th key map frame in the feature map library, is the -th j -th component of the t -th local descriptor of the

[0033] Furthermore, in the step of dynamically correcting the visual-inertial fusion result and the feature map matching result to obtain the multi-source fusion positioning result of the visual-inertial feature map, it includes:

[0034] Design an error-state Kalman filter and select an error-state vector in combination with the visual-inertial fusion result;

[0035] Propagate the error-state vector based on the motion model of the visual-inertial fusion to obtain the state propagation equation;

[0036] Based on the state propagation equation, predict the error-state variable using the IMU integration value between two key map frames;

[0037] Construct a measurement model based on the feature map matching result, and update the error state vector according to the measurement model;

[0038] Dynamically correct the visual-inertial fusion result using the updated error state vector to obtain the multi-source fusion localization result of the visual-inertial feature map.

[0039] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0040] In the embodiments of the present disclosure, through the above-mentioned autonomous localization method based on multi-source fusion of visual-inertial feature maps, on the one hand, a tightly coupled optimization model of visual-inertial fusion is constructed, which can provide continuous and stable localization calculation results in a dynamic environment, significantly improving the robustness and adaptability of the system. Secondly, a coarse-to-fine hierarchical retrieval strategy is adopted to efficiently achieve fast retrieval of the feature map library in a large-scale scene, and combined with the 2D-2D epipolar geometric constraint, the global pose of the current image frame is accurately calculated, effectively improving the accuracy and efficiency of localization. On the other hand, an error state Kalman filter is designed to dynamically combine the feature map matching result with the estimation result of visual-inertial fusion, realizing the collaborative optimization of global and local information. In a known scene, the feature map provides a strong global constraint, which can effectively correct the cumulative drift of visual-inertial localization; in a dynamic or unknown scene, visual-inertial fusion localization ensures the continuity and real-time performance of the system, thereby enhancing the adaptability and stability of the system in a complex dynamic environment. Description of the Drawings

[0041] The drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0042] Figure 1 A step diagram showing an autonomous localization method based on multi-source fusion of visual-inertial feature maps in an exemplary embodiment of the present disclosure;

[0043] Figure 2 A specific flowchart showing an autonomous localization method based on multi-source fusion of visual-inertial feature maps in an exemplary embodiment of the present disclosure;

[0044] Figure 3 A trajectory comparison diagram of a visual-inertial feature map multi-source fusion localization algorithm in an exemplary embodiment of the present disclosure. Detailed Embodiments

[0045] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0046] In addition, the accompanying drawings are only schematic illustrations of the embodiments of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0047] An autonomous positioning method based on multi-source fusion of visual inertial feature maps is provided in this example embodiment. Referring to Figure 1 as shown, the autonomous positioning method based on multi-source fusion of visual inertial feature maps may include: Step S101 to Step S103.

[0048] Step S101: Perform visual inertial fusion positioning on IMU information and visual image information to obtain a visual inertial fusion result;

[0049] Step S102: Perform feature map matching positioning on the visual inertial fusion result to obtain a feature map matching result;

[0050] Step S103: Perform dynamic correction on the visual inertial fusion result and the feature map matching result to obtain a multi-source fusion positioning result of the visual inertial feature map.

[0051] Through the above autonomous positioning method based on multi-source fusion of visual inertial feature maps, on the one hand, a visual inertial fusion tightly coupled optimization model is constructed, which can provide continuous and stable positioning calculation results in a dynamic environment, significantly improving the robustness and adaptability of the system. Secondly, a coarse-to-fine hierarchical retrieval strategy is adopted to efficiently achieve fast retrieval of the feature map library in a large-scale scene, and combined with the 2D-2D epipolar geometry constraint, the global pose of the current image frame is accurately calculated, effectively improving the accuracy and efficiency of positioning. On the other hand, an error state Kalman filter is designed to dynamically combine the feature map matching result with the estimation result of visual inertial fusion, realizing the collaborative optimization of global and local information. In a known scene, the feature map provides a strong global constraint, which can effectively correct the cumulative drift of visual inertial positioning; in a dynamic or unknown scene, visual inertial fusion positioning ensures the continuity and real-time performance of the system, thereby improving the adaptability and stability of the system in a complex dynamic environment.

[0052] Next, referring to Figures 1 to 3The following provides a more detailed description of each step of the above-mentioned autonomous positioning method based on multi-source fusion of visual inertial feature maps in this exemplary embodiment.

[0053] In step S101, visual inertial fusion positioning is performed on the IMU information and visual image information to obtain a visual inertial fusion result.

[0054] Specifically, for visual inertial fusion positioning: First, pre-integration processing is performed on the IMU (Inertial Measurement Unit) information, and feature points are extracted from the visual image information and matched. On this basis, the IMU pre-integration residual and the visual reprojection residual are calculated, and a tightly coupled optimization model is established. By constructing a sliding window, the pose of the carrier is optimized and solved. Within the sliding window, the optimized state variables include the positions, velocities, poses, accelerometer biases, gyroscope biases of all image frames, and the inverse depths of all key points. The specific definitions of the state variables are as follows: , where represents the number of image frames in the sliding window; represents the total number of all keys in the sliding window; , where in the formula is the position of the carrier corresponding to the th image frame in the local coordinate system, is the velocity of the carrier corresponding to the th image frame in the local coordinate system, is the pose of the carrier corresponding to the th image frame in the local coordinate system, , respectively represent the accelerometer bias and gyroscope bias of the IMU; represents the inverse depth of the th key point.

[0055] In step S102, feature map matching positioning is performed on the visual inertial fusion result to obtain a feature map matching result.

[0056] Specifically, for feature map matching positioning: For each image frame, a global descriptor is extracted, and the global features are used to quickly screen out the candidate key frames similar to the current frame in the feature map library. Subsequently, the Hamming distance of the local features between the current frame and the candidate key frames is calculated to complete the precise retrieval of the feature map library. On this basis, through 2D-2D epipolar geometry constraint optimization, the pose of the current image frame is solved.

[0057] In step S103, dynamic correction is performed on the visual inertial fusion result and the feature map matching result to obtain a multi-source fusion positioning result of visual inertial feature maps.

[0058] Specifically, for multi-source fusion positioning of visual inertial feature maps: Design an error-state Kalman filter and select As the error state vector, where: is the position error vector, is the velocity error vector, is the attitude error vector, is the accelerometer bias error, is the gyroscope bias error, is the gravitational acceleration error vector. First, use the IMU data between image frames to predict the error state vector. Subsequently, when the feature map matching positioning solution is successful, use the matching result as the observation value to update the error state vector. Finally, use the visual-inertial fusion positioning result as the nominal state variable, and combine the updated error state vector to correct the cumulative error, thereby realizing the collaborative optimization of global and local information.

[0059] In a specific embodiment, Figure 2 shows the idea of this application, including three main modules: visual-inertial fusion positioning, feature map matching positioning, and multi-source fusion positioning of visual-inertial feature maps. First, extract and match feature points from visual image information, and perform pre-integration processing on IMU information; subsequently, calculate the IMU pre-integration residual and the visual reprojection residual, establish a tightly coupled optimization model, and obtain the visual-inertial fusion positioning result through optimization and solution. At the same time, adopt a coarse-to-fine hierarchical retrieval strategy to achieve fast retrieval of the feature map library in a large-scale scene, and combine the 2D-2D epipolar geometry constraint to accurately calculate the global pose of the current image frame. Finally, by designing an error state Kalman filter, dynamically combine the feature map matching result with the estimation result of visual-inertial fusion to achieve the collaborative optimization of global and local information.

[0060] Specific process:

[0061] Visual-inertial fusion positioning:

[0062] First, perform pre-integration processing on IMU information, extract feature points from visual image information and perform matching. On this basis, calculate the IMU pre-integration residual and the visual reprojection residual , establish a tightly coupled optimization model, and use the sliding window method to optimize and solve the pose of the carrier. The optimization objective function is: .

[0063] The optimization variables include the position, velocity, attitude, accelerometer bias, gyroscope bias of all image frames within the sliding window, and the inverse depth of all key points. Use the Levenberg-Marquardt algorithm to solve the optimization problem and obtain a high-precision carrier pose estimation result. The specific definitions of the optimization variables are as follows: , where represents the number of image frames in the sliding window; represents the total number of all keys in the sliding window; , where is the position of the carrier corresponding to the -th image frame in the local coordinate system, is the velocity of the carrier corresponding to the -th image frame in the local coordinate system, is the attitude of the carrier corresponding to the -th image frame in the local coordinate system, , respectively represent the accelerometer bias and gyroscope bias of the IMU; represents the inverse depth of the -th key point.

[0064] Feature map matching positioning

[0065] First, calculate the Euclidean distance between the current image frame and each key map frame in the feature map as a similarity measure in the rough retrieval stage:

[0066]

[0067] where is the -th component of the global descriptor of the current image frame , is the -th component of the global descriptor of the -th key map frame in the map library.

[0068] By setting a similarity threshold to filter out the preliminary candidate key frame set . When the number of the candidate key frame set is less than the threshold , it means that there are not enough candidate frames in the map library and the current image retrieval fails; otherwise, we sort the similarity scores of each map frame in and select the frames with the highest similarity scores as the candidate key frame set of the query image frame , where is the candidate key frame set, are the 1st, 2nd, 3rd,..., M-th candidate key frames respectively.

[0069] Secondly, calculate the Hamming distance as the similarity measure in the fine retrieval stage. Let be the descriptor set of the current image frame , where is the number of descriptors; let is the descriptor set of the th candidate key map frame, where is the number of descriptors. We calculate the sum of the Hamming distances between each descriptor in the th candidate key map frame and all descriptors in the current image frame :

[0070]

[0071] where is the descriptor in the current image frame and the descriptor in the th candidate key map frame. The calculation formula for the Hamming distance is:

[0072]

[0073] The smaller the Hamming distance, the more similar the features are. Calculate the sum of the Hamming distances between all descriptors of the current image frame and all candidate key frames in turn , and select the key frame with the smallest value as the final matching frame.

[0074] Finally, solve the global pose of the current frame by combining the 2D-2D epipolar geometry constraint.

[0075] Multi-source fusion localization of visual-inertial feature map:

[0076] First, select the error state vector as:

[0077]

[0078] where is the position error vector, is the velocity error vector, is the attitude error vector, is the accelerometer bias error, is the gyroscope bias error, is the gravity acceleration error vector.

[0079] Secondly, propagate the error state vector based on the motion model of visual-inertial fusion to obtain the state propagation equation of the system as , where is the state vector at k time, is the state vector at k-1 time, is the non-linear dynamic model of the state, is the IMU measurement value, is the process noise, which follows a zero-mean Gaussian distribution with covariance . After further linearization, we get , where is the state transition matrix, which depends on the current state and control input, is the process noise distribution matrix.

[0080]

[0081] In the formula is the identity matrix, is the time interval between two frames of IMU data, is k the rotation matrix at time is the accelerometer measurement, is the gyroscope measurement, is the skew-symmetric matrix.

[0082] Predict the error state variable using the IMU integration value between two key frames:

[0083]

[0084] where and are the mean square error matrix of the state estimate at the previous time and the mean square error matrix of the state prediction at the current time, respectively.

[0085] Next, introduce the feature map matching result to construct the measurement model as:

[0086]

[0087] where is the non-linear function of the measurement, is the measurement noise, which follows a zero-mean Gaussian distribution with covariance . Through linearization, the measurement error can be further expressed as:

[0088]

[0089] where is the measurement error, is the position obtained by visual-inertial fusion localization solution, is the position obtained by feature map matching localization solution, is the inverse of the attitude matrix obtained by visual-inertial fusion localization solution, is the attitude matrix obtained by feature map matching localization solution, is to convert the rotation matrix to an angle, and the measurement matrix is:

[0090]

[0091] When the feature map matching and positioning is successful, the update of the error state starts:

[0092]

[0093] Among them, is the Kalman gain, is the updated error state vector, is the updated state estimation mean square error matrix.

[0094] Finally, after completing the Kalman filtering of the error state, the visual-inertial fusion positioning result is corrected by combining the error state vector, and finally a high-precision fusion positioning result is obtained. The specific implementation of the error correction is as follows:

[0095]

[0096] Among them, is the finally obtained fusion positioning result, is the current system state, is the attitude error quaternion, and the expression is: .

[0097] Through the above implementation steps, the present application realizes efficient and stable multi-source fusion positioning, and significantly improves the positioning accuracy and system stability in complex dynamic environments.

[0098] Figure 3 shows the comparison of the experimental running trajectories of the visual-inertial feature map multi-source fusion positioning algorithm and the visual-inertial fusion positioning algorithm in a complex dynamic environment. In the figure, the solid line "— RTK" represents the positioning trajectory obtained by a high-precision real-time kinematic (RTK) system, which is used as a benchmark for accuracy comparison; the dotted line "···GLNS" represents the trajectory of the visual-inertial feature map multi-source fusion positioning algorithm; "—·— VIO" is the trajectory of the visual-inertial fusion positioning algorithm. The experimental results show that the visual-inertial fusion positioning algorithm accumulates large errors and produces obvious trajectory drifts during long-term operation, while the trajectory of the visual-inertial feature map multi-source fusion positioning algorithm proposed in the present application almost completely coincides with the benchmark trajectory RTK, proving that this algorithm can achieve continuous, stable and accurate navigation positioning in complex dynamic environments.

[0099] Through the above-mentioned autonomous positioning method based on multi-source fusion of visual inertial feature maps, on the one hand, a tightly-coupled optimization model for visual inertial fusion is constructed, which can provide continuous and stable positioning calculation results in a dynamic environment, significantly improving the robustness and adaptability of the system. Secondly, a coarse-to-fine hierarchical retrieval strategy is adopted to efficiently achieve fast retrieval of the feature map library in a large-scale scene, and combined with the 2D-2D epipolar geometry constraint, the global pose of the current image frame is accurately calculated, effectively improving the accuracy and efficiency of positioning. On the other hand, an error-state Kalman filter is designed to dynamically combine the feature map matching result with the estimation result of visual inertial fusion, realizing the collaborative optimization of global and local information. In a known scene, the feature map provides a strong global constraint, which can effectively correct the cumulative drift of visual inertial positioning; in a dynamic or unknown scene, visual inertial fusion positioning ensures the continuity and real-time performance of the system, thereby improving the adaptability and stability of the system in a complex dynamic environment.

[0100] It should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. in the above description is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the embodiments of the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the embodiments of the present disclosure.

[0101] In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, "a plurality" means two or more, unless otherwise specifically defined.

[0102] In the embodiments of the present disclosure, unless otherwise clearly specified and limited, the terms "install", "connect", "connect", "fix", etc. should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.

[0103] In the embodiments of the present disclosure, unless otherwise clearly specified or limited, the first feature being "on" or "under" the second feature may include direct contact between the first and second features, or may include indirect contact between the first and second features through additional features therebetween. Moreover, the first feature being "above", "over" and "on top of" the second feature includes the first feature being directly above and obliquely above the second feature, or merely indicating that the horizontal height of the first feature is higher than that of the second feature. The first feature being "under", "below" and "beneath" the second feature includes the first feature being directly below and obliquely below the second feature, or merely indicating that the horizontal height of the first feature is lower than that of the second feature.

[0104] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0105] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only to be regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the appended claims.

Claims

1. An autonomous positioning method based on multi-source fusion of visual inertial feature maps, characterized in that: The method includes: Perform visual-inertial fusion positioning on IMU information and visual image information to obtain visual-inertial fusion results; Perform feature map matching and positioning on the visual inertial fusion results to obtain feature map matching results; The visual-inertial fusion results and feature map matching results are dynamically corrected to obtain the visual-inertial feature map multi-source fusion positioning results; The step of performing feature map matching and positioning on the visual inertial fusion result to obtain the feature map matching result includes: Calculate the Euclidean distance between the current image frame and each key map frame in the feature map library to obtain a coarse retrieval similarity measure; Calculate the Hamming distance between the current image frame and each key map frame in the feature map library to obtain a fine retrieval similarity measure; Using a coarse-to-fine hierarchical retrieval strategy, the current image frame is associated with each key map frame in the feature map library, and the feature map matching result is obtained through geometric constraints and optimization algorithms; The expression of rough retrieval similarity measurement is: in, The current image frame The global descriptor of Quantity, The feature map library The global descriptor of the keymap frame Quantity; The expression of fine retrieval similarity measurement is: in, The current image frame No. i A local descriptor, The current image frame No. i The local descriptor t Quantity, The feature map library Keymap frame j A local descriptor, The feature map library Keymap frame j The local descriptor t components, D is the total number of feature dimensions contained in each local descriptor; The step of dynamically correcting the visual-inertial fusion result and the feature map matching result to obtain the visual-inertial feature map multi-source fusion positioning result includes: Design an error state Kalman filter and select the error state vector based on the visual inertial fusion results; The error state vector is propagated based on the motion model of visual-inertial fusion to obtain the state propagation equation; Based on the state propagation equation, the error state variable is predicted using the IMU integral value between two key map frames; A measurement model is constructed according to the feature map matching results, and the error state vector is updated according to the measurement model; The updated error state vector is used to dynamically correct the visual-inertial fusion result to obtain the multi-source fusion positioning result of the visual-inertial feature map.

2. The autonomous positioning method based on multi-source fusion of visual inertial feature maps according to claim 1 is characterized in that: The steps of performing visual-inertial fusion positioning on the IMU information and the visual image information to obtain the visual-inertial fusion result include: Pre-integrate the IMU information to obtain IMU pre-integral; Establishing IMU pre-integration residual according to IMU pre-integration; Performing image feature extraction on the visual image information of the current image frame to obtain image feature points of the current image frame; Match image feature points to establish visual reprojection residuals; A tightly coupled optimization model is established based on the IMU pre-integration residual and the visual reprojection residual; Based on the tightly coupled optimization model, the sliding window method is used to perform nonlinear optimization on the position and attitude of the carrier to obtain the visual-inertial fusion result.

3. The autonomous positioning method based on multi-source fusion of visual inertial feature maps according to claim 2 is characterized in that: The expression of the optimization variable in nonlinear optimization is: in, is the IMU pre-integration residual, is the visual reprojection residual.

4. The autonomous positioning method based on multi-source fusion of visual inertial feature maps according to claim 3 is characterized in that: The optimization variables include the position, velocity, attitude, accelerometer bias, gyroscope bias, and inverse depth of key points for all image frames within the sliding window.

Images