Method for improving positioning precision of visual inertial odometer

Abstract

The invention discloses a method, a device, a medium and equipment for improving the positioning precision of a visual inertial odometer, and the method comprises the steps: carrying out the unified representation of point features, line features, plane features and obtained camera attitudes through a unified representation model, determining a six-dimensional attitude according to the conversion relation between the positions and directions of geometric features, and carrying out the positioning of the visual inertial odometer. Screening effective information in the six-dimensional attitude vector by using a self-binding matrix; performing feature association based on the uniformly expressed features, generating feature descriptors, and matching feature correspondence between continuous image frames; the method comprises the following steps: acquiring motion data of an unmanned aerial vehicle through an IMU (Inertial Measurement Unit), dynamically selecting an integration method and adjusting a time step length according to a current motion state and an environmental condition, and pre-integrating the data acquired by the IMU to obtain an integration result so as to estimate displacement, speed and rotation increment; by introducing the visual inertial odometer and optimizing the data processing method thereof, the positioning precision of the unmanned aerial vehicle can be improved, and the high efficiency and reliability of the unmanned aerial vehicle during task execution are ensured.

Description

Technical Field

[0001] This application relates to the field of visual inertial odometry technology, and in particular to a method, apparatus, medium and device for improving the positioning accuracy of visual inertial odometry. Background Technology

[0002] In complex and ever-changing environments such as jungles and mountains, drones need to frequently and drastically adjust their three-dimensional coordinates to avoid obstacles and ensure flight safety and mission success in order to complete missions efficiently. However, although drone positioning technology in two-dimensional space is relatively mature, improving drone positioning accuracy in three-dimensional space, especially in rapidly changing scenarios, remains a challenge, particularly in ensuring accurate positioning in both horizontal and vertical directions.

[0003] In such environments, due to complex terrain and variable environmental features, UAVs not only face problems such as motion ambiguity (violent UAV movement leading to blurred images and feature loss in visual camera acquisition) and motion coupling (coupling of horizontal and vertical motion), but traditional visual SLAM (Simultaneous Localization and Mapping) technology also struggles to maintain high-precision positioning due to error accumulation, directly impacting mission success rates. To address these issues and improve UAV positioning accuracy, Visual-Inertial Odometry (VIO) offers an effective solution. VIO combines data from computer vision and an inertial measurement unit (IMU), calculating device motion by analyzing continuously captured image sequences and supplementing with acceleration and angular velocity information provided by the IMU, thereby providing reliable relative positioning information even in the absence of GPS or with weak GPS signals. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, medium, and device for improving the positioning accuracy of visual inertial odometry, aiming to solve the technical problem of improving the positioning accuracy of visual inertial odometry.

[0005] To achieve the above objectives, this application provides a method for improving the positioning accuracy of visual inertial odometry, comprising: acquiring image data through a UAV camera, and extracting point features, line features, and planar features from the image data using a hybrid feature extraction method; using a unified representation model to uniformly represent the point features, line features, planar features, and the acquired camera attitude, wherein the unified representation model includes setting an attached coordinate system for each feature element, describing the position and orientation of geometric features through the transformation from the world coordinate system to the local coordinate system, determining a six-dimensional attitude based on the transformation relationship between the position and orientation of geometric features, and using a self-binding matrix to filter effective information in the six-dimensional attitude vector; performing feature association based on the features of the unified representation, generating feature descriptors, and matching feature correspondences between consecutive image frames; acquiring UAV motion data through an IMU, and dynamically selecting an integration method and adjusting the time step according to the current motion state and environmental conditions, pre-integrating the IMU-acquired data to obtain integration results to estimate displacement, velocity, and rotation increments; fusing the feature correspondences and pre-integration results to perform pose estimation and generate UAV positioning data.

[0006] Optionally, the step of extracting point features, line features, and planar features from the image data using a hybrid feature extraction method includes: extracting key points from the image data using the FAST algorithm and representing the key points using binary robust independent elementary features to obtain point feature descriptors; simultaneously downsampling the three-dimensional point cloud of the input image data and continuously fitting line features and planar features using the RANSAC method until the number of remaining three-dimensional points drops below a preset value of the original point cloud number to obtain line features and planar features.

[0007] Optionally, the step of setting an attached coordinate system for each feature element includes: for point features, the origin is itself, and the direction of the coordinate system is freely set; for line features, the origin is any point on the straight line, and the x-axis is arranged along the direction of the straight line; for planar features, the origin is any point on the plane, and the z-axis is consistent with the normal vector of the plane.

[0008] Optionally, the coordinate system to which the camera is attached is the world coordinate system. The step of determining the six-dimensional attitude based on the transformation relationship between the position and orientation of geometric features and filtering the effective information in the six-dimensional attitude vector using a self-binding matrix includes: determining the transformation relationship between the attached coordinate systems corresponding to any two geometric features among point features, line features, surface features, and camera position features based on preset translation and rotation formulas; determining the transformation matrix between coordinate systems based on the transformation relationship between coordinate systems, and determining the six-dimensional attitude vector based on the transformation matrix; determining the first to third self-binding matrices for each of the point features, line features, surface features, and their respective symmetry properties, and determining the corresponding fourth self-binding feature based on the camera's attitude features; and obtaining the filtered effective information by multiplying the first to fourth self-binding matrices with the six-dimensional attitude vector.

[0009] Optionally, the step of performing feature association based on the features of the unified representation, generating feature descriptors, and matching feature correspondences between consecutive image frames includes: detecting key points from RGB images using the ORB algorithm, quickly finding corner points or salient points in the RGB images as key points using the FAST algorithm, calculating a binary descriptor for each key point using the BRIEF algorithm, and combining the binary descriptor with the position, scale, and orientation information of the key point to form a complete point feature descriptor; acquiring 3D point cloud data from a depth image, processing the 3D point cloud data using the RANSAC algorithm, randomly selecting a set of points as samples and fitting a straight line, calculating the distance from the remaining points to the straight line, and determining points with a distance less than a set threshold as... For lines belonging to the same straight line, the model containing the most interior points is selected as the final line feature model after repeated processing. Line feature descriptors are generated based on the endpoint positions and direction vectors of the line feature model. Three-dimensional point cloud data is acquired from depth images and processed using the RANSAC algorithm. A set of points is randomly selected as samples and fitted to a plane. The distance from the remaining points to the plane is calculated, and points with a distance less than a set threshold are considered to belong to the same plane. This process is repeated multiple times, and the model containing the most interior points is selected as the final plane feature model. Surface feature descriptors are generated based on the normal vector of the plane feature model and the position information of a point on the plane. Based on the generated point feature descriptors, line feature descriptors, and surface feature descriptors, the correspondence between consecutive image frames is matched.

[0010] Optionally, the step of collecting UAV motion data through an IMU and dynamically selecting the integration method and adjusting the time step based on the current motion state and environmental conditions includes: determining the UAV's angular velocity and acceleration based on the data collected by the IMU, and determining whether the environment around the UAV is a high-speed motion environment, a rapidly changing environment, or a stable environment based on the degree of change in the surrounding environment; if it is a high-speed motion environment or a rapidly changing environment, then a preset first step size and a third-order Adams-Bashforth algorithm are used to pre-integrate the data collected by the IMU; if it is a stable environment, then a preset second step size and a first-order Euler algorithm are used to pre-integrate the data collected by the IMU; wherein, the first step size is smaller than the second step size.

[0011] Optionally, the step of pre-integrating the IMU-acquired data to obtain the integration result for estimating the displacement, velocity, and rotation increments includes: pre-integrating the IMU-acquired data using a third-order Adams-Bashforth algorithm to obtain the velocity increment, displacement increment, and rotation increment, respectively; wherein the pre-integration formula for the velocity increment is:

[0012]

[0013] In the formula, Δv ij From time t i to t j Speed ​​increment, R k It is the attitude rotation matrix at the k-th sampling time, a k b is the acceleration measurement value at the k-th sampling time. a It is the deviation of the accelerometer, Δt k It is the time interval of the kth sampling moment;

[0014] The pre-integral formula for the displacement increment is:

[0015]

[0016] In the formula, Δp ij From time t i to t j Position increment, v k It is the velocity at the k-th sampling time.

[0017] The pre-integral formula for the rotation increment is:

[0018]

[0019] In the formula, ΔR ij From time t i to t j The rotation increment, ω k It is the angular velocity measurement value at the k-th sampling time, b gIt's a deviation in the gyroscope.

[0020] Furthermore, to achieve the above objectives, this application also provides an improved visual inertial odometry positioning device, comprising: a feature extraction module, used to acquire image data through a UAV camera and extract point features, line features, and planar features from the image data using a hybrid feature extraction method; and an effective information acquisition module, used to uniformly represent the point features, line features, planar features, and the acquired camera pose using a unified representation model, wherein the unified representation model includes setting an attached coordinate system for each feature element, describing the position and orientation of geometric features through the transformation from the world coordinate system to the local coordinate system, and determining the effective information acquisition module based on the transformation relationship between the position and orientation of the geometric features. The system defines a six-dimensional attitude and uses a self-binding matrix to filter effective information from the six-dimensional attitude vector. A feature correspondence determination module is used to perform feature association based on the unified representation, generate feature descriptors, and match feature correspondences between consecutive image frames. An incremental estimation module is used to collect UAV motion data through an IMU and dynamically select an integration method and adjust the time step according to the current motion state and environmental conditions. It pre-integrates the IMU-collected data to obtain integration results, estimating displacement, velocity, and rotation increments. A positioning data generation module is used to fuse the feature correspondences and pre-integration results to perform pose estimation and generate positioning data for the UAV.

[0021] To achieve the above objectives, this application also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method for improving the positioning accuracy of a visual inertial odometry provided in the above embodiments.

[0022] To achieve the above objectives, this application also provides an electronic device, the electronic device comprising: at least one processor, a memory, and an input / output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the method for improving the positioning accuracy of a visual inertial odometry provided in any of the foregoing embodiments.

[0023] This application proposes a method, apparatus, medium, and device for improving the positioning accuracy of visual inertial odometry. It utilizes a unified representation model to uniformly represent point features, line features, planar features, and the acquired camera attitude. This unified representation model includes setting an attached coordinate system for each feature element, describing the position and orientation of geometric features through a transformation from the world coordinate system to the local coordinate system, determining the six-dimensional attitude based on the transformation relationship between the position and orientation of geometric features, and using a self-binding matrix to filter effective information from the six-dimensional attitude vector. Based on the features of the unified representation, feature association is performed to generate feature descriptors and match feature correspondences between consecutive image frames. Appropriate feature extraction algorithms are used for different types of features, and these features are uniformly represented. By fully utilizing the geometric information between different features, the position and attitude of the UAV can be estimated more accurately. The UAV motion data is collected by an IMU, and the integration method and time step are dynamically selected and adjusted according to the current motion state and environmental conditions. Pre-integration of the IMU-collected data yields integration results to estimate displacement, velocity, and rotation increments. For IMU data processing, appropriate IMU pre-integration algorithms are selected according to different UAV motion states, effectively reducing error accumulation caused by rapid motion. This approach not only enhances the system's adaptability to dynamic environmental changes but also significantly improves the positioning accuracy of UAVs in complex scenarios. By introducing visual inertial odometry and optimizing its data processing methods, this application can significantly improve the positioning accuracy of UAVs in scenarios with drastic changes in three-dimensional space, ensuring their efficiency and reliability during mission execution. This strategy provides a new perspective and technical support for addressing the challenges faced by UAVs in complex environments. Attached Figure Description

[0024] Figure 1 A flowchart illustrating an embodiment of the method for improving the positioning accuracy of a visual inertial odometer according to this application; Figure 2 A flowchart is provided for an embodiment of the method for improving the positioning accuracy of a visual inertial odometer according to this application; Figure 3 A flowchart illustrating the feature matching process provided in an embodiment of the method for improving the positioning accuracy of a visual inertial odometer according to this application; Figure 4 A schematic diagram of the attachment of feature elements in a coordinate system provided in an embodiment of the method for improving the positioning accuracy of a visual inertial odometer according to this application; Figure 5 This is a schematic diagram of the pre-integration process provided in an embodiment of the method for improving the positioning accuracy of a visual inertial odometer according to this application.

[0025] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0030] In complex and ever-changing environments such as jungles and mountains, drones must frequently and drastically adjust their three-dimensional coordinates to avoid obstacles in order to efficiently complete missions and ensure flight safety and mission success rates. However, in these challenging scenarios, the rapid changes in complex terrain and environmental features not only lead to problems such as motion blur and motion coupling, but also cause errors in traditional visual SLAM technology to accumulate rapidly, thus seriously affecting positioning accuracy and reducing mission success rates.

[0031] To address the aforementioned issues, this invention proposes a visual inertial odometry (VIO) method specifically for UAVs operating in rapidly changing environments. This method introduces a hybrid feature extraction algorithm, enabling the extraction of more types of geometric information (e.g., line and planar features) from complex environments, thereby increasing the amount of information available for pose estimation. Furthermore, by combining dynamically adjusted time steps and an IMU pre-integration strategy, this method effectively reduces the rate of error accumulation, enhancing the system's stability and accuracy.

[0032] This innovative approach not only provides a new concept and technical means to improve the positioning accuracy of UAVs in drastically changing environments, but also demonstrates its potential value in multiple application areas. Specifically, this invention is expected to significantly improve work efficiency, reduce labor costs, and lower operational risks. The results of practical applications further validate the effectiveness and reliability of the proposed method, proving its significant practical value in solving the error accumulation problem encountered by UAVs in complex environments such as jungles and mountains. The application of this method is of great significance for advancing UAV technology in fields such as search and rescue, topographic mapping, and environmental monitoring.

[0033] The main solution of this application achieves the following objectives through two core innovations. First, a multi-feature fusion feature matching algorithm is employed to increase the variety and quantity of features, thereby overcoming the problem of insufficient feature points caused by image blurring. This method not only enhances the robustness of the system in complex environments but also improves the ability to extract effective information from low-quality images. Second, a strategy for dynamically adjusting the pre-integration of the inertial measurement unit (IMU) is introduced to reduce the accumulation of errors generated by the UAV during violent movements. This adaptive adjustment mechanism can optimize the IMU data processing process according to the actual motion state of the UAV, thereby providing more accurate state estimation.

[0034] These two improvements work together to effectively enhance the positioning accuracy of UAVs in rapidly changing environments. Theoretically, this application provides new ideas and technical support for solving the positioning challenges of UAVs in highly dynamic environments. In practical applications, this method is particularly suitable for complex and variable environments such as jungles and mountains, enabling UAVs to perform tasks efficiently under these conditions, such as search and rescue, terrain mapping, and environmental monitoring. This technological advancement not only broadens the application scope of UAVs but also lays a solid foundation for their use in more challenging tasks.

[0035] The following is a detailed description of some of the improvement methods used in this application:

[0036] 1. Visual Inertial Odometry Based on Kalman Filters: By optimizing the design of the Kalman filter, this method can more effectively fuse data from visual sensors and inertial measurement units, thereby improving the accuracy and stability of positioning. This method is particularly suitable for dynamic environments, can better handle noise, and provide consistently stable estimates.

[0037] 2. Visual Inertial Odometry Based on Point and Line Features: Compared to traditional methods that rely solely on point features, introducing line features not only increases the acquisition of information about the environmental geometry but also significantly improves the robustness of the system in low-texture or rapidly changing lighting conditions. This method is particularly suitable for situations where point features are difficult to detect or track reliably, providing additional stability and reliability for navigation in complex environments.

[0038] 3. Optimized Visual Inertial Odometry: While nonlinear optimization can achieve high-precision state estimation, traditional methods face challenges due to long operating times and long motion trajectories. The improved optimization algorithm aims to enhance real-time performance, ensuring efficient computation and accurate estimation even with increasing data volumes.

[0039] 4. Visual Inertial Odometry Based on Direct Methods: Unlike traditional methods that rely on specific image feature extraction and matching, direct methods utilize image brightness information for state estimation, reducing preprocessing steps and improving processing speed and adaptability to changes in illumination. This method is particularly suitable for high-speed moving scenes and effectively alleviates problems caused by missing features.

[0040] 5. Event Camera-Based Visual Inertial Odometry: Utilizing the unique asynchronous brightness change detection capability of event cameras, this method demonstrates significant advantages in handling high-speed motion and extreme lighting conditions. Compared to traditional cameras, event cameras can provide more stable and accurate data in rapidly changing lighting conditions or extreme environments, greatly expanding the application scope of VIO.

[0041] 6. Visual inertial odometry based on deep learning: This method not only has powerful feature extraction capabilities and can adapt to complex situations such as featureless regions and motion blur, but also can directly process raw high-dimensional data through an end-to-end learning model.

[0042] Currently, UAV positioning technology in two-dimensional planes is relatively mature both domestically and internationally. However, when it comes to precise positioning in three-dimensional space, especially in complex and rapidly changing scenarios, many challenges remain. Specifically, images acquired by visual cameras in rapidly changing scenes are often blurry, and the cumulative effect of errors exacerbates this, leading to a significant decrease in the positioning accuracy of UAVs in these environments.

[0043] 1. Visual-inertial odometry based on Kalman filter: Although this method can effectively fuse visual and inertial data, in high dynamic scenarios, the existence of model linearization error may lead to lower-than-expected estimation accuracy.

[0044] 2. Visual inertial odometry based on point and line features: Although introducing line features can enhance the robustness of the system in scenes with low texture or large changes in lighting conditions, finding enough effective line features may be difficult in some cases, which limits its widespread application.

[0045] 3. Optimization-based visual inertial odometry: Nonlinear optimization methods can provide high-precision state estimation, but as the running time increases and the motion trajectory grows, real-time optimization becomes increasingly difficult and may cause computational delay problems.

[0046] 4. Visual inertial odometry based on direct methods: This method does not rely on specific image feature extraction, but directly uses image brightness information for state estimation. Although this improves processing speed and adaptability to changes in illumination, it may not perform well under poor lighting conditions.

[0047] 5. Visual inertial odometry based on event cameras: Although event cameras perform well in high-speed motion and extreme lighting conditions, the effective use of such data is still in the exploration and development stage and has not yet been fully matured.

[0048] 6. Visual inertial odometry based on deep learning: These methods usually require a large amount of training data, and the design and training process of the model is relatively complex, which also brings certain challenges to its practical application.

[0049] This application aims to propose a visual inertial odometry (VIO) method specifically for unmanned aerial vehicles (UAVs) in rapidly changing scenarios, significantly improving their positioning accuracy. This application employs a combination of theoretical analysis and numerical simulation, focusing on the design of a feature matching algorithm for multi-feature fusion and a dynamically adjusted IMU pre-integration strategy. A series of simulation experiments are used to verify and optimize the proposed method, forming a complete invention loop. Figure 2 The technical approach of this application is illustrated.

[0050] First, raw data is acquired using a visual camera and an IMU, and then preprocessed using a multi-feature fusion feature extraction algorithm and a dynamically adjusted IMU pre-integration strategy. This step aims to enhance the diversity and quantity of feature points while reducing error accumulation caused by the drone's vigorous movements.

[0051] Next, the pose information estimated visually is aligned with the results of IMU pre-integration to complete the system initialization process. Subsequently, the visual information and IMU data are combined using a sliding window technique to perform nonlinear local optimization, thereby outputting a more accurate pose estimation result.

[0052] Furthermore, loop closure detection plays a crucial role in the long-term operation of the system, helping to eliminate accumulated errors. When a loop closure is detected, the system comprehensively considers camera constraints, IMU constraints, and the additional constraints provided by loop closure detection, performing global nonlinear optimization. This process, based on previous local optimization results, further refines the pose estimation along the entire trajectory, ensuring that the final output positioning result has higher accuracy and reliability.

[0053] Specifically, to address the issues of insufficient feature points and feature loss caused by motion blur in rapidly changing environments, this application proposes a visual-inertial odometry calculation method based on multi-feature fusion. This algorithm aims to increase the number of features and provide additional geometric constraints by introducing a hybrid feature extraction method to further extract line and planar features on top of traditional point features. This improves the accuracy and robustness of feature matching and reduces errors in pose estimation.

[0054] The operation flow of the feature matching algorithm with multi-feature fusion is as follows: Figure 3As shown, firstly, the accelerated segment test (FAST) algorithm in the ORB-SLAM2 framework is used to detect keypoints in the image, and these keypoints are represented using the binary robust independent elementary feature (BRIEF) descriptor. For line and planar features, the Random Sample Consensus (RANSAC) method is used to fit the 3D point cloud data obtained from the depth image to improve the diversity and stability of feature extraction. Secondly, to simplify the calculation process and ensure that different types of features can be processed efficiently under the same framework, an SP model based on an optimized framework is proposed for a unified representation of features. All types of features and camera poses are transformed into a compact SP representation, and a self-binding matrix is ​​introduced to filter effective information in the six-dimensional pose vector and eliminate uncertainties. This method not only improves the efficiency of the system but also avoids the conversion problem between different representation methods. Finally, a new feature association scheme is proposed, which comprehensively considers the geometric relationships between similar and dissimilar features, as well as the geometric information in the environment. By generating feature descriptors and using Euclidean distance to evaluate the distance between predicted features from the previous frame and features in the current frame, two features are considered related when the distance is less than a set threshold. Furthermore, the nearest neighbor (NN) algorithm is also used to match feature correspondences between consecutive image frames, enhancing the system's adaptability in cases of feature sparsity.

[0055] Reference Figure 1 The method for improving the positioning accuracy of a visual inertial odometry provided in the first embodiment of this application may include the following execution process:

[0056] S10. Collect image data using a drone camera, and extract point features, line features, and planar features from the image data using a hybrid feature extraction method;

[0057] Specifically, step S10 may include the following execution process:

[0058] S101. Use the FAST algorithm to extract key points from image data, and represent the key points using binary robust independent elementary features to obtain point feature descriptors.

[0059] S102. Simultaneously, the 3D point cloud of the input image data is downsampled, and the RANSAC method is used to continuously fit line features and planar features until the number of remaining 3D points is reduced to below a preset value of the original point cloud number, thus obtaining line features and planar features.

[0060] Notably, in the feature extraction stage, this application employs the ORB-SLAM2 framework for point feature extraction and matching. Specifically, the Faster Segment Test (FAST) algorithm is used to detect key points in the image, and the descriptors of these key points are calculated using the Binary Robust Independent Elementary Features (BRIEF) algorithm. For the extraction of line and planar features, the Random Sample Consensus (RANSAC) method is used to process the 3D point cloud data obtained from the input depth image. This process first downsamples the 3D point cloud to reduce computation, and then continuously fits line or planar features until the number of remaining 3D points in the point cloud is reduced to less than 10% of the number of points in the original point cloud. Through the above methods, point, line, and planar geometric features are obtained respectively, providing a foundation for subsequent unified representation and multi-feature fusion.

[0061] In another embodiment, the feature extraction and matching algorithm of this application can also adopt feature extraction algorithms widely used in computer vision, such as ORB, RANSAC, FAST, SIFI, etc., which can directly extract the required features using deep learning network models.

[0062] S20. Use a unified representation model to uniformly represent the point features, line features, planar features, and the acquired camera pose. The unified representation model includes setting an attached coordinate system for each feature element, describing the position and orientation of the geometric features through the transformation from the world coordinate system to the local coordinate system, determining the six-dimensional pose based on the transformation relationship between the position and orientation of the geometric features, and using a self-binding matrix to filter the effective information in the six-dimensional pose vector.

[0063] The process of setting the attachment coordinate system for each feature element may include the following execution steps:

[0064] The point feature has itself as the origin, and the direction of the coordinate system can be freely set;

[0065] The origin of the coordinate system for the linear feature is any point on the straight line, and the x-axis is arranged along the direction of the straight line.

[0066] The origin of the plane characteristic coordinate system is located at any arbitrary point on the plane, and the z-axis is consistent with the normal vector of the plane.

[0067] For example, point features are represented as:

[0068] P = (x, y, z) T (1)

[0069] In the formula, (x, y, z) T Three-dimensional spatial coordinates of point features

[0070] The characteristics of a straight line are represented as follows:

[0071] L = [, P T] T (2)

[0072] In the formula, n L ∈R 3 Let P be the unit direction vector of the line, ∈ R. 3 Let be the coordinates of a point on the straight line.

[0073] Planar features are represented as:

[0074] S = [, a] T (3)

[0075] In the formula, n S ∈R 8 Let d be the unit direction vector of the line, d∈R S Let be the coordinates of a point on the straight line.

[0076] In the unified feature representation stage, point features, line features, planar features, and camera pose are all represented uniformly and compactly using the SP model. For any feature element in 3D space (including but not limited to points, lines, and surfaces), it is assumed that each feature element is attached to a coordinate system, such as... Figure 4 As shown. Specifically, for point features, the attached coordinate system has itself as its origin, and the direction of the coordinate system can be freely set; for line features, the origin of the attached coordinate system is located at any point on the line, and the x-axis is arranged along the direction of the line; for planar features, the origin of the attached coordinate system is located at any point on the plane, and the z-axis is in the same direction as the plane's normal vector.

[0077] Specifically, the coordinate system to which the camera is attached is the world coordinate system. The process of determining the six-dimensional pose based on the transformation relationship between the position and orientation of geometric features, and using a self-binding matrix to filter the effective information in the six-dimensional pose vector, may include the following execution process:

[0078] Based on preset translation and rotation formulas, the transformation relationship between the attached coordinate systems corresponding to any two geometric features among point features, line features, surface features and camera position features is determined;

[0079] Among them, the relative positional relationship: the relative positional relationship between two geometric features can be characterized by the transformation between the coordinate systems to which the two features are attached.

[0080] Transformation matrix: The transformation between two coordinate systems A and B consists of translation and rotation, as shown in the following formula:

[0081]

[0082] in, This represents the attitude transformation of coordinate system B relative to coordinate system A, i.e., the transformation from A to B. These transformations together form the complete transformation matrix from coordinate system A to coordinate system B, which describes the position and attitude of coordinate system B relative to coordinate system A.

[0083] This application uses the world coordinate system as a basis. The conversion of the visual camera's coordinate system to the world coordinate system is performed using the previous formula. The local coordinate systems of point, line, and surface features are first converted to the camera coordinate system, and then converted to the world coordinate system.

[0084] The following describes the methods for transforming between coordinate systems:

[0085] For the transition from the local coordinate system to the camera coordinate system:

[0086] Suppose we have a local coordinate system L with geometric features. To transform it to the camera coordinate system C, we can use the following transformation matrix:

[0087]

[0088] Where, Trans(x) L ,y L ,z L ) indicates a translation operation along the coordinate axis.

[0089] Rot z ( L Rot y ( L Rot x ( L ) represent rotation operations around the z, y, and x axes, respectively.

[0090] For the transition from the camera coordinate system to the world coordinate system:

[0091] To transform a point in the camera coordinate system C to the world coordinate system W, the camera's pose information (including translation and rotation) is required:

[0092]

[0093] Where, Trans(x) C ,y C ,z C ) represents the translation of the camera relative to the world coordinate system.

[0094] Rot z ( C Rot y ( C Rot x ( C ) represent the rotation of the camera relative to the world coordinate system.

[0095] For comprehensive conversion:

[0096] To transform the local coordinate system of a geometric feature to the world coordinate system, the two steps described above can be combined.

[0097]

[0098] The transformation matrix between coordinate systems is determined based on the transformation relationship between them, and the six-dimensional attitude vector is determined based on the transformation matrix.

[0099] Based on point features, line features, surface features, and their respective symmetry properties, determine the first to third self-binding matrices for each of the point features, line features, and surface features, and determine the corresponding fourth self-binding feature based on the camera's pose features.

[0100] The filtered valid information is obtained by multiplying the first to fourth self-binding matrices with the six-dimensional attitude vector.

[0101] For example, in the SP model, the self-binding matrix is ​​used to filter valid information from the six-dimensional pose vector and exclude uncertain information. The self-binding matrix is ​​an n×6 row selection matrix, constructed by selecting the corresponding rows in the identity matrix I6 that are not contained in the symmetric transformation subgroup of the feature, where n≤6. For instance, for planar features, due to their inherent symmetry, their pose in three-dimensional space remains unchanged even after specific transformations, although their six-dimensional pose vector may change. Therefore, based on the symmetric properties of geometric features, the first self-binding matrix for point feature P, the second self-binding matrix for line feature L, and the third self-binding matrix for planar feature S can be determined respectively.

[0102]

[0103] Similarly, the fourth self-binding matrix B corresponds to the coordinate system to which camera C on the mobile robot is attached. C Defined as:

[0104]

[0105] Combine the first to fourth self-binding matrices B with the attitude vector Multiplying these yields effective attitude information of the geometric features in space.

[0106] The effective pose information of the point features is as follows:

[0107]

[0108] In the formula, (x, y, z) T The coordinates of the point feature are in three-dimensional space.

[0109] The effective pose information of the line feature is:

[0110]

[0111] In the formula, y and z are the coordinate values ​​from the origin O(0, 0, 0) of the coordinate system to the perpendicular point D(xd, yd, zd) of the line feature, i.e., y = yd, z = zd; θ, These are the direction angles of the line features.

[0112] The effective pose information of the planar feature is:

[0113]

[0114] In the formula, z is the coordinate value from the origin O(0, 0, 0) of the world coordinate system to the perpendicular point D(xd, yd, zd) of the planar feature, i.e., z = zd; θ, These are the direction angles of the normal vectors of the planar features.

[0115] S30. Based on the features of the unified representation, perform feature association, generate feature descriptors, and match feature correspondences between consecutive image frames;

[0116] In one embodiment of this application, the execution process of S30 can be as follows:

[0117] The ORB algorithm is used to detect key points in RGB images. The FAST algorithm is used to quickly find corner points or salient points in RGB images as key points. The BRIEF algorithm is used to calculate a binary descriptor for each key point. The binary descriptor is combined with the location, scale and orientation information of the key point to form a complete point feature descriptor.

[0118] Three-dimensional point cloud data is obtained from depth images. After processing the three-dimensional point cloud data using the RANSAC algorithm, a set of points is randomly selected as samples and a straight line is fitted. The distance from the remaining points to the straight line is calculated. Points with a distance less than a set threshold are determined to belong to the same straight line. After repeating this process multiple times, the model containing the most interior points is selected as the final line feature model. Line feature descriptors are generated based on the endpoint positions and direction vectors of the line feature model.

[0119] 3D point cloud data is obtained from depth images. The RANSAC algorithm is used to process the 3D point cloud data. A set of points is randomly selected as samples and fitted to a plane. The distance from the remaining points to the plane is calculated. Points with a distance less than a set threshold are determined to belong to the same plane. After repeating this process multiple times, the model containing the most interior points is selected as the final planar feature model. A surface feature descriptor is generated based on the normal vector of the planar feature model and the position information of a point on the plane.

[0120] Based on the generated point feature descriptors, line feature descriptors, and area feature descriptors, the correspondence between consecutive image frames is matched.

[0121] The process of generating point feature descriptors can be as follows:

[0122] Keypoint extraction: Keypoints are detected from RGB images using the ORB (Oriented FAST and Rotated BRIEF) algorithm. The FAST algorithm is used to quickly find corners or salient points in an image as keypoints.

[0123] Descriptor Calculation: For each keypoint, its descriptor is calculated using the BRIEF (Binary Robust Independent Elementary Features) algorithm. BRIEF is a binary descriptor that generates a binary string by comparing a set of pixel pairs around the keypoint.

[0124] Constructing the descriptor: The calculated descriptor is combined with the location information (such as x, y coordinates), scale information, and orientation information of key points to form a complete descriptor. This is done to provide more comprehensive information, enabling more accurate identification of corresponding features in subsequent matching processes.

[0125] The process of generating line feature descriptors can be as follows:

[0126] Extracting 3D point cloud data: Obtaining 3D point cloud data from depth images.

[0127] Fitting a straight line model: The RANSAC (Random Sample Consensus) algorithm is used to process the 3D point cloud data. A set of points is randomly selected as samples, and a straight line is fitted. The distance from the remaining points to this straight line is calculated. If the distance is less than a set threshold, these points are considered to belong to the same straight line. The above process is repeated multiple times, and the model containing the most interior points is selected as the final line feature model.

[0128] Calculate line feature parameters: For each line feature, a descriptor can be generated based on its endpoint positions, direction vector, and other information. For example, it can be represented as: direction vector n L =(a L b L c L This can be transformed to the world coordinate system using a rotation matrix. It contains the coordinates of endpoints or the position information of any point on the line.

[0129] Generating a descriptor: This involves combining the line's direction vector with the coordinates of the line segment's endpoints (or other key points) to form a complete descriptor. This typically includes the line's direction vector and the coordinates of some specific points to facilitate subsequent matching operations.

[0130] The process of generating surface feature descriptors can be as follows:

[0131] Extracting 3D point cloud data: Similarly, 3D point cloud data is obtained from the depth image. Fitting a planar model: The RANSAC algorithm is used to process the 3D point cloud data. A set of points is randomly selected as samples, and a plane is fitted. The distance from the remaining points to this plane is calculated. If the distance is less than a set threshold, these points are considered to belong to the same plane. The above process is repeated multiple times, and the model containing the most interior points is selected as the final planar feature model.

[0132] Calculating planar feature parameters: Each planar feature can be described by its normal vector and the position information of a point on the plane. For example, it can be represented as: normal vector n S =(a S , b S c S The position information D(x) of a point on the plane can be transformed to the world coordinate system using a rotation matrix. d y d , z d ).

[0133] Descriptor generation: This involves combining the plane's normal vector with the position information of a specific point on the plane to form a complete descriptor. This typically includes the plane's normal vector and the coordinates of a reference point to facilitate subsequent matching operations.

[0134] It's important to note that feature matching is a preliminary matching operation based on the specific numerical values ​​of feature descriptors. It first requires generating feature descriptors and calculating their similarity (e.g., Euclidean distance) to determine which features might correspond to the same physical object. During the feature matching phase, the system has already identified which features might correspond. This provides the necessary foundational data for subsequent cross-correlation. For example, after point feature matching, it's known which points might correspond to points in the same physical location; the same logic applies to line and planar features. In the cross-correlation phase, the system relies not only on the similarity of feature descriptors but also on the geometric relationships between features (e.g., points on lines, points on planes, etc.). This correlation enhances the system's robustness and accuracy, but only if a basic correspondence has been established. During the cross-correlation process, the system comprehensively considers multiple features and their interrelationships to construct a more complete environmental model. This multi-level correlation helps eliminate false matches and improves the overall estimation accuracy. However, this process depends on the results of the initial feature matching already completed.

[0135] S40. Collect UAV motion data through IMU, and dynamically select the integration method and adjust the time step according to the current motion state and environmental conditions. Pre-integrate the IMU-collected data to obtain the integration result, in order to estimate displacement, velocity, and rotation.

[0136] The process of acquiring UAV motion data via IMU and dynamically selecting the integration method and adjusting the time step based on the current motion state and environmental conditions may include the following execution steps:

[0137] Based on the data collected by the IMU, the angular velocity and acceleration of the drone, as well as the degree of change in the surrounding environment, are determined to determine whether it is a high-speed motion environment, a rapidly changing environment, or a stable environment.

[0138] In high-speed motion and rapidly changing environments, the data collected by the IMU is pre-integrated using the preset first step length and third-order Adams-Bashforth algorithm.

[0139] If the environment is stable, the data collected by the IMU is pre-integrated using a preset second step size and a first-order Euler algorithm.

[0140] In this case, the length of the first step is less than the length of the second step.

[0141] It should be noted that in the rapidly changing environment of drones, using a camera alone can easily result in insufficient feature information, while IMUs suffer from drift errors. Therefore, it is necessary to combine the advantages of both cameras and IMUs to achieve more accurate and robust mapping.

[0142] Here, we first review classic IMU pre-integration methods, which typically refer to IMU pre-integration based on the first-order Euler method. The first-order Euler method is a simple numerical integration technique that uses the derivative value at the current time step to predict the value at the next time step. In specific applications of IMU pre-integration, the first-order Euler method is mainly used to estimate the displacement, velocity, and rotation increments of the IMU.

[0143] The speed increment is expressed as follows:

[0144]

[0145] In the formula Δv ij From time t i to t j Speed ​​increment, R k It is the attitude rotation matrix at the k-th sampling time, a k b is the acceleration measurement value at the k-th sampling time. a It is the deviation of the accelerometer, Δt k It is the time interval of the kth sampling moment.

[0146] The displacement increment is expressed as follows:

[0147]

[0148] (where Δp) ij From time t i to t j Position increment, v k It is the velocity at the k-th sampling time.

[0149] The rotation increment is represented as follows:

[0150]

[0151] In the formula ΔR ij From time t i to t j The rotation increment, ω k It is the angular velocity measurement value at the k-th sampling time, b g It's a deviation in the gyroscope.

[0152] This application represents an improvement on IMU pre-integration based on Adams-Bashforth. Specifically, to address the error accumulation problem arising from the rapidly changing environment of UAVs, this application proposes to employ a third-order Adams-Bashforth method for IMU pre-integration. The third-order Adams-Bashforth method is an explicit multi-step method that predicts the value of the next time step using three previous time steps. In IMU pre-integration, the third-order Adams-Bashforth method is used to estimate the displacement, velocity, and rotation increments of the IMU, thereby significantly improving the accuracy and stability of pre-integration, reducing error accumulation, and further improving positioning accuracy.

[0153] In another alternative embodiment of this application, the method selected for IMU pre-integration in the case of severe drone scenarios is based on the Adams-Bashforth method. However, the method based on the fourth-order Runge-Kutta method can also be used to replace the selected scheme and achieve the same effect.

[0154] For example, the speed increment is represented as follows:

[0155]

[0156] In the formula Δv ij From time t i to t j Speed ​​increment, R k It is the attitude rotation matrix at the k-th sampling time, a k b is the acceleration measurement value at the k-th sampling time. a It is the deviation of the accelerometer, Δt k It is the time interval of the kth sampling moment.

[0157] The displacement increment is expressed as follows:

[0158]

[0159] In the formula Δp ij From time t i to t j Position increment, v k It is the velocity at the k-th sampling time.

[0160] The rotation increment is represented as follows:

[0161]

[0162] In the formula ΔR ij From time t i to t j The rotation increment, ω kIt is the angular velocity measurement value at the k-th sampling time, b g It's a deviation in the gyroscope.

[0163] By improving the IMU pre-integration using the third-order Adams-Bashforth method, the local truncation error of the third-order Adams-Bashforth method is O(h). 4 The global error is O(h). 3 The local truncation error of the first-order Euler method is O(h), and the global error is O(h). This means that, for the same time step, the third-order Adams-Bashforth method has higher accuracy in velocity increments. Similarly, the third-order Adams-Bashforth method also has higher accuracy in position and rotation increments.

[0164] It should also be noted that the first-order Euler method, among the strategies for dynamically adjusting the time step and integration method, is widely used in practical applications due to its computational simplicity and strong real-time performance, making it particularly suitable for resource-constrained embedded systems. However, this method has lower accuracy in highly dynamic environments and is prone to large error accumulation. In contrast, the third-order Adams-Bashforth method performs excellently in highly volatile environments due to its high accuracy and good stability, making it particularly suitable for handling complex motion states. Although its computational complexity is slightly higher than the first-order Euler method, it can more accurately capture the nonlinear characteristics of motion, thereby significantly reducing error accumulation.

[0165] To balance system robustness and computational complexity, this application proposes a strategy for dynamically adjusting the time step and integration method. For example, in high-speed motion or rapidly changing environments, a shorter time step and a third-order Adams-Bashforth method can be chosen for pre-integration to improve positioning accuracy; while in stable motion or simple environments, a longer time step and a first-order Euler method can be chosen to improve computational efficiency. This dynamic adjustment strategy not only improves the accuracy of pre-integration but also enhances the system's robustness. Its workflow diagram is shown below. Figure 5 As shown:

[0166] First, the system assesses the current motion state and environmental conditions, including the drone's speed, acceleration, and the degree of change in the surrounding environment. Based on these assessments, the system selects the most suitable pre-integration method: in high-speed motion or rapidly changing environments, it tends to use more accurate but computationally complex higher-order methods (such as the third-order Adams-Bashforth method); while in stable environments, it chooses the simple and fast first-order Euler method to save resources.

[0167] The system then dynamically adjusts the pre-integration time step. When higher precision is required, the time step is shortened; conversely, when lower precision is needed, the time step is lengthened. This flexible time step adjustment mechanism helps balance computational efficiency and positioning accuracy, ensuring optimal performance under various conditions.

[0168] After selecting the pre-integration method and time step, the system performs pre-integration processing on the IMU data. This step involves estimating displacement, velocity, and rotation increments, and calculating corresponding statistical information such as the covariance matrix. This provides a solid foundation for subsequent data fusion and optimization. The pre-integration results are then used to assist visual odometry data processing, such as addressing the feature point sparsity problem caused by motion blur, or as an additional constraint in back-end nonlinear optimization, thereby reducing error accumulation and improving overall positioning accuracy.

[0169] The entire process forms a closed-loop feedback system. Through continuous loop closure detection and other correction mechanisms, the system can further refine the trajectory estimation, ensuring robustness and accuracy over long-term operation. Each iteration optimizes parameter settings based on the latest motion state and environmental conditions, continuously improving system performance and ensuring that the UAV maintains efficient and stable autonomous navigation capabilities in complex and ever-changing environments.

[0170] In summary, this application employs an advanced multi-feature fusion feature matching algorithm, which can extract various types of features, thereby significantly increasing the number of features available for pose estimation. By introducing additional geometric constraints, this method not only improves the accuracy and robustness of feature matching but also effectively reduces errors in pose estimation. This method is particularly suitable for localization problems in complex environments, enhancing system stability while maintaining high accuracy. Addressing the specific needs of UAVs in rapidly changing scenarios, this application designs an innovative dynamically adjusted IMU pre-integration algorithm. This strategy dynamically adjusts the IMU pre-integration process by calculating the rate of change of UAV acceleration and angular velocity in real time.

[0171] The second embodiment of this application provides an improved visual inertial odometry positioning device, comprising: a feature extraction module, used to acquire image data through a UAV camera and extract point features, line features, and planar features from the image data using a hybrid feature extraction method; an effective information acquisition module, used to uniformly represent the point features, line features, planar features, and acquired camera pose using a unified representation model, wherein the unified representation model includes setting an attached coordinate system for each feature element, describing the position and orientation of geometric features through the transformation from the world coordinate system to the local coordinate system, determining a six-dimensional pose based on the transformation relationship between the position and orientation of geometric features, and using a self-binding matrix to filter effective information in the six-dimensional pose vector; a feature correspondence determination module, used to perform feature association based on the features of the unified representation, generate feature descriptors, and match feature correspondences between consecutive image frames; an incremental estimation module, used to acquire UAV motion data through an IMU, and dynamically select an integration method and adjust the time step according to the current motion state and environmental conditions, pre-integrate the IMU-acquired data to obtain an integration result, so as to estimate displacement, velocity, and rotation increments; and a positioning data generation module, used to fuse the feature correspondences and pre-integration results, perform pose estimation, and generate positioning data generated by the UAV.

[0172] It is not difficult to see that this embodiment is a system embodiment corresponding to the above method embodiments, and this embodiment can be implemented in conjunction with the above method embodiments. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0173] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent from this embodiment.

[0174] A third embodiment of this application also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method for improving the positioning accuracy of a visual inertial odometry provided in the above embodiments.

[0175] The fourth embodiment of this application also provides an electronic device, the electronic device comprising: at least one processor, a memory, and an input / output unit; wherein, the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the method for improving the positioning accuracy of a visual inertial odometry provided in any of the foregoing embodiments.

[0176] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and the memory. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0177] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Magnetic memory can be used to store data used by the processor during operation.

[0178] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0179] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, ROM (Read-Only Memory), RAM (Random Access Memory), a magnetic disk, or an optical disk.

[0180] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for improving the positioning accuracy of a visual inertial odometry system, characterized in that, include: Image data is acquired using a drone camera, and point features, line features, and planar features are extracted from the image data using a hybrid feature extraction method. A unified representation model is used to represent the point features, line features, planar features, and the acquired camera pose. The unified representation model includes setting an attached coordinate system for each feature element, describing the position and orientation of the geometric features through the transformation from the world coordinate system to the local coordinate system, determining the six-dimensional pose based on the transformation relationship between the position and orientation of the geometric features, and using a self-binding matrix to filter the effective information in the six-dimensional pose vector. Based on the features of the unified representation, feature association is performed to generate feature descriptors and match feature correspondences between consecutive image frames. The IMU collects motion data of the UAV, and the integration method and time step are dynamically selected and adjusted according to the current motion state and environmental conditions. The IMU-collected data is pre-integrated to obtain the integration result, so as to estimate the displacement, velocity and rotation increment. By fusing the feature correspondence and pre-integration results, pose estimation is performed and positioning data of the UAV is generated.

2. The method for improving the positioning accuracy of a visual inertial odometry system as described in claim 1, characterized in that, The method of extracting point features, line features, and planar features from the image data using a hybrid feature extraction method includes: The FAST algorithm is used to extract key points from image data, and the key points are represented by binary robust independent elementary features to obtain point feature descriptors. Simultaneously, the 3D point cloud of the input image data is downsampled, and the RANSAC method is used to continuously fit line features and planar features until the number of remaining 3D points is reduced to below the preset value of the original point cloud, thus obtaining line features and planar features.

3. The method for improving the positioning accuracy of a visual inertial odometry system as described in claim 1, characterized in that, The step of setting an attached coordinate system for each feature element includes: The point feature has itself as the origin, and the direction of the coordinate system can be freely set; The origin of the coordinate system for the linear feature is any point on the straight line, and the x-axis is arranged along the direction of the straight line. The origin of the plane characteristic coordinate system is located at any arbitrary point on the plane, and the z-axis is consistent with the normal vector of the plane.

4. The method for improving the positioning accuracy of a visual inertial odometry system as described in claim 1, characterized in that, The camera is attached to the world coordinate system. The process of determining the six-dimensional pose based on the transformation relationships between geometric features in terms of position and orientation, and using a self-binding matrix to filter effective information from the six-dimensional pose vector, includes: Based on preset translation and rotation formulas, the transformation relationship between the attached coordinate systems corresponding to any two geometric features among point features, line features, surface features and camera position features is determined; The transformation matrix between coordinate systems is determined based on the transformation relationship between them, and the six-dimensional attitude vector is determined based on the transformation matrix. Based on point features, line features, surface features, and their respective symmetry properties, determine the first to third self-binding matrices for each of the point features, line features, and surface features, and determine the corresponding fourth self-binding feature based on the camera's pose features. The filtered valid information is obtained by multiplying the first to fourth self-binding matrices with the six-dimensional attitude vector.

5. The method for improving the positioning accuracy of a visual inertial odometry system as described in claim 1, characterized in that, The step of performing feature association based on the unified representation, generating feature descriptors, and matching feature correspondences between consecutive image frames includes: The ORB algorithm is used to detect key points in RGB images. The FAST algorithm is used to quickly find corner points or salient points in RGB images as key points. The BRIEF algorithm is used to calculate a binary descriptor for each key point. The binary descriptor is combined with the location, scale and orientation information of the key point to form a complete point feature descriptor. Three-dimensional point cloud data is obtained from depth images. After processing the three-dimensional point cloud data using the RANSAC algorithm, a set of points is randomly selected as samples and a straight line is fitted. The distance from the remaining points to the straight line is calculated. Points with a distance less than a set threshold are determined to belong to the same straight line. After repeating this process multiple times, the model containing the most interior points is selected as the final line feature model. Line feature descriptors are generated based on the endpoint positions and direction vectors of the line feature model. 3D point cloud data is obtained from depth images. The RANSAC algorithm is used to process the 3D point cloud data. A set of points is randomly selected as samples and fitted to a plane. The distance from the remaining points to the plane is calculated. Points with a distance less than a set threshold are determined to belong to the same plane. After repeating this process multiple times, the model containing the most interior points is selected as the final planar feature model. A surface feature descriptor is generated based on the normal vector of the planar feature model and the position information of a point on the plane. Based on the generated point feature descriptors, line feature descriptors, and area feature descriptors, the correspondence between consecutive image frames is matched.

6. The method for improving the positioning accuracy of a visual inertial odometry system as described in claim 1, characterized in that, The process of acquiring UAV motion data via IMU and dynamically selecting the integration method and adjusting the time step based on the current motion state and environmental conditions includes: Based on the data collected by the IMU, the angular velocity and acceleration of the drone, as well as the degree of change in the surrounding environment, are determined to determine whether it is a high-speed motion environment, a rapidly changing environment, or a stable environment. In high-speed motion and rapidly changing environments, the data collected by the IMU is pre-integrated using the preset first step length and third-order Adams-Bashforth algorithm. If the environment is stable, the data collected by the IMU is pre-integrated using a preset second step size and a first-order Euler algorithm. In this case, the length of the first step is less than the length of the second step.

7. The method for improving the positioning accuracy of a visual inertial odometry as described in claim 1, characterized in that, The pre-integration of the IMU-acquired data to obtain the integration result for estimating displacement, velocity, and rotation increments includes: The third-order Adams-Bashforth algorithm is used to pre-integrate the IMU-acquired data to obtain the velocity increment, displacement increment, and rotation increment, respectively. The pre-integral formula for the velocity increment is: In the formula, Δv ij From time t i to t j Speed ​​increment, R k It is the attitude rotation matrix at the k-th sampling time, a k b is the acceleration measurement value at the k-th sampling time. a It is the deviation of the accelerometer, Δt k It is the time interval of the kth sampling moment; The pre-integral formula for the displacement increment is: In the formula, Δp ij From time t i to t j Position increment, v k It is the velocity at the k-th sampling time. The pre-integral formula for the rotation increment is: In the formula, ΔR ij From time t i to t j The rotation increment, ω k It is the angular velocity measurement value at the k-th sampling time, b g It's a deviation in the gyroscope.

8. A device for improving visual inertial odometry positioning, characterized in that, include: The feature extraction module is used to acquire image data through a drone camera and extract point features, line features, and planar features from the image data using a hybrid feature extraction method. The effective information acquisition module is used to uniformly represent the point features, line features, planar features and the acquired camera pose using a unified representation model. The unified representation model includes setting an attached coordinate system for each feature element, describing the position and orientation of the geometric features through the transformation from the world coordinate system to the local coordinate system, determining the six-dimensional pose based on the transformation relationship between the position and orientation of the geometric features, and using a self-binding matrix to filter the effective information in the six-dimensional pose vector. The feature correspondence determination module is used to perform feature association based on the features of the unified representation, generate feature descriptors, and match feature correspondences between consecutive image frames. The incremental estimation module is used to collect UAV motion data through IMU, and dynamically select the integration method and adjust the time step according to the current motion state and environmental conditions. It pre-integrates the IMU-collected data to obtain the integration result, so as to estimate the displacement, velocity and rotation increments. The positioning data generation module is used to fuse the feature correspondence and pre-integration results to perform pose estimation and generate positioning data for the UAV.

9. A computer-readable storage medium, characterized in that, It includes instructions that, when executed on a computer, cause the computer to perform the method for improving the positioning accuracy of a visual inertial odometry as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The electronic device includes: At least one processor, memory, and input / output unit; The memory is used to store computer programs, and the processor is used to call the computer programs stored in the memory to execute the method for improving the positioning accuracy of visual inertial odometry according to any one of claims 1 to 7.

Images