State estimation method based on hybrid feature tracking and adaptive key frame selection

Abstract

The application discloses a state estimation method based on mixed features and adaptive key frame selection, and belongs to the field of autonomous navigation of unmanned systems.The application is realized by the following steps: collecting multiple images, synchronously acquiring inertial measurement data of a six-axis IMU;according to the inertial measurement data, performing key frame decision on a current frame image, and classifying the image into a key frame image and a non-key frame image;acquiring a key frame feature matching set through an entropy maximization self-supervised feature extractor and a graph attention matcher, and constructing a visual re-projection error item based on triangulated map points;adopting an improved corner detector to extract non-key frame feature points, obtaining feature displacement through IMU constraint optical flow tracking, and generating a visual re-projection error and an IMU pre-integration residual error item;based on a visual-inertial tight coupling optimization module, solving a system, combining the visual re-projection error, the IMU pre-integration residual error and an aerodynamic constraint residual error to construct a tight coupling optimization problem, and solving an optimal pose state.The application is helpful for navigation of small-sized high-speed unmanned aerial vehicles.

Description

Technical Field

[0001] This invention belongs to the field of autonomous navigation technology for unmanned systems, specifically relating to a state estimation method based on hybrid feature tracking and adaptive keyframe selection. Background Technology

[0002] Visual inertial odometry (VIO), a core technology for autonomous navigation of small high-speed unmanned aerial vehicles (UAVs), faces severe challenges in high-speed penetration and precision strike missions. Traditional solutions such as ORB-SLAM3 and VINS-Mono rely on manually designed geometric features (e.g., ORB, FAST) and descriptor matching. In high-speed motion scenarios exceeding 30 m / s for unmanned systems, the reliability of feature matching significantly decreases due to severe motion blur and illumination changes. Although direct methods (e.g., DSO, VI-DSO) improve adaptability to low-texture environments by minimizing photometric errors, their computational cost is high and they are highly sensitive to motion blur, making it difficult to meet the real-time requirements of embedded platforms for high-speed unmanned systems. In recent years, deep learning solutions (e.g., SuperPoint, SuperGlue) have shown advantages in feature rotation and scale changes, but SuperGlue's graph attention mechanism has high computational complexity, and LightGlue's adaptive pruning still cannot effectively solve the real-time processing problem under high-speed motion. Embedded optimization solutions such as MobileVIO suffer from drastic performance degradation in low-texture environments and motion-blurred scenes, while neural radiation field methods (such as NeRF-SLAM) cannot be practically deployed due to excessive GPU resource requirements.

[0003] Existing technologies suffer from three core contradictions: First, traditional feature extraction methods suffer from a 70% drop in feature extraction quantity due to image blurring during the dive phase of small, high-speed UAVs, leading to tracking interruptions. Second, deep learning methods exceed the 25W power consumption limit during continuous operation on embedded platforms such as Jetson Xavier NX, directly conflicting with the stringent power consumption requirements of small, high-speed UAVs. Third, fixed keyframe strategies suffer from a 42% misjudgment rate in high-maneuver scenarios above 3g and lack the ability to quantitatively assess the uncertainty of IMU motion, resulting in positioning errors being amplified by more than three times in vibration environments. Therefore, an innovative architecture is urgently needed that can guarantee sub-meter-level positioning accuracy in high-speed scenarios of 30m / s while meeting the real-time and low-power requirements of embedded platforms. Summary of the Invention

[0004] To address the issue of inaccurate pose estimation caused by the impact of positioning robustness on high-speed mobile platforms under conditions of intense maneuvering, complex battlefield environments, and resource constraints, this invention aims to provide a state estimation method based on hybrid feature tracking and adaptive keyframe selection. It integrates the advantages of deep features and optical flow tracking: keyframes employ entropy-maximizing self-supervised feature extraction and graph attention-based feature matching for high-precision tracking to ensure pose estimation accuracy; non-keyframes achieve millisecond-level tracking through IMU-constrained corner detection and optical flow matching. More significantly, it introduces IMU pre-integration covariance analysis into keyframe decision-making, constructing an adaptive processing mechanism driven by motion uncertainty. Unlike existing single-path methods, this invention deeply couples adaptive computational load allocation with the physical motion model, providing a novel technological paradigm for navigation systems of small high-speed unmanned aerial vehicles.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] The state estimation method based on hybrid feature tracking and adaptive keyframe selection disclosed in this invention includes the following steps:

[0007] S1. Acquire multiple images using an industrial camera and simultaneously obtain inertial measurement data from a six-axis IMU.

[0008] S2. Based on the inertial measurement data, make keyframe decisions for the current frame image and classify the image into keyframe images and non-keyframe images.

[0009] Keyframes and non-keyframes are distinguished using equation (1):

[0010]

[0011] Where Δp is the IMU pre-integration translation, θ(ΔR) is the IMU pre-integration rotation angle, and Σ IMU' Let τ be the enhanced pre-integral covariance matrix after vibration compensation. p τ is the displacement threshold, which is the minimum displacement change that triggers keyframe processing. θ τ is the rotation threshold, which is the minimum rotation angle that triggers keyframe processing. Σ The uncertain threshold is Σ IMU The trace upper limit is ||·||, which is the norm operation, i.e., Euclidean distance, and tr(·) is the trace operation of the matrix, i.e., the sum of the diagonal elements of the square is used to quantify the total uncertainty of motion estimation.

[0012] The τ p ,τ θ ,τ Σ Dynamically adjust based on computing platform load:

[0013] τ p =τ p0×(1+αη GPU (2)

[0014] τ θ =τ θ0 ×(1+βη CPU (3)

[0015] τ Σ =τ Σ0 ·(1+γη MEM (4)

[0016] Where, η GPU ∈[0,1] represents the real-time GPU utilization, η CPU ∈[0,1] represents the CPU real-time utilization, η MEM ∈[0,1] represents memory usage, and α,β,γ∈[0.1,0.5] are adjustable coefficients.

[0017] The IMU pre-integration covariance matrix Σ IMU Calculated using the error state propagation equation:

[0018]

[0019] Where Σ k That is Σ IMU In the intermediate states of the iterative calculation process, F is the state transition matrix, G is the noise Jacobian matrix, Q is the IMU noise covariance, and k is the time series.

[0020] S3. Keyframe processing: Obtain keyframe feature matching set through entropy maximization self-supervised feature extractor and graph attention matcher, and construct visual reprojection error term based on triangulated map points.

[0021] Feature points are extracted from keyframe image I using an entropy-maximizing self-supervised feature extractor to generate a feature point set:

[0022]

[0023] Where p(x) i ) represents the pixel coordinates of the feature point, τ sp =0.015 is the probability threshold.

[0024] The entropy-maximizing self-supervised feature extractor is as follows:

[0025]

[0026] Where ι = 0.5 is the entropy weighting coefficient. Let p(x) be the entropy information objective function, representing the pixel position. The probability of a feature point. This represents the original detection loss.

[0027] The feature matcher using the graph attention mechanism calculates the matching relationships between feature points:

[0028]

[0029] Where α ij This is an attention mechanism, where i represents the i-th feature point extracted in the current keyframe, and j represents the j-th feature point extracted in the previous keyframe. For querying feature vectors, Query Let d be the key feature vector, and d be the feature dimension. This is the feature matching set.

[0030] The matcher employs an attention mechanism:

[0031]

[0032] in These are the query feature vector Query for the i-th feature point in the reference image with feature dimension d = 256 and the key feature vector Key for the j-th feature point in the target image, respectively. They learn cross-frame feature correlations through the Transformer architecture. This is a scaling factor to prevent the gradient from vanishing due to excessively large dot product values.

[0033] For the matching set For each pair of matched feature points (i,j), combined with their corresponding historical keyframe poses, their corresponding 3D map points are calculated using a triangulation method. The coordinates in the world coordinate system are obtained using the feature pixel coordinates p of the current frame. i Visual reprojection error term r constructed with 3D map points vis :

[0034]

[0035] Where π(·) is the camera projection model, and T is the pose of the current keyframe relative to the world coordinate system.

[0036] S4. Non-key frame processing: An improved corner detector is used to extract feature points of non-key frames, and feature displacement is obtained by IMU-constrained optical flow tracking. Visual reprojection error and IMU pre-integration residual terms are generated.

[0037] An improved corner detector is used to extract feature points Score(x,y) from the non-keyframe image I′:

[0038] Score(x,y)=min(λ1,λ2)>τ GFTT (13)

[0039] Where λ1 and λ2 are the eigenvalues ​​of the autocorrelation matrix, representing the intensity of gray-level changes in the pixel neighborhood, and τ GFTT is the threshold for the feature point response, and Score(·) is the corner response function.

[0040] IMU-constrained optical flow tracing is performed on the feature point Score(x,y) to balance the weights of visual consistency and IMU priors:

[0041]

[0042] Where Δx and Δy are the horizontal and vertical displacement components of the feature point between adjacent frames, and Δp is the amount of movement of the feature point between adjacent frames on the image plane. IMU I represents the image displacement that the same feature point, inferred from IMU pre-integration data, should appear between two frames. t (p) represents the gray value at position p in the image at time t, I t+1 Similarly, λ represents the regularization coefficient. (p+△p)

[0043] The visual reprojection error term r of the non-key frame is obtained through equation (12). vis The pose increment Δp obtained by IMU pre-integration IMU and its covariance Σ IMU Transformed into IMU pre-integrated residual term r IMU .

[0044] S5. Solve the system based on the vision-inertial tightly coupled optimization module. Combine the visual reprojection error, IMU pre-integration residual and aerodynamic constraint residual to construct a tightly coupled optimization problem and solve for the optimal pose state of the system.

[0045] By combining visual reprojection error with IMU pre-integration residuals, a lightweight maximum a posteriori estimation problem is constructed:

[0046] T * =argmin T (∑||r vis || 2 +||r IMU || 2 +||r aero || 2 (15)

[0047] Where argmin T (·) represents the value of T that minimizes the objective function, r aero State constraints for integrating aerodynamics:

[0048] r aero =||v IMU-v wind-tunnel || 2 (16)

[0049] Where v IMU For IMU to measure velocity, v wind-tunnel This is the theoretical speed in the wind tunnel.

[0050] The visual inertial odometry system employed in this invention consists of five core modules working collaboratively: The image acquisition module uses a global shutter camera and employs a hardware synchronization mechanism to eliminate image distortion caused by high-speed motion, ensuring geometric consistency in feature extraction. The inertial measurement module integrates a six-axis IMU sensor, providing high-frequency angular velocity and acceleration measurements to construct an inertial reference for motion state estimation. The adaptive decision module includes a motion uncertainty analysis unit, which dynamically adjusts keyframe triggering conditions by real-time fusion of IMU pre-integrated covariance and computational platform load status. The hybrid processing module implements a differentiated processing architecture for keyframes and non-keyframes. The keyframe path uses deep learning feature extraction and a matcher to achieve sub-pixel-level matching accuracy, while the non-keyframe path uses traditional optical flow algorithms to achieve millisecond-level efficient tracking. The two paths exchange feature state data through shared memory. The tightly coupled optimization module uses a sliding window optimization framework, combining visual reprojection constraints, inertial measurement constraints, and platform dynamics constraints to output a six-degree-of-freedom pose estimate. All modules are interconnected via a high-speed data bus, forming a closed-loop processing link that meets the stringent real-time and power consumption requirements of embedded platforms while ensuring positioning accuracy.

[0051] Beneficial effects:

[0052] This invention achieves multi-dimensional performance breakthroughs in the navigation of small high-speed unmanned aerial vehicles through a hybrid feature tracking and adaptive keyframe selection mechanism:

[0053] 1. Accuracy Level: The state estimation method based on hybrid feature tracking and adaptive keyframe selection disclosed in this invention significantly improves positioning accuracy. By introducing IMU-constrained optical flow technology, it effectively overcomes the feature tracking failure problem caused by high-speed motion, greatly enhancing the stability of pose estimation in complex maneuvering scenarios. Aerodynamic constraint fusion suppresses measurement drift during the dive phase, and combined with a vibration interference compensation model, it significantly improves positioning reliability in battlefield vibration environments.

[0054] 2. Efficiency: The state estimation method based on hybrid feature tracking and adaptive keyframe selection disclosed in this invention comprehensively optimizes computational efficiency. The heterogeneous processing architecture decouples deep learning feature extraction and traditional optical flow tracking in the spatiotemporal dimension, enabling intelligent allocation of computational load. The dynamic resource management mechanism adaptively adjusts the processing strategy based on the platform's real-time state, optimizing computational resource utilization while ensuring accuracy, thus meeting the demands of high frame rate real-time processing.

[0055] 3. Adaptability: The state estimation method disclosed in this invention, based on hybrid feature tracking and adaptive keyframe selection, significantly enhances environmental adaptability. It utilizes a covariance-driven decision-making mechanism based on motion uncertainty to achieve intelligent perception and dynamic adjustment triggered by keyframes. The system supports seamless switching between multiple task modes such as cruise and dive, and has built-in rapid fault recovery capabilities to ensure robust continuous operation under extreme conditions.

[0056] 4. Resource Level: The state estimation method based on hybrid feature tracking and adaptive keyframe selection disclosed in this invention achieves a breakthrough improvement in resource efficiency. Feature compression technology effectively reduces storage and transmission overhead, and combined with power consumption optimization design, a high-performance computing closed loop is built on an embedded platform. While ensuring sub-meter level positioning accuracy, the system significantly improves battlefield data exchange efficiency and combat sustainability, and helps high-speed mobile platforms provide reliable navigation. Attached Figure Description

[0057] Figure 1 This is a diagram of the system hardware composition in an embodiment of the present invention.

[0058] Figure 2 This is a flowchart of the method in an embodiment of the present invention.

[0059] Figure 3 This is a diagram showing the effect of feature extraction and matching of key frames and non-key frames in an embodiment of the present invention.

[0060] Figure 4 This is an absolute trajectory error diagram in an embodiment of the present invention.

[0061] Figure 5 This is a trajectory fitting diagram from an embodiment of the present invention. Detailed Implementation

[0062] To better illustrate the purpose and advantages of the present invention, the invention will be further described below in conjunction with the accompanying drawings and examples.

[0063] like Figure 1 As shown, the sensor module consists of a global shutter camera and a six-axis IMU, with the camera frame rate ≥20Hz and the IMU sampling rate ≥200Hz, and both are hardware synchronized. The processing module is deployed on an embedded platform and includes a deep learning unit for performing entropy-maximizing self-supervised feature extraction and graph attention-based matching, a traditional vision unit for performing improved corner detection and IMU-constrained optical flow tracking, and a motion uncertainty decision-making unit based on IMU pre-integration covariance analysis. The optimization module employs a sliding window-based vision-inertial tightly coupled optimizer to minimize the weighted sum of visual reprojection error and IMU pre-integration error.

[0064] Front-end processing is executed on the CPU, including improved corner feature detection, IMU data pre-integration, motion blur compensation, and optical flow tracking. Back-end processing is executed on the GPU, including entropy-maximizing self-supervised feature extraction, graph attention-based feature matching, and covariance matrix calculation. The decision layer dynamically allocates computing resources based on real-time GPU utilization, reducing the keyframe processing frequency when utilization exceeds 80%.

[0065] The camera-IMU rigid connection satisfies the following external parameter calibration error:

[0066] ||e t ||<1mm,||φ e ||<0.1° (17)

[0067] Among them, ||e t || represents the position calibration error between the camera and the origin of the IMU coordinate system, ||φ e || represents the orientation alignment error angle between the camera and the IMU coordinate system.

[0068] like Figure 2 As shown, the three-level collaborative processing architecture designed in this embodiment achieves technological breakthroughs through the following methods: the adaptive decision module integrates IMU motion uncertainty analysis to dynamically adjust the key frame triggering frequency. Key frames employ a deep learning feature extractor to perform high-precision matching, ensuring accurate pose estimation. Non-key frames utilize traditional optical flow algorithms for efficient tracking, significantly reducing computational load. This architecture deeply couples the spatiotemporal characteristics of visual and inertial modes, and by optimizing resource allocation strategies in GPS-denied environments, it provides small, high-speed unmanned aerial vehicles with sub-meter-level pose estimation capabilities that meet real-time requirements, achieving coordinated optimization of accuracy, efficiency, and robustness.

[0069] like Figure 2 As shown in the figure, the state estimation method based on hybrid feature tracking and adaptive keyframe selection disclosed in this example has the following specific implementation steps:

[0070] S1. Acquire multiple images using an industrial camera and simultaneously obtain inertial measurement data from a six-axis IMU.

[0071] S2. Based on the inertial measurement data, make keyframe decisions for the current frame image and classify the image into keyframe images and non-keyframe images.

[0072] Keyframes and non-keyframes are distinguished using equation (18):

[0073]

[0074] Where Δp is the IMU pre-integration translation, θ(ΔR) is the IMU pre-integration rotation angle, and Σ IMU'Let τ be the enhanced pre-integral covariance matrix after vibration compensation. p τ is the displacement threshold, which is the minimum displacement change that triggers keyframe processing. θ τ is the rotation threshold, which is the minimum rotation angle that triggers keyframe processing. Σ The uncertain threshold is Σ IMU The trace upper limit is ||·||, which is the norm operation, i.e., Euclidean distance, and tr(·) is the trace operation of the matrix, i.e., the sum of the diagonal elements of the square is used to quantify the total uncertainty of motion estimation.

[0075] The τ p ,τ θ ,τ Σ Dynamically adjust based on computing platform load:

[0076] τ p =τ p0 ×(1+αη GPU (19)

[0077] τ θ =τ θ0 ×(1+βη CPU (20)

[0078] τ Σ =τ Σ0 ·(1+γη MEM ) (twenty one)

[0079] Where, η GPU ∈[0,1] represents the real-time GPU utilization, η CPU ∈[0,1] represents the CPU real-time utilization, η MEM ∈[0,1] represents memory usage, and α,β,γ∈[0.1,0.5] are adjustable coefficients.

[0080] When the number of consecutive tracking failure frames N fail When the value is greater than 3, keyframe processing is forcibly triggered and τ is reset. p ←0.5τ p .

[0081] The IMU pre-integration covariance matrix Σ IMU Calculated using the error state propagation equation:

[0082]

[0083] Where Σ k That is Σ IMU In the intermediate states of the iterative calculation process, F is the state transition matrix, G is the noise Jacobian matrix, Q is the IMU noise covariance, and k is the time series.

[0084] Hybrid feature tracking is performed under a high-speed motion environment of 30 m / s, in which the dynamic threshold τ is applied during the projectile's dive phase. p The depth was reduced to 0.5m. The IMU pre-integration error term was constrained according to the projectile dynamics model, and an acceleration compensation term caused by aerodynamic drag was added. Keyframe map data with an 80% compression rate was transmitted via an RF link.

[0085] Vibration-resistant unit: Embedding the projectile dynamics model in IMU pre-integration

[0086]

[0087] Where Σ IMU Let J be the standard IMU pre-integral covariance matrix, γ be the engine gain coefficient, and J be the standard IMU pre-integral covariance matrix. vib Q is the vibrational Jacobian matrix. vib Let be the vibration noise covariance matrix.

[0088] Radio frequency map transmission module:

[0089]

[0090] in For the compressed descriptor, d i The original descriptor is ProductQuantization(.), which is the product quantization process, and 8×32bit is the product quantization configuration.

[0091] The displacement threshold τ will be set during the landing phase. p Adjusted to 0.3m, rotation threshold τ θ Adjust to 5°. Initiate multi-scale optical flow pyramid tracking with ≥3 pyramid layers. Integrate barometer altitude information to optimize vertical position estimation; visual inertial odometry weights and barometer weights are adaptively adjusted according to altitude.

[0092] S3. Keyframe processing: Obtain keyframe feature matching set through entropy maximization self-supervised feature extractor and graph attention matcher, and construct visual reprojection error term based on triangulated map points.

[0093] Feature points are extracted from keyframe image I using an entropy-maximizing self-supervised feature extractor to generate a feature point set:

[0094]

[0095] Where p(x) i ) represents the pixel coordinates of the feature point, τ sp =0.015 is the probability threshold.

[0096] The entropy-maximizing self-supervised feature extractor is as follows:

[0097]

[0098] Where ι = 0.5 is the entropy weighting coefficient. Let p(x) be the entropy information objective function, representing the pixel position. The probability of a feature point. This represents the original detection loss.

[0099] The feature matcher using the graph attention mechanism calculates the matching relationships between feature points:

[0100]

[0101] Where α ij This is an attention mechanism, where i represents the i-th feature point extracted in the current keyframe, and j represents the j-th feature point extracted in the previous keyframe. For querying feature vectors, Query Let d be the key feature vector, and d be the feature dimension. This is the feature matching set.

[0102] The matcher employs an attention mechanism:

[0103]

[0104] in These are the query feature vector Query for the i-th feature point in the reference image with feature dimension d = 256 and the key feature vector Key for the j-th feature point in the target image, respectively. They learn cross-frame feature correlations through the Transformer architecture. This is a scaling factor to prevent the gradient from vanishing due to excessively large dot product values.

[0105] For the matching set For each pair of matched feature points (i,j), combined with their corresponding historical keyframe poses, their corresponding 3D map points are calculated using a triangulation method. The coordinates in the world coordinate system are obtained using the feature pixel coordinates p of the current frame. i Visual reprojection error term r constructed with 3D map points vis :

[0106]

[0107] Where π(·) is the camera projection model, and T is the pose of the current keyframe relative to the world coordinate system.

[0108] S4. Non-key frame processing: An improved corner detector is used to extract feature points of non-key frames, and feature displacement is obtained by IMU-constrained optical flow tracking. Visual reprojection error and IMU pre-integration residual terms are generated.

[0109] An improved corner detector is used to extract feature points Score(x,y) from the non-keyframe image I′:

[0110] Score(x,y)=min(λ1,λ2)>τ GFTT (32)

[0111] Where λ1 and λ2 are the eigenvalues ​​of the autocorrelation matrix, representing the intensity of gray-level changes in the pixel neighborhood, and τ GFTT is the threshold for the feature point response, and Score(·) is the corner response function.

[0112] When the IMU angular velocity ω satisfies ||ω||·t exp >δ blur At that time, motion blur compensation is performed on the input image:

[0113]

[0114] Where I, I′ are the input and output images, and t exp For exposure time, δ blur =0.05rad, For two-dimensional discrete Fourier transform and inverse transform, sinc(·) is used to describe the fuzzy transfer function caused by uniform rotation.

[0115] Non-keyframe feature point storage uses lossy compression, with a compression ratio ρ satisfying:

[0116]

[0117] W = diag([0.9, 0.7, 0.5]) (35)

[0118] Where d i is the original descriptor, and W is the weight matrix.

[0119] IMU-constrained optical flow tracing is performed on the feature point Score(x,y) to balance the weights of visual consistency and IMU priors:

[0120]

[0121] Where Δx and Δy are the horizontal and vertical displacement components of the feature point between adjacent frames, and Δp is the amount of movement of the feature point between adjacent frames on the image plane. IMU I represents the image displacement that the same feature point, inferred from IMU pre-integration data, should appear between two frames. t (p) represents the gray value at position p in the image at time t, I t+1 Similarly, λ represents the regularization coefficient. (p+△p)

[0122] like Figure 3 As shown, this invention analyzes the feature extraction and matching effects of key frames (entropy-maximizing self-supervised feature extraction and graph attention-based feature matching) and non-key frames (improved corner detection and IMU-constrained optical flow tracking) based on data image scenes collected by a small high-speed unmanned aerial vehicle.

[0123] The visual reprojection error term r of the non-key frame is obtained through equation (31). vis The pose increment Δp obtained by IMU pre-integration IMU and its covariance Σ IMU Transformed into IMU pre-integrated residual term r IMU .

[0124] S5. Solve the system based on the vision-inertial tightly coupled optimization module. Combine the visual reprojection error, IMU pre-integration residual and aerodynamic constraint residual to construct a tightly coupled optimization problem and solve for the optimal pose state of the system.

[0125] By combining visual reprojection error with IMU pre-integration residuals, a lightweight maximum a posteriori estimation problem is constructed:

[0126] T * =argmin T (∑||r vis || 2 +||r IMU || 2 +||r aero || 2 (37)

[0127] Where argmin T (·) represents the value of T that minimizes the objective function, r aero State constraints for integrating aerodynamics:

[0128] r aero =||v IMU -v wind-tunnel || 2 (38)

[0129] Where v IMU For IMU to measure velocity, v wind-tunnel This is the theoretical speed in the wind tunnel.

[0130] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any one of embodiments S1-S5.

[0131] An embedded navigation device includes a processor, a memory, and a sensor interface, characterized in that: when running any one of the methods in embodiments S1-S5, the power consumption is less than 25W and the positioning update rate is ≥50Hz.

[0132] This embodiment calculates ATE (Absolute Track Error) based on flight data. For example... Figure 4 As shown, in the ATE error assessment test, the method of the present invention exhibits low translational error and stable rotational error. The RMSE (root mean square error) of the method of the present invention is 11.971 m, and the error remains stable with increasing time and mileage. The above results verify that the present invention possesses excellent accuracy and robustness.

[0133] To further quantify performance, such as Figure 5 As shown, this embodiment visualizes the three-dimensional trajectory of the algorithm of the present invention and marks the starting point of the pentagram and the ending point of the dot. Trajectory comparison shows that the method of the present invention closely matches the baseline in all time periods. In particular, it is evident that as the flight time progresses, the trajectory line of the method of the present invention stably conforms to the baseline, fully demonstrating its significant advantages in improving trajectory accuracy and robustness.

[0134] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A state estimation method based on hybrid features and adaptive keyframe selection, characterized in that: Includes the following steps: S1. Acquire multiple images using an industrial camera and simultaneously obtain inertial measurement data from a six-axis IMU; S2. Based on inertial measurement data, perform keyframe decision-making on the current frame image, classifying the image into keyframe images and non-keyframe images: S3. Keyframe processing: Obtain keyframe feature matching set through entropy maximization self-supervised feature extractor and graph attention matcher, and construct visual reprojection error term based on triangulated map points; The implementation method of S3 is as follows: Feature points are extracted from keyframe image I using an entropy-maximizing self-supervised feature extractor to generate a feature point set: （2） in These are the pixel coordinates of the feature point. This is the probability threshold; The feature matcher using the graph attention mechanism calculates the matching relationships between feature points: （3） （4） in For attention mechanisms, Represents the first keyframe extracted from the current keyframe. One feature point, Represents the first keyframe extracted from the previous keyframe One feature point, For querying feature vectors, Query Let d be the key feature vector, and d be the feature dimension. For feature matching set; For the matching set Each pair of matching feature points By combining the corresponding historical keyframe poses, the corresponding 3D map points are calculated using the triangulation method. Coordinates in the world coordinate system, using the feature pixel coordinates of the current frame. Visual reprojection error term constructed with 3D map points : （5） in Let T be the pose of the current keyframe relative to the world coordinate system, which is the camera projection model. S4. Non-key frame processing: An improved corner detector is used to extract feature points of non-key frames, and feature displacement is obtained by IMU-constrained optical flow tracking. Visual reprojection error and IMU pre-integration residual terms are generated. The implementation method of S4 is as follows: Employing an improved corner detector, from non-keyframe images Extracting feature points : （6） in The eigenvalues ​​of the autocorrelation matrix characterize the intensity of gray-level changes in the pixel neighborhood. The threshold for the feature point response. Corner response function; For feature points IMU-constrained optical flow tracking is performed to balance the weights of visual consistency and IMU priors. （7） in These are the horizontal and vertical displacement components of the feature points between adjacent frames. This represents the amount of movement of a feature point between adjacent frames on the image plane. This represents the image displacement that the same feature point, predicted from the IMU pre-integration data, should appear between two frames. This represents the gray value at position p in the image at time t. Similarly, Represents the regularization coefficient; The visual reprojection error term for non-key frames is obtained through equation (5). The pose increment obtained by IMU pre-integration and its covariance Transformed into IMU pre-integrated residuals ; S5. Solve the system based on the vision-inertial tightly coupled optimization module. Combine the visual reprojection error, IMU pre-integration residual and aerodynamic constraint residual to construct a tightly coupled optimization problem and solve for the optimal pose state of the system. The implementation method of S5 is as follows: By combining visual reprojection error with IMU pre-integration residuals, a lightweight maximum a posteriori estimation problem is constructed: （8） in For IMU pre-integration residuals; in To use the value of T that minimizes the objective function, State constraints for integrated aerodynamics: （9） in For IMU to measure speed, This is the theoretical speed in the wind tunnel.

2. The method as described in claim 1, characterized in that: The implementation method of S2 is as follows: Keyframes and non-keyframes are distinguished using equation (1): （1） in For IMU pre-integration translation, For the IMU pre-integration rotation angle, This is the enhanced pre-integral covariance matrix after vibration compensation. The displacement threshold is the minimum displacement change that triggers keyframe processing. The rotation threshold is the minimum rotation angle that triggers keyframe processing. For uncertain thresholds, i.e. The upper limit of the trace, Norm operations are performed on Euclidean distance. The trace operation of a matrix, i.e., the sum of the diagonal elements, is used to quantify the total uncertainty of motion estimation.

3. The method as described in claim 2, characterized in that: The Dynamically adjust based on computing platform load: （10） （11） （12） in, For GPU real-time utilization, For CPU real-time utilization, For memory usage, This is an adjustable coefficient.

4. The method according to claim 2, characterized in that: The IMU pre-integration covariance matrix Calculated using the error state propagation equation: （13） in that is Intermediate state during iterative calculation Here is the state transition matrix. The noise Jacobian matrix, Let be the IMU noise covariance, and k be the time series.

5. The method according to claim 1, characterized in that: The entropy-maximizing self-supervised feature extractor described in S3 is... （14） （15） in For entropy weighting coefficients, The objective function is entropy information. Indicates pixel position The probability of being a feature point; This represents the original detection loss.

6. The method according to claim 1, characterized in that: The matcher described in S3 employs an attention mechanism: （16） in , feature dimensions The reference image of the first The query feature vector Query of the i-th feature point and the i-th feature point in the target image The key feature vector Key for each feature point learns cross-frame feature correlations through the Transformer architecture. This is a scaling factor to prevent the gradient from vanishing due to excessively large dot product values.

7. The method according to any one of claims 1 to 6, characterized in that: The visual inertial odometry system consists of five core modules that work together: the image acquisition module uses a global shutter camera and eliminates image distortion caused by high-speed motion through a hardware synchronization mechanism to ensure geometric consistency of feature extraction; the inertial measurement module integrates a six-axis IMU sensor to provide high-frequency angular velocity and acceleration measurements, and builds an inertial reference for motion state estimation; The adaptive decision-making module includes a motion uncertainty analysis unit that integrates IMU pre-integrated covariance and computing platform load status in real time to dynamically adjust keyframe triggering conditions. The hybrid processing module implements a differentiated processing architecture for keyframes and non-keyframes. The keyframe path uses deep learning feature extraction and a matcher to achieve sub-pixel-level matching accuracy, while the non-keyframe path uses traditional optical flow algorithms to achieve millisecond-level efficient tracking. The two paths exchange feature status data through shared memory. The tightly coupled optimization module uses a sliding window optimization framework that combines visual reprojection constraints, inertial measurement constraints, and platform dynamic constraints to output a six-degree-of-freedom pose estimate. All modules are interconnected through a high-speed data bus to form a closed-loop processing link.

Images