Visual-Inertial Odometry Fusion With Dual-Thread PnP and SBA
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional visual-inertial odometry techniques face challenges in providing accurate and precise position and orientation determination for machines, especially in environments with low texture, high motion blur, or dynamic object obstruction, as they fail to effectively fuse visual and inertial measurements.
Innovation Solution
The system employs a dual-thread processing approach, utilizing perspective-n-point (PnP) and sparse bundle adjustment (SBA) techniques to better fuse image and motion data, with adaptive selection based on IMU sensor states, enhancing the accuracy and reliability of state estimation even in challenging conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional visual odometry techniques are used to determine machine position and orientation, then the system can operate with simple processing, but the measurement precision deteriorates in challenging environments such as low texture, high motion blur, or dynamic object obstruction
Solution Approach 1:
The system segments the visual-inertial odometry problem into two distinct processing threads: a first thread that executes PnP techniques to determine machine states from image data, and a second thread that performs SBA techniques to adjust states and refine measurements. This segmentation allows each thread to specialize in specific processing tasks, improving measurement precision while managing system complexity through modular design.
Solution Approach 2:
The system dynamically selects between different PnP techniques (e.g., standard PnP, inertial PnP, soft inertial PnP) based on the current IMU sensor state and environmental conditions. This dynamic adaptation enables the system to optimize measurement precision for challenging conditions like high motion blur or low texture by choosing the most appropriate processing technique for the current situation.
2Measurement precision
If conventional techniques fuse visual and inertial measurements, then the system can improve measurement robustness, but the fusion accuracy deteriorates due to inadequate integration methods
Solution Approach 1:
The system implements dynamic adaptability by monitoring IMU sensor states and automatically selecting between multiple PnP techniques (standard PnP, inertial PnP, soft inertial PnP) based on the current operational conditions. This allows the fusion system to adapt to different sensor states and environmental challenges, improving state estimation accuracy while maintaining versatility across various operating conditions.
Solution Approach 2:
The system creates a composite processing approach by integrating multiple PnP techniques and SBA techniques into a unified visual-inertial odometry system. This composite methodology combines the strengths of different processing techniques, enabling accurate state estimation by leveraging both visual measurements from cameras and inertial measurements from IMU sensors through a unified fusion framework.
3Measurement precision
If the system uses multiple processing techniques (PnP and SBA) to improve accuracy, then the measurement precision improves, but the computing time increases
Solution Approach 1:
The system segments processing into two concurrent threads that operate in parallel: the first thread handles PnP-based state determination and the second thread handles SBA-based state refinement. This parallel segmentation enables both techniques to execute simultaneously, improving measurement precision through combined processing while minimizing computing time by avoiding sequential execution.
Solution Approach 2:
The first thread performs preliminary state determination using PnP techniques, generating initial state estimates that are then refined by the second thread using SBA techniques. This preliminary action approach allows the system to establish baseline measurements quickly and then apply more computationally intensive refinement only where needed, optimizing the balance between precision and processing time.
Data Source
AI summary
In various examples, sensor fusion for visual-inertial odometry in autonomous and semi-autonomous systems and applications is described herein. Systems and methods are disclosed that split processing into at least two components. For example, the first component may be configured to process incoming frames, execute one or more perspective-n-point techniques to determine states of a machine, update states associated with one or more inertial measurement unit sensors of the machine, and add new frames to a map. The second component may be configured to adjust states (e.g., poses) associated with the machine using one or more sparse bundle adjustment techniques, adjust points within an environment, and adjust IMU-related parameters using a history of camera states. In some examples, the PnP technique and/or the SBA technique may be selected based on states associated with the IMU sensor(s).