Event Camera Visual-Inertial Odometry Motion Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-art visual-inertial odometry (VIO) and SLAM algorithms face limitations in accuracy and processing speed, especially in high-speed motions and high-dynamic range scenes, due to motion blur and limited dynamic range of frame-based cameras, which restrict their use in real-time applications with event cameras.
Innovation Solution
A method for generating motion-corrected images from event cameras using a monocular event camera rigidly connected to an IMU, where events are corrected by assigning positions based on estimated camera poses, utilizing IMU data for global motion correction and depth information to eliminate motion blur and enhance feature detection, allowing for real-time processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If frame-based cameras are used for visual-inertial odometry, then standard imaging is achieved, but motion blur occurs during high-speed motions reducing measurement precision
Solution Approach 1:
The patent changes the fundamental parameter of image capture from periodic frame-based sampling to continuous event-based detection. Event cameras only output data when brightness changes occur, with timestamps at microsecond resolution, eliminating the motion blur inherent in frame-based systems while maintaining measurement precision for visual-inertial odometry.
Solution Approach 2:
The patent replaces the mechanical shutter and periodic exposure mechanism of frame-based cameras with an event-driven detection system. Instead of mechanically capturing frames at fixed intervals, the system uses asynchronous event streams triggered by brightness changes, substituting mechanical timing with event-based temporal sampling.
2Measurement precision
If frame-based cameras with limited dynamic range are used, then standard imaging is achieved, but large regions are over- or under-exposed reducing measurement precision
Solution Approach 1:
The patent changes the dynamic range parameter from 60 dB in frame-based cameras to 130 dB in event cameras. This allows the sensor to capture brightness changes across a much wider range of illumination conditions without over- or under-exposure, maintaining measurement precision in high-dynamic-range scenes.
Solution Approach 2:
The patent introduces dynamic adaptability through event-based detection, where each pixel independently responds to brightness changes in real-time. This dynamic response allows the system to adapt to rapidly changing illumination conditions, preventing over- and under-exposure in scenes with wide variations in brightness.
3Measurement precision
If event cameras are used for visual-inertial odometry, then motion blur is eliminated and dynamic range is increased, but computational complexity increases for real-time processing
Solution Approach 1:
The patent extracts only the essential information from the event stream by integrating events between keyframes using IMU data for motion compensation. Instead of processing all events independently, the system extracts relative motion information and integrates it with inertial measurements, reducing computational complexity while maintaining measurement precision.
Solution Approach 2:
The patent merges event camera data with IMU data in a unified visual-inertial odometry framework. By combining asynchronous event streams with inertial measurements, the system achieves real-time processing efficiency, leveraging the complementary strengths of both sensors to reduce overall computational complexity.
4Measurement precision
If non-linear optimization methods are used for event camera VIO, then accuracy is improved, but processing speed decreases limiting real-time application
Solution Approach 1:
The patent performs preliminary motion compensation using IMU data before applying non-linear optimization to event integration. By pre-compensating for camera motion using inertial measurements, the system reduces the computational burden of subsequent optimization, enabling real-time processing while maintaining high accuracy in visual-inertial odometry.
Solution Approach 2:
The patent implements a dynamic processing pipeline that adapts the level of optimization based on computational requirements. The system uses IMU pre-integration for rapid motion estimation and applies non-linear optimization selectively, allowing real-time performance while maintaining accuracy when computational resources are available.
Data Source
Figure 1~2
Figure 3A~3C
AI summary
The invention relates a method for generating a motion-corrected image for visual- inertial odometry comprising an event camera rigidly connected to an inertia! measurement unit (IMU), wherein the event camera comprises pixels arranged in an image plane that are configured to output events in presence of brightness changes in a scene at the time they occur, wherein each event comprises the time at which it is recorded and a position of the respective pixel that detected the brightness change, the method comprising the steps of: Acquiring at least one set of events (S), wherein the at least one set (S) comprises a plurality of subsequent events (e); Acquiring I MU data (D) for the duration of the at least one set (S); Generating a motion-corrected image from the at least one set (S) of events (e), wherein the motion-corrected image is obtained by assigning the position (x j ) of each event (e j ) recorded at its corresponding event time (t j ) at an estimated event camera pose (T tj ) to an adjusted event position (x'j), wherein the adjusted event position (x'j) is obtained by determining the position of the event (e j ) for an estimated reference camera pose (T t f k ) at a reference time (t f k ), wherein the estimated camera pose (T tj ) at the event time (t j ) and the reference camera pose (T t f k ) at the reference time (t f k ) are estimated by means of the IMU data (D).