Sensor Fusion Motion Estimation Using IMU and Camera Threshold Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sensor fusion methods for estimating device motion often face challenges in accurately combining data from inertial measurement units (IMUs) and cameras, particularly in scenarios where the differences in estimated motion exceed a threshold, leading to inconsistencies in position and velocity calculations.
Innovation Solution
A method that involves receiving data from both IMUs and cameras over a sliding time window, determining independent estimations of motion, and reconciling discrepancies by incorporating additional positional data from GPS and other sensors to provide a comprehensive estimation of device motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor fusion methods combine IMU and camera data to estimate device motion, then motion estimation can be performed, but inconsistencies and inaccuracies occur when differences in estimated motion exceed a threshold
Solution Approach 1:
The system continuously monitors the difference between IMU-based motion estimation and camera-based motion estimation. When the difference exceeds a predetermined threshold, the system triggers a feedback mechanism to detect and correct inconsistencies. This feedback loop ensures that position and velocity calculations remain reliable by adjusting the fusion process based on real-time comparison of sensor data, thereby resolving the contradiction between measurement precision and reliability.
2Productivity
If the system uses only IMU data for motion estimation, then processing is simple and fast, but accuracy deteriorates when camera images remain constant or IMU data indicates movement
Solution Approach 1:
The system merges IMU data and camera data into a unified motion estimation framework. The fusion process combines the complementary strengths of both sensors: IMU provides rapid responses and smooth motion tracking, while camera data provides visual confirmation and correction. By merging these data sources and using a threshold-based validation mechanism, the system achieves both high processing speed and high accuracy, resolving the contradiction between productivity and measurement precision.
3Device complexity
If the system uses only camera data for motion estimation, then data processing is straightforward, but reliability decreases when camera images remain constant despite device movement
Solution Approach 1:
The system introduces an intermediary fusion mechanism that mediates between camera data and IMU data. When camera images remain constant despite device movement, the intermediary system detects this discrepancy and uses IMU data as a supplement to maintain reliable motion tracking. This intermediary fusion process adds minimal complexity while significantly improving reliability, resolving the contradiction between device complexity and motion tracking reliability.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
Methods and systems for determining estimation of motion of a device are provided. An example method includes receiving data from an inertial measurement unit (IMU) of a device and receiving images from a camera of the device for a sliding time window. The method also includes determining an IMU estimation of motion of the device based on the data from the IMU, and a camera estimation of motion of the device based on feature tracking in the images. The method includes, based on the IMU estimation and the camera estimation having a difference more than a threshold amount, determining one or more of a position or a velocity of the device for the sliding time window, and determining an overall estimation of motion of the device as supported by the data from the IMU and the position or velocity of the device.