Rolling Shutter Visual-Inertial Tracking With Adaptive Pose Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rolling shutter cameras capture images by scanning across a scene, leading to artifacts when there is relative movement between the camera and the scene during image acquisition, which complicates visual-inertial tracking systems.
Innovation Solution
The system adjusts the number of poses calculated for rolling shutter cameras based on the movement of the eyewear device, allowing for more efficient processing by conserving resources when faster movement is not necessary for acceptable augmented reality results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the system computes a fixed number of poses for rolling shutter camera images, then processing is simplified, but processing resources are wasted when movement is slow
Solution Approach 1:
The system dynamically adjusts the number of poses computed based on the detected movement of the eyewear device. When movement exceeds a threshold, more poses are computed to maintain accuracy; when movement is minimal, fewer poses are computed to save resources. This dynamic adaptation resolves the contradiction between processing efficiency and resource consumption.
Solution Approach 2:
The system changes the parameter of pose quantity based on movement conditions. By monitoring device movement and adjusting the number of computed poses accordingly, the system optimizes processing resources while maintaining acceptable augmented reality results, directly addressing the efficiency-resource tradeoff.
2Measurement precision
If the system computes more poses for rolling shutter camera images, then pose estimation accuracy improves, but processing complexity increases
Solution Approach 1:
The system dynamically adjusts computational effort by computing varying numbers of poses based on real-time movement detection. This dynamic approach maintains high accuracy when needed (during fast movement) while reducing complexity during slow movement, resolving the contradiction between precision and complexity.
Solution Approach 2:
The system changes the pose quantity parameter in response to movement conditions, optimizing the balance between accuracy and complexity. By adjusting this parameter dynamically, the system achieves high measurement precision only when necessary, thereby managing processing complexity effectively.
3Loss of energy
If the system adjusts pose calculation based on device movement, then processing resources are optimized, but system complexity increases
Solution Approach 1:
The system performs preliminary movement detection and threshold comparison before committing to pose calculation. By pre-evaluating movement conditions and determining the appropriate number of poses in advance, the system optimizes resource usage without requiring complex real-time adjustments during processing.
Solution Approach 2:
The system uses feedback from movement detection to adjust pose calculation parameters. The movement detection unit provides feedback about device motion, which the processor uses to determine the appropriate number of poses to compute, creating a closed-loop system that optimizes resources with manageable complexity.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Visual-inertial tracking of an eyewear device using a rolling shutter camera(s). The eyewear device includes a position determining system. Visual-inertial tracking is implemented by sensing motion of the eyewear device. An initial pose is obtained for a rolling shutter camera and an image of an environment is captured. The image includes feature points captured at a particular capture time. A number of poses for the rolling shutter camera is computed based on the initial pose and sensed movement of the device. The number of computed poses is responsive to the sensed movement of the mobile device. A computed pose is selected for each feature point in the image by matching the particular capture time for the feature point to the particular computed time for the computed pose. The position of the mobile device is determined within the environment using the feature points and the selected computed poses for the feature points.