Hybrid Global and Rolling Shutter Sensor System for AR Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Visual SLAM techniques face reduced tracking accuracy due to rolling shutter distortion in rolling shutter sensors, while global shutter sensors struggle with high resolution and frame rate at low cost.
Innovation Solution
An image processing apparatus utilizing a global shutter sensor for accurate position/orientation estimation and a rolling shutter sensor for high-resolution image capture, with a rendering unit to superimpose CG objects based on generated position and orientation information, allowing for both accurate estimation and high-definition view images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a global shutter sensor is used, then rolling shutter distortion is eliminated and position/orientation estimation accuracy is improved, but power consumption increases and it is difficult to achieve high resolution and high frame rate
Solution Approach 1:
The patent divides the imaging function into two separate sensors: a global shutter sensor dedicated to position and orientation estimation, and a rolling shutter sensor dedicated to high-definition image capture. This segmentation allows each sensor to be optimized for its specific purpose, with the global shutter sensor operating at lower resolution and frame rate to reduce power consumption while maintaining accurate motion estimation.
Solution Approach 2:
The patent applies different quality levels to different parts of the imaging system. The global shutter sensor operates at lower resolution specifically for the purpose of motion estimation, while the rolling shutter sensor captures high-resolution images for display. This local differentiation of quality allows the system to achieve accurate position/orientation estimation without requiring the entire imaging system to consume high power.
2Ease of manufacture
If a rolling shutter sensor is used, then power consumption is reduced and high resolution is achieved, but rolling shutter distortion occurs and tracking accuracy is reduced
Solution Approach 1:
The patent segments the imaging functions so that the rolling shutter sensor is dedicated solely to high-resolution image capture for display, while position and orientation estimation is performed exclusively by the global shutter sensor. This eliminates the harmful interaction between rolling shutter distortion and tracking accuracy.
Solution Approach 2:
The global shutter sensor acts as an intermediary that provides accurate motion estimation data without being affected by rolling shutter distortion. This intermediary sensor bridges the gap between the need for high-resolution imaging (provided by the rolling shutter sensor) and the need for accurate tracking (provided by the global shutter sensor).
3Measurement precision
If a rolling shutter sensor operates at high speed, then rolling shutter distortion is reduced, but power consumption increases and it is difficult to achieve high frame rate with high resolution
Solution Approach 1:
The patent segments the functional requirements: the global shutter sensor handles the power-intensive high-speed operation needed for accurate motion estimation, while the rolling shutter sensor operates at standard speeds for high-resolution image capture. This segmentation allows the system to achieve distortion reduction without requiring the rolling shutter sensor to consume excessive power.
4Device complexity
If a single sensor is used for both position/orientation estimation and high-definition imaging, then device complexity is reduced, but it is impossible to simultaneously achieve accurate estimation and high-definition images
Solution Approach 1:
The patent segments the imaging system into two specialized sensors: a global shutter sensor for position and orientation estimation, and a rolling shutter sensor for high-definition image capture. Although this increases the number of sensors, each sensor is simplified in its specific function, and the overall system complexity is managed through dedicated processing pipelines for each sensor type.
Solution Approach 2:
The patent creates a multi-functional imaging system where two sensors work together to provide both accurate motion estimation and high-definition imaging. The system integrates data from both sensors to achieve multiple functions that neither sensor could achieve alone, with the global shutter sensor providing motion data and the rolling shutter sensor providing high-resolution visual data.
Data Source
AI summary
There is provided with an image processing apparatus. A first image sensor outputs a first image. The first image sensor has a relatively small amount of image deterioration caused by a motion of an object. A second image sensor outputs a second image. The second image sensor has a relatively large amount of image deterioration caused by the motion of the object. An estimation unit analyzes the first image and generates position and orientation information of the image processing apparatus. A rendering unit renders a CG object on the second image such that the CG object is superimposed at a position determined based on the position and orientation information.