Rolling Shutter Image Processing for Projected Pattern Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of inexpensive rolling shutter cameras in image processing systems for capturing projected images is hindered by the reflection of multiple frames in a single captured frame, making it difficult to extract pattern images and necessitating the use of expensive global shutter cameras, thereby increasing costs.
Innovation Solution
An image processing apparatus that calculates differences between discontinuous frames of a captured image obtained by projecting alternating positive and negative pattern images at a higher frame rate than the projected image, using a rolling shutter scheme, to extract and correct projected images while synchronizing projection and imaging timing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a rolling shutter camera is used to capture projected images, then the cost is reduced, but multiple frames are reflected in a single captured frame making it difficult to extract pattern images
Solution Approach 1:
The captured image containing multiple reflected frames is divided into multiple time-separated frame segments. The difference calculation section calculates differences between these segmented frames to isolate the pattern image from the mixed content, enabling extraction despite the rolling shutter effect.
Solution Approach 2:
Instead of trying to capture a single clean frame directly, the method projects alternating positive and negative pattern images and calculates the difference between captured frames. This inversion approach—projecting opposites and subtracting—allows the pattern image to be extracted from the mixed frame reflections.
2Measurement precision
If a global shutter camera is used to capture projected images, then pattern images can be extracted accurately, but the cost increases
Solution Approach 1:
The patent replaces the expensive global shutter camera with a cheap rolling shutter camera. The system accepts the limitations of the inexpensive device and compensates through software processing—calculating differences between multiple captured frames to achieve the pattern extraction that would otherwise require expensive hardware.
Solution Approach 2:
The method changes the temporal parameter by capturing images at a higher frame rate than the projection frame rate. This parameter change allows the rolling shutter camera to capture enough temporal information to reconstruct the pattern image through difference calculation, achieving the same result as a global shutter camera would provide in a single frame.
3Measurement precision
If pattern images are projected at a higher frame rate than the projected image, then correspondence points can be detected accurately, but the processing complexity increases
Solution Approach 1:
The system performs preliminary actions by projecting alternating positive and negative pattern images before the actual content projection. These preliminary pattern projections are captured and processed to detect correspondence points, which are then used to guide the geometric correction of the main projected image.
Solution Approach 2:
The difference calculation between captured frames provides feedback about the pattern image position and distortion. This feedback is used to detect correspondence points between the projector and camera, which then feed back into the geometric correction process to improve image projection accuracy.
Data Source
AI summary
The present disclosure relates to an image processing apparatus and an image processing method capable of realizing grasping of a state of a projected image at a lower cost. A difference between discontinuous frames of a captured image configured with a plurality of frames obtained by capturing a projected image which is obtained by sequentially projecting predetermined pattern images opposite in sign, which are positive and negative pattern images, at a higher frame rate than a frame rate of the projected image by a rolling shutter scheme is calculated. The present disclosure can be applied to, for example, an image processing apparatus, a projection apparatus, an imaging apparatus, a projection imaging apparatus, a controller, a projection imaging system, or the like.