Rolling Shutter Artifact Correction via Line-by-Line Motion Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cameras with rolling shutter sensors face issues with rolling shutter artifacts due to non-linear motion during frame exposure, leading to undesirable artifacts in video images, particularly from pitch, yaw, and roll motion.
Innovation Solution
The implementation of electronic image stabilization (EIS) systems that measure non-linear motion using motion sensors and adjust the position of each line in a frame to create a modified frame, correcting for pitch, yaw, and roll motion by interpolating and shifting pixels, thereby eliminating artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If rolling shutter readout is used to capture video frames, then the camera can capture images efficiently with sequential line exposure, but rolling shutter artifacts occur due to non-linear motion during the exposure time of different lines
Solution Approach 1:
The patent divides the frame correction process into line-by-line operations, where each line's position is adjusted independently based on its specific exposure time and measured motion. This segmentation allows precise correction of rolling shutter artifacts for each line while maintaining overall frame coherence, resolving the contradiction between efficient rolling shutter capture and artifact-free output.
Solution Approach 2:
The patent changes the position parameter of each line in the frame based on measured motion data. By adjusting the vertical position of each line according to the camera's motion during its specific exposure time, the system compensates for rolling shutter artifacts while preserving the efficient rolling shutter capture mechanism.
2Object-affected harmful factors
If non-linear motion correction is applied to each line of the frame, then rolling shutter artifacts are eliminated, but the processing complexity and computational load increase
Solution Approach 1:
The patent performs motion measurement during the exposure time of each line, before the frame is fully captured. By having motion data available in advance for each line, the system can apply corrections during frame assembly or post-processing without requiring complex real-time adjustments, thereby reducing overall processing complexity while effectively eliminating artifacts.
Solution Approach 2:
The patent replaces complex mechanical stabilization systems with computational methods. Instead of using physical gyroscopes or mechanical gimbals to prevent motion, the system uses software-based line position adjustment based on motion sensor data, simplifying the overall system while achieving effective artifact correction.
3Object-affected harmful factors
If the position of each line is adjusted based on measured non-linear motion, then artifact-free video images are produced, but the processing time and computational resources required increase
Solution Approach 1:
The patent uses the camera's own motion sensors to provide the correction data needed for artifact removal. By measuring motion during capture and using that same data for correction, the system eliminates the need for separate processing steps or additional hardware, reducing processing time and computational overhead while effectively removing rolling shutter artifacts.
Data Source
AI summary
A method and system are provided to perform electronic image stabilization of images captured by an image sensor on a camera device. In the method and system, non-linear motion of a camera device is measured with a motion sensor during an exposure time for each line of a frame captured by the image sensor of the camera device. A first line of the frame has a first exposure time, and subsequent lines of the frame have an exposure time that is later than a previous line of the frame, each line of the frame having an associated position in the frame. The position of each line of the frame is adjusted based, at least in part, on the measured non-linear motion to create a modified frame that corrects for non-linear motion that occurred for each line of the frame.