Rolling Shutter Reduction Using Timestamped Motion Sensors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
CMOS image sensors exhibit rolling shutter distortions due to sequential readout, causing geometric distortions in images and videos, especially during device movement, which are aesthetically unpleasing and difficult to correct in small form factor devices without substantial power or processing overhead.
Innovation Solution
Employing timestamped positional sensor data from gyroscopic and accelerometer sensors to apply perspective transformations to image segments, reducing rolling shutter distortions by correlating motion data with image capture times and using anchor rows for motion estimation and correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If CMOS sensors are used for video capture, then cost and power consumption are reduced, but rolling shutter geometric distortions occur during device movement
Solution Approach 1:
The system performs preliminary action by capturing motion data from gyroscopic and accelerometer sensors at timestamps corresponding to when each row of the image sensor is read out. This motion data is then used to calculate perspective transformation parameters before the actual image correction is applied, allowing the system to pre-compensate for rolling shutter effects during device movement
Solution Approach 2:
The system changes parameters by applying perspective transformation matrices to correct the geometric distortions. By calculating transformation parameters based on motion data and applying different corrections to different portions of the image, the system compensates for the rolling shutter effect while maintaining low power consumption characteristics of CMOS sensors
2Manufacturing precision
If image correction techniques are applied to reduce rolling shutter effects, then image quality improves, but computational complexity and processing time increase
Solution Approach 1:
The system segments the image correction process by dividing the image into multiple portions and applying perspective transformations independently to each segment. This segmentation allows for more efficient processing compared to correcting the entire image uniformly, reducing overall computational complexity while maintaining correction accuracy
Solution Approach 2:
The system uses motion data from gyroscopic and accelerometer sensors as an intermediary to facilitate the correction process. Instead of directly analyzing and correcting geometric distortions in the image, the system uses motion data as a mediator to calculate perspective transformation parameters, simplifying the correction process and reducing computational burden
3Manufacturing precision
If real-time rolling shutter correction is implemented, then image quality is improved during device movement, but power consumption increases
Solution Approach 1:
The system performs preliminary action by capturing motion data from gyroscopic and accelerometer sensors at timestamps corresponding to when each row of the image sensor is read out. This motion data is then used to calculate perspective transformation parameters before the actual image correction is applied, allowing the system to pre-compensate for rolling shutter effects during device movement
Solution Approach 2:
The system uses self-service by leveraging motion data from sensors that are already present in the device for other functions. By repurposing these existing sensors for rolling shutter correction, the system avoids adding dedicated correction hardware that would increase power consumption, instead using available resources to achieve image quality improvement
Data Source
AI summary
This disclosure pertains to devices, methods, and computer readable media for reducing rolling shutter distortion effects in captured video frames based on timestamped positional information obtained from positional sensors in communication with an image capture device. In general, rolling shutter reduction techniques are described for generating and applying image segment-specific perspective transforms to already-captured segments of a single image or images in a video sequence, to compensate for unwanted distortions that occurred during the read out of the image sensor. Such distortions may be due to, for example, the use of CMOS sensors combined with the movement of the image capture device. In contrast to the prior art, rolling shutter reduction techniques described herein may be applied to captured images or videos in real-time or near real-time using positional sensor information and without intensive image processing that would require an analysis of the content of the underlying image data.