Rolling Shutter Correction Using IMU Attitude Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital cameras with rolling shutter mechanisms suffer from geometrical distortion due to motion, especially when used in movable objects like UAVs, and mechanical image stabilization systems are inadequate for rapid motions and are costly, making it difficult to correct the rolling shutter effect in real-time.
Innovation Solution
An image processing method that uses attitude information from sensors like IMUs to derive the positional state of pixels and apply real-time corrections to image frames, allowing for concurrent image capture and processing to address the rolling shutter effect without requiring offline calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If mechanical image stabilization systems are used to compensate for camera motion, then image stability is improved, but device cost and complexity increase significantly
Solution Approach 1:
The patent replaces mechanical image stabilization systems with a computational approach. An image processing apparatus uses sensors to detect camera motion and applies digital transformation to correct rolling shutter distortion, eliminating the need for mechanical stabilization components while achieving image stability through software-based correction.
2Stability of the object's composition
If mechanical image stabilization systems are used to compensate for rapid high frequency motions, then image stability is improved, but the system becomes unsuitable for consumer devices due to cost and form factor constraints
Solution Approach 1:
The patent substitutes mechanical stabilization components with a software-based image processing system that uses sensor data to correct rolling shutter effects. This approach dramatically reduces manufacturing complexity and cost while maintaining effectiveness in compensating for rapid high-frequency motions, making it suitable for consumer devices.
3Ease of manufacture
If rolling shutter mechanism is used in CMOS sensors, then device cost is reduced, but geometrical distortion occurs due to motion during sequential row exposure
Solution Approach 1:
The patent uses a computational correction system to compensate for the geometrical distortion inherent in rolling shutter mechanisms. By detecting camera motion through sensors and applying digital transformation to each row of pixels based on its exposure time, the system maintains the cost advantages of rolling shutter CMOS sensors while correcting the resulting geometric inaccuracies.
4Manufacturing precision
If offline batch image processing is used to correct rolling shutter effect, then correction accuracy is improved, but real-time video quality cannot be enhanced
Solution Approach 1:
The patent performs motion detection and correction calculations concurrently with image capture rather than after completion. The image processing apparatus detects camera motion during the exposure period and applies correction transformations as each row is captured, enabling real-time correction output without requiring offline batch processing while maintaining accuracy.
Data Source
AI summary
An image processing method includes obtaining an image frame through an imaging device over a period of time. The image frame includes a plurality of groups of pixels that are exposed to light at different time points within the period of time. The method further includes obtaining attitude information of the imaging device during the period of time, deriving positional state of an individual group of pixels in the plurality of groups of pixels based on the attitude information of the imaging device, and processing the image frame using the positional state.