Omnidirectional Image Processing With Sensor-Based Rotation Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image capturing technologies struggle to accurately correct rotational distortions and blurs in omnidirectional images, particularly when the camera body is inclined or rotates, leading to misaligned zeniths and distorted horizontal lines.
Innovation Solution
An omnidirectional camera system with dual fish-eye lenses and sensors, combined with acceleration and angular velocity sensors, performs zenith and rotation corrections using metadata to align images with a reference direction, reducing rotational distortions and blurs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If posture information is detected and correction-amount data is calculated during shooting, then image correction is achieved, but computational load increases during shooting
Solution Approach 1:
The patent separates the correction process into two stages: during shooting, only essential posture information is detected and stored with minimal processing; the computationally intensive correction-amount calculation is performed later during reproduction. This preliminary action during shooting avoids heavy computation while capturing necessary data for subsequent correction.
Solution Approach 2:
The patent divides the image correction function into two independent components: (1) posture detection and data association during shooting, and (2) correction-amount calculation during reproduction. This segmentation allows the shooting phase to have low computational load while the reproduction phase handles the intensive correction calculations.
2Measurement precision
If only inclination correction is made using acceleration sensor, then zenith alignment is improved, but rotational distortion around vertical direction remains uncorrected
Solution Approach 1:
The patent introduces angular velocity sensor data as an intermediary to capture rotational motion around the vertical direction. This intermediary measurement complements the acceleration sensor data, enabling both zenith alignment (from acceleration sensor) and rotational distortion correction (from angular velocity sensor) to be achieved together.
Solution Approach 2:
The patent makes the posture detection system multi-functional by combining acceleration sensor data (for zenith alignment) and angular velocity sensor data (for rotational correction). This universal approach allows a single correction system to handle multiple types of camera motion errors simultaneously.
3Measurement precision
If correction is performed only in global coordinate system, then inclination correction is achieved, but small vibration information in horizontal plane is lost
Solution Approach 1:
The patent transforms the correction approach from a single global coordinate system to multiple coordinate systems. It performs correction in both the global coordinate system (for inclination) and the camera body coordinate system (for horizontal plane vibrations). This dimensional expansion preserves vibration information that would be lost in a single-coordinate-system approach.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Effectively corrects rotational blurs and aligns images with the reference direction, ensuring stable and comfortable viewing of omnidirectional images, with reduced computational load during shooting.
Implementation Method 1
an acceleration sensor disposed inside the image capturing apparatus for detecting an inclination angle of the image capturing apparatus with reference to the vertical direction
Implementation Method 2
an angular velocity sensor disposed inside the image capturing apparatus for detecting a rotational angle of the image capturing apparatus around the vertical direction
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
An image processing apparatus including a storage unit, a correction unit, and an output unit. The storage unit stores moving-image data including a plurality of frames captured by an image-capturing device communicable with the image processing apparatus, time-series data of an inclination angle with reference to a reference direction of the image-capturing device, and time-series data of an angle velocity of the image-capturing device. The correction unit, based on the time-series data of the angle velocity, rotates an image of each of the plurality of frames of the moving-image data to reduce a rotational distortion around the reference direction within a prescribed frequency range. The output unit outputs image data of the rotated image of each of the plurality of frames to an external device communicable with the image processing apparatus.