Rotation Correction for Omnidirectional Video Stabilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Omni-directional videos captured by cameras can cause motion sickness in viewers due to rotation around the reference axis, as the front direction of the video follows the cameraman's intentional movements, leading to difficulties in viewing, especially when the cameraman turns their head or changes direction significantly.
Innovation Solution
An image processing apparatus that acquires video and sensor data, and includes rotation correction means to cancel rotation changes around the reference axis, fixing the display reference to a specific direction throughout multiple frames, using angular and acceleration sensors to determine and correct for image stabilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the front direction of the video follows the movement of the omni-directional camera during image capturing, then the video reflects the cameraman's intentional motion, but the viewer experiences motion sickness due to considerable rotation of the video
Solution Approach 1:
The patent segments the rotation correction into two distinct components: high-frequency shaking correction (around vertical axis) and low-frequency intentional motion correction (front direction). This segmentation allows different correction strategies to be applied to different rotational components, resolving the contradiction by maintaining adaptability for intentional motion while eliminating harmful rotations that cause motion sickness.
Solution Approach 2:
Instead of allowing the video front direction to follow the camera's intentional motion (which causes motion sickness), the patent inverts the approach by fixing the video reference direction to a specific direction (e.g., initial front direction or traveling direction). This inversion eliminates the harmful effect while preserving the ability to capture intentional motion in the video content itself.
2Manufacturing precision
If coordinate conversion is performed to correct small shaking components on the horizontal plane, then high frequency shaking rotation is corrected, but the front direction still follows intentional motion such as head turning
Solution Approach 1:
The patent introduces dynamic adjustment of the video reference direction based on detection of intentional motion. When intentional motion (such as head turning) is detected, the system dynamically changes the reference direction to follow the traveling direction rather than the camera's front direction. This dynamic adaptation allows the system to maintain shaking correction precision while avoiding the harmful effect of following intentional motion that causes motion sickness.
Solution Approach 2:
The patent employs feedback mechanisms where the system detects the cameraman's intentional motion (through sensors or analysis of video content) and uses this information to adjust the video reference direction. This feedback loop enables the system to distinguish between shaking (to be corrected) and intentional motion (to be accommodated by changing reference), thereby resolving the contradiction between correction precision and motion sickness prevention.
3Object-affected harmful factors
If the video reference direction is fixed to the initial front direction throughout image capturing, then motion sickness is suppressed, but the video may not reflect the cameraman's intended viewing direction
Solution Approach 1:
The patent inverts the conventional approach by fixing the video reference direction to a specific direction (initial front direction or traveling direction) rather than allowing it to follow the camera's front direction. This inversion successfully suppresses motion sickness while the video content itself still captures the cameraman's intended viewing direction through the fixed reference frame.
Solution Approach 2:
The patent dynamically adjusts the fixed reference direction based on the cameraman's traveling direction. When the cameraman moves, the system updates the reference direction to follow the traveling direction, maintaining a fixed reference relative to the cameraman's motion. This dynamic adjustment preserves motion sickness suppression while adapting the video to reflect the cameraman's intended viewing direction.
Data Source
AI summary
Provided is an image processing apparatus that processes a video, including continuous images expressed by a coordinate system which includes at least an angular coordinate around a predetermined axis. The image processing apparatus includes: video data acquisition means that acquires video data; sensor data acquisition means that acquires sensor data corresponding to the video data; and rotation correction means that corrects the image to cancel rotation change around the reference axis based on the sensor data so that a reference of display of the image is fixed to a specific direction during the image capturing throughout a plurality of frames of the video.