360-Degree Video Stabilization via Probability-Based FOV Rotation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for stabilizing 360-degree videos fail to effectively address user discomfort due to contradictory sensory inputs, leading to VR sickness, as they cannot stabilize videos with diverse moving elements within the same scene.
Innovation Solution
An electronic device allocates probability values to pixels in a 360-degree video based on their likelihood of being in the user's field of view, determines a 3D rotation, and applies this rotation to generate a stabilized video, using features like object type, depth, visual importance, and motion vectors, and optionally employs machine learning for improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If traditional video stabilization techniques are applied to 360-degree videos, then camera shaking is reduced, but user discomfort and VR sickness persist due to inability to stabilize with respect to all moving parts in the scene
Solution Approach 1:
The patent applies different stabilization strategies to different regions of the 360-degree video based on their likelihood of being in the user's field of view. High-probability regions (likely to be viewed) receive stronger stabilization, while low-probability regions receive less processing. This local differentiation resolves the contradiction by optimizing stability where it matters most to the user while avoiding over-processing that causes discomfort in peripheral regions.
Solution Approach 2:
The patent changes the stabilization parameters dynamically based on probability values assigned to different pixels or regions. By adjusting stabilization strength according to the predicted field of view probability, the system achieves optimal balance between reducing camera shake and maintaining natural scene motion, thereby reducing VR sickness while preserving video stability.
2Stability of the object's composition
If video stabilization is applied to all parts of the scene, then comprehensive stability is achieved, but processing complexity and computational load increase significantly
Solution Approach 1:
The patent applies stabilization selectively rather than uniformly across the entire scene. By focusing computational resources on high-probability regions (those most likely to be in the user's field of view), the system achieves sufficient overall stability without the excessive processing burden of stabilizing every pixel equally. This partial action approach resolves the contradiction between comprehensive stability and processing complexity.
Solution Approach 2:
The patent segments the 360-degree video into multiple regions with different probability values indicating likelihood of being in the user's field of view. Each segment is then processed with appropriate stabilization strength, allowing the system to manage computational complexity by dividing the problem into manageable regions rather than processing the entire scene uniformly.
3Stability of the object's composition
If strong stabilization is applied to stabilize camera movements, then shaking is reduced, but natural motion of scene elements is distorted
Solution Approach 1:
The patent applies different stabilization strengths to different regions based on their probability of being in the user's field of view. High-probability regions receive stronger stabilization to reduce camera shake, while low-probability regions maintain more natural motion characteristics. This local differentiation resolves the contradiction by reducing distortion in regions where users are unlikely to notice while maintaining stability in important regions.
Data Source
AI summary
Provided is an electronic device for stabilizing a 360-degree video, the electronic device including a memory storing one or more instructions, and a processor for executing the one or more instructions stored in the memory, wherein the processor is configured to execute the one or more instructions to: when a 360-degree video is reproduced, allocate probability values to a plurality of pixels included in a frame of the 360-degree video, based on a possibility that each of the plurality of pixels is included in a user's field of view (FOV), determine a three-dimensional (3D) rotation for the 360-degree video, based on the allocated probability values, and generate a stabilized 360-degree video by applying the 3D rotation to the 360-degree video.