360 Video View Optimization via Foveated DCT Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current view optimization methods for 360 degrees video in virtual reality devices suffer from redundancy and latency issues due to independent viewpoint arrays, leading to inefficient storage and longer latency when switching views.
Innovation Solution
A method and system that applies a foveated region of interest to 360 degrees video frames, using Discrete Cosine Transform (DCT) coefficients and view adaptive filtering and quantization based on the foveated region of interest to optimize video encoding, reducing redundancy and latency by prioritizing areas within the field of view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional view optimization methods are used with independent viewpoint arrays, then video quality is preserved, but storage redundancy increases and view switching latency increases
Solution Approach 1:
The patent merges multiple independent viewpoint arrays into a unified hierarchical structure where a base viewpoint array shares common information with super-viewpoint arrays. This combining approach eliminates redundant storage while maintaining video quality across different viewpoints.
Solution Approach 2:
The patent implements a nested hierarchical structure where super-viewpoint arrays contain and share information with base viewpoint arrays. The base array serves as a foundation that is embedded within and reused by multiple super-viewpoint arrays, reducing overall storage requirements while preserving quality.
2Reliability
If traditional view optimization methods are used with independent viewpoint arrays, then video quality is preserved, but view switching latency increases
Solution Approach 1:
The patent performs preliminary encoding of a base viewpoint array that contains common information reusable across multiple viewpoints. This advance preparation allows super-viewpoint arrays to be generated quickly by applying incremental transformations to the pre-encoded base array, reducing view switching latency.
Solution Approach 2:
The nested hierarchical structure enables efficient view switching by allowing the system to navigate between base and super-viewpoint arrays. When switching views, the system can leverage the pre-encoded base array and apply minimal transformations, significantly reducing latency compared to processing independent viewpoint arrays.
3Reliability
If 360 degrees video is stored in full resolution, then video quality is maintained, but bandwidth consumption and storage requirements increase significantly
Solution Approach 1:
The patent applies local quality optimization by encoding different viewpoints with different quality levels based on their importance. The base viewpoint array maintains high quality while super-viewpoint arrays use compressed representations, allocating bandwidth efficiently according to local quality requirements rather than uniformly across all viewpoints.
Solution Approach 2:
By merging common information into a single base viewpoint array that serves multiple super-viewpoints, the patent dramatically reduces total bandwidth consumption. Instead of transmitting redundant high-resolution data for each viewpoint, the system transmits the shared base array once and derives other viewpoints through efficient transformations.
Data Source
AI summary
A method and system for view optimization of a 360 degrees video is provided. The method includes generating two-dimensional video frame from the 360 degrees video. The macroblocks are generated for the two-dimensional video frame. A foveated region of interest for the two-dimensional video frame is defined based on a given view orientation. DCT (Discrete Cosine Transform) coefficients are generated for the macroblocks. View adaptive DCT domain filtering is then performed on the DCT coefficients using the foveated region of interest. Quantization offset is calculated for the DCT coefficients using the foveated region of interest. The DCT coefficients are quantized using the quantization offset to generate encoded two-dimensional video frame for the view orientation. A new view orientation is then set as the given view orientation and steps of generating, performing, calculating, and quantizing are performed for each view orientation and each video frame to generate view optimized video.