Layer-Based 360-Degree Video Encoding for View Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for encoding 360-degree videos for virtual reality (VR) devices are inefficient, leading to resource wastage and increased latency due to treating all viewing angles equally and requiring multiple view ports, which results in redundancy and high storage needs.
Innovation Solution
A layer-based view optimization encoding method that generates a first layer with full spherical view coverage and sub-layers with view-dependent coverage, using temporal sub-sampling and 2D encoding techniques to optimize bitrate and reduce redundancy among view ports.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple independent view ports are created for smooth view switching, then view switching quality is improved, but storage requirements and device complexity increase significantly
Solution Approach 1:
The patent segments the video encoding into multiple layers: a base layer containing common information for all views, and multiple enhancement layers each containing view-specific information. This segmentation allows smooth view switching while avoiding the need to store complete independent copies of all view ports, thereby reducing storage requirements and device complexity.
Solution Approach 2:
The patent implements a nested layer structure where enhancement layers are built upon the base layer. The base layer contains I-frames and common P-frames that are nested within all view ports, while enhancement layers add view-specific details. This nesting allows efficient view switching by reusing the nested base layer information across multiple views.
2Ease of manufacture
If traditional 2D encoding methods are used for 360-degree video, then encoding simplicity is maintained, but bitrate efficiency deteriorates due to equal treatment of all viewing angles
Solution Approach 1:
The patent applies local quality by encoding different portions of the 360-degree video at different quality levels. The base layer provides uniform quality across all views, while enhancement layers provide higher quality for specific view ports. This allows bitrate-efficient encoding by allocating more bits to important views while maintaining acceptable quality for all views.
Solution Approach 2:
The patent transitions from traditional 2D encoding to a multi-layered encoding structure that adds a temporal and hierarchical dimension. By organizing frames into I-frames, P-frames, and B-frames across multiple layers, the system achieves better bitrate efficiency while maintaining encoding feasibility through structured complexity.
3Manufacturing precision
If large I-frames are used in each view port to ensure quality, then video quality is improved, but encoding time and latency increase
Solution Approach 1:
The patent performs preliminary encoding by creating the base layer with essential frame information first. This base layer contains the fundamental structure and common information that can be reused across all view ports. Subsequently, enhancement layers are added more efficiently, avoiding the need to encode large I-frames independently for each view port, thereby reducing encoding time and latency.
Solution Approach 2:
The patent discards redundant information by not encoding complete independent I-frames for each view port. Instead, it recovers the necessary quality information through enhancement layers that reference the base layer. This approach maintains video quality while significantly reducing the encoding time and computational resources required.
Data Source
AI summary
A method and system for layer based encoding of a 360 degrees video is provided. The method includes receiving, by a server, an input video. The input video includes multiple groups of pictures (GOPs). Each GOP starts from a major anchor frame of the input video and includes frames till next major anchor frame. The method also includes generating a first layer. The first layer includes one encoded frame per GOP. The method further includes generating a first sub-layer. The first sub-layer includes encoded frames of multiple mini-GOPs and reconstructed frames of encoded frames of the first layer. Each mini-GOP includes frames between two major anchor frames. Furthermore, the method includes outputting encoded video including the first layer and the first sub-layer.