Partial Decoding for 360VR Video Line Buffer Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video encoding and decoding technologies for 360-degree virtual reality (360VR) videos waste computing resources by decoding entire frames, even though users typically view only a small sub-region, leading to inefficiencies in resource utilization.
Innovation Solution
The implementation of partial decoding methods that identify and decode only the necessary data units and pixel blocks for the selected view region, bypassing unnecessary data, and optimizing video encoding to maximize performance by minimizing intra-predicted blocks in slave frames and using inter-prediction, while rearranging 360VR video frames into narrower formats for efficient line buffer usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If entire 360VR video frames are decoded, then complete view region is available, but computing resource consumption increases
Solution Approach 1:
The patent divides the 360VR video frame into multiple cubic faces (typically 6 faces arranged in a cube projection), allowing selective decoding of only those faces that correspond to the user's current view region. This segmentation enables the system to process partial frame data rather than complete frames, reducing computing resource consumption while maintaining view quality for the displayed region.
Solution Approach 2:
The patent extracts and decodes only the necessary data units and pixel blocks that correspond to the selected view region, bypassing unnecessary data in other regions. This extraction principle allows the decoder to focus computational resources solely on the relevant portion of the video frame, directly reducing energy consumption while preserving view quality.
2Ease of manufacture
If cubic faces are arranged in wide layout, then encoding simplicity is maintained, but line buffer width increases
Solution Approach 1:
The patent rearranges the cubic faces from a traditional wide horizontal layout into a vertical columnar arrangement (or other compact configurations). This dimensional reorganization transforms the spatial arrangement of video data, allowing the same 6 cubic faces to be stored and processed in a narrower format that reduces line buffer width requirements while maintaining encoding simplicity through systematic face ordering.
3Adaptability or versatility
If more intra-predicted blocks are used in slave frames, then encoding flexibility increases, but decoding performance decreases
Solution Approach 1:
The patent applies different prediction strategies to different regions of the video frame based on their importance to the view region. Slave frames that contain data for the current view region use inter-prediction with reference to master frames, while regions outside the view region can use intra-prediction. This local quality approach optimizes decoding performance for critical regions while maintaining encoding flexibility overall.
Solution Approach 2:
The patent prepares master frames in advance with full decoding and high quality, which then serve as reference frames for subsequent slave frames. This preliminary action allows slave frames to use efficient inter-prediction modes when needed, improving decoding performance by leveraging pre-computed reference data rather than requiring complex intra-prediction calculations during real-time decoding.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Apparatus and methods are disclosed for partially decoding video frames when a sub-region of the video is selected for viewing. The method identifies and decodes data units and pixel blocks of video frames needed to display the sub-region while bypassing data units and pixel blocks that are identified as unnecessary for displaying the sub-region. A video encoder receives a video frame comprising a plurality of cubic surfaces in a first configuration corresponding to a full sized 360VR image. Each cubic surface corresponds to a different surface of a cube. The encoder reformats the received video frame by rearranging the plurality of cubic surfaces according to a second configuration that is different than the first configuration. The second configuration re-arranges the six surfaces of a cubic 360VR image in order to fully utilize the line buffer and allow the line buffer to be narrower than the full sized 360VR image.