VR Sequence Coding via 2D Block Structure and Reference Reorientation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards, such as AVC and HEVC, face inefficiencies when encoding Virtual Reality (VR) sequences due to the complexity of VR formats like cube projection, Equal Rectangular Projection, and Equal Area Projection, which require improved reference lists and indexing to account for the unique temporal and spatial differences in VR pictures.
Innovation Solution
The method organizes VR sequences as a single 2D block structure, creating extended reference and subpicture lists, allowing for reorientation of reference subpictures to match VR subpictures, and uses motion vector prediction blocks to enhance coding efficiency by indexing subpictures based on temporal and spatial distances, enabling efficient encoding and decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing video coding standards (AVC/HEVC) are used to code VR sequences, then the coding process can be implemented with standard tools, but the coding efficiency is insufficient due to the complexity of VR formats and the need to handle temporal and spatial differences in cube projection, ERP, and EAP formats
Solution Approach 1:
The patent divides the VR picture into multiple subpictures (e.g., 6 subpictures for cube projection) and creates separate reference subpicture lists for each. This segmentation allows independent handling of each subpicture's reference relationships, simplifying the overall coding process while maintaining efficiency for complex VR formats
Solution Approach 2:
The patent introduces reference subpicture lists as an intermediary structure between the current picture and reference pictures. These intermediate lists store processed reference subpictures with proper orientation, enabling efficient motion estimation and compensation without directly handling the complex transformations between different VR projection formats
2Productivity
If reference pictures are used for motion estimation and compensation in VR sequences, then temporal redundancy can be removed, but the reference pictures may have improper orientation compared to current VR pictures, requiring additional processing
Solution Approach 1:
The patent performs preliminary orientation adjustment of reference subpictures before they are stored in the reference subpicture lists. By pre-rotating and reorienting reference subpictures to match the current picture's orientation, the system eliminates the need for complex real-time orientation matching during motion estimation, improving both efficiency and accuracy
3Ease of manufacture
If VR sequences are encoded using standard 2D video coding methods, then the implementation can be straightforward, but the unique spatial and temporal characteristics of VR formats (cube projection, ERP, EAP) are not adequately handled
Solution Approach 1:
The patent creates a universal coding framework that can handle multiple VR projection formats (cube projection, ERP, EAP) through the same reference subpicture list mechanism. The system universally applies subpicture segmentation and orientation adjustment across all VR formats, maintaining implementation simplicity while accurately handling format-specific characteristics through configurable parameters
Data Source
AI summary
An improved method for coding video is provided that includes Virtual Reality (VR) sequences that enables more efficient encoding by organizing the VR sequence as a single 2D block structure. In the method, reference picture and subpicture lists are created and extended to account for coding of the VR sequence. To further improve coding efficiency, reference indexing can be provided for the temporal and spatial difference between a current VR picture block and the reference pictures and subpictures for the VR sequence. Further, because the reference subpictures for the VR sequence may not have the proper orientation once the VR sequence subpictures are organized into the VR sequence, reorientation of the reference subpictures is made so that the reference subpicture orientations match the current VR subpicture orientations.