Pyramid Projection Format for 360-Degree VR Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image/video processing technologies face challenges in efficiently processing and storing 360-degree virtual reality images due to distortions introduced by projection methods like equirectangular projection, which affect coding efficiency, especially near the poles.
Innovation Solution
The use of a viewport-based pyramid projection format that includes a primary viewport face and four secondary viewport faces, with parameters such as yaw, pitch, FOV angle, and non-uniform factors being signaled through syntax elements to enable efficient encoding and decoding of 360-degree virtual reality images, allowing for reduced bandwidth and improved image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If equirectangular projection is used to convert 360-degree VR images to 2D format, then the images can be stored in a flat format, but severe stretching and distortion occur near the north and south poles
Solution Approach 1:
The patent divides the 360-degree spherical image into multiple planar face projections (typically 6 faces corresponding to the faces of a cube or similar polyhedron). Each face captures a specific region of the sphere, avoiding the severe distortion that occurs at the poles in equirectangular projection. This segmentation allows each face to be stored and processed as a separate 2D image with minimal distortion.
Solution Approach 2:
The patent transitions from a 2D equirectangular representation to a 3D-like multi-face projection structure that can be flattened into 2D for storage. By using multiple planar faces arranged in a specific geometry (such as a cube net), the system preserves spatial relationships and reduces distortion while maintaining 2D storage compatibility.
2Productivity
If equirectangular projection is used, then the entire spherical surface can be mapped to a flat image, but predictive coding tools fail to make good prediction due to distortions
Solution Approach 1:
By segmenting the spherical image into multiple planar faces, each face can be independently processed with predictive coding tools. The local planar geometry of each face maintains better spatial coherence and reduces distortion artifacts, allowing predictive coding algorithms to make more accurate predictions and achieve higher coding efficiency.
3Manufacturing precision
If pyramid projection format with multiple viewport faces is used, then image quality is improved, but device complexity increases
Solution Approach 1:
The patent designs a universal pyramid projection framework that can handle multiple viewport faces using a consistent set of syntax elements and processing rules. The same encoding and decoding mechanisms are applied to each face, making the system multi-functional while maintaining manageable complexity through standardization.
Solution Approach 2:
The patent introduces a set of parameters (such as face orientation angles, viewport positions, and projection factors) that can be signaled in the bitstream to define the specific geometry of the pyramid projection. By changing these parameters, the system can adapt to different viewport configurations and image qualities without requiring fundamentally different processing approaches.
Data Source
AI summary
According to one method, at a source side or an encoder side, a selected viewport associated with the 360-degree virtual reality images is determined. One or more parameters related to the selected pyramid projection format are then determined. According to the present invention, one or more syntax elements for said one or more parameters are included in coded data of the 360-degree virtual reality images. The coded data of the 360-degree virtual reality images are provided as output data. At a receiver side or a decoder side, one or more syntax elements for one or more parameters are parsed from the coded data of the 360-degree virtual reality images. A selected pyramid projection format associated with the 360-degree virtual reality images is determined based on information including said one or more parameters. The 360-degree virtual reality images are then recovered according to the selected viewport.