Spherical Video Cube Mapping for Low-Distortion VR Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The encoding of 360-degree virtual reality content in an equirectangular format results in severe distortion of straight lines, leading to increased data requirements and bandwidth challenges, especially in stereoscopic formats, which complicates efficient streaming and rendering.
Innovation Solution
A pre-processing technique that converts equirectangular data into a cube map format, arranging face images on a sheet to minimize distortion, adding gaps between edges, and using mip-maps to avoid artifacts during rendering, allowing for efficient encoding and decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If equirectangular format is used to encode 360-degree VR content, then the spherical content can be represented in a rectangular 2D image, but severe distortion of straight lines occurs and data requirements increase
Solution Approach 1:
The patent divides the spherical 360-degree content into multiple face images that are arranged in a cube map format. This segmentation approach allows each face to be encoded separately with reduced distortion compared to the entire sphere projected in equirectangular format, while maintaining compatibility with standard rectangular video encoders.
Solution Approach 2:
The patent transitions from representing spherical content in a 2D equirectangular projection to organizing it as a 3D cube map with six faces. This dimensional change allows for more efficient encoding by treating each face as a separate planar surface, reducing the severe distortion inherent in flat equirectangular projections while still fitting within standard video encoding frameworks.
2Adaptability or versatility
If equirectangular format is used, then spherical content can be displayed, but bandwidth requirements increase making streaming challenging
Solution Approach 1:
By segmenting the spherical content into six cube map faces, the patent enables more efficient compression. Each face can be encoded independently with standard video codecs, allowing for better bandwidth optimization compared to encoding the entire equirectangular projection as a single distorted image, while still providing complete 360-degree coverage.
3Reliability
If stereoscopic format is used for VR, then 3D viewing experience is improved, but image data for each eye needs to be encoded and transmitted doubling the bandwidth requirement
Solution Approach 1:
The patent combines the cube map faces for both left and right eyes into a single encoded video stream. By organizing the stereoscopic content in a unified cube map structure, the system can encode and transmit both eyes' data together, reducing bandwidth requirements compared to transmitting separate stereoscopic streams, while maintaining high-quality 3D viewing experience.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
In one embodiment, a method receives spherical content for video and generates face images from the spherical content to represent an image in the video. A two dimensional sheet for the face images is generated. A size of the face images is reduced and a pixel frame around each of the plurality of face images is added on the sheet. Also, a plurality of gaps are added on the sheet in between edges of the face images that are neighboring. The method then adds gap content in the plurality of gaps where the gap content is based on content in an area proximate to the plurality of gaps. The method encodes the face images, the pixel frame, and gap content on the sheet and sends the encoded sheet to a decoder. The face images are decoded for placement on an object structure to display the spherical content.