Remapping Sample Locations in Projection-Based Video Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video processing technologies face challenges in efficiently remapping sample locations from a projection-based frame on a two-dimensional plane to a three-dimensional sphere in virtual reality applications, particularly with high-resolution 360-degree content, leading to bitrate issues and artifacts due to discontinuous layout boundaries and face edges in projection layouts.
Innovation Solution
A method and apparatus for decoding a bitstream to generate a projection-based frame and remapping sample locations using a conversion circuit, which adjusts and converts local sample locations within projection faces to accurate locations on a sphere, utilizing guard bands and syntax elements to correct for layout discontinuities and edge artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If high-resolution 360-degree content is used to improve visual quality and immersion, then the field of view and image quality are improved, but the bitrate requirement increases significantly
Solution Approach 1:
The 360-degree video is divided into multiple projection faces (e.g., front, back, left, right, top, bottom) arranged in a specific layout. Each face is processed and encoded separately, allowing selective transmission and decoding. This segmentation enables the system to maintain high image quality while reducing the effective bitrate by only transmitting necessary portions of the omnidirectional content.
2Area of moving object
If projection layout with multiple faces is used to represent 360-degree content, then the field of view is improved, but layout discontinuities and edge artifacts occur at boundaries
Solution Approach 1:
Guard bands are introduced as intermediary regions between adjacent projection faces. These guard bands act as transition zones that smooth the boundaries between faces, reducing visible discontinuities and edge artifacts. The guard bands are specifically designed to blend the projection content seamlessly, making the transitions between different viewing directions less noticeable to the user.
3Device complexity
If sample locations are directly mapped from projection frame to sphere without adjustment, then the processing complexity is reduced, but mapping accuracy and rendering quality deteriorate
Solution Approach 1:
Different adjustment operations are applied to different regions (projection faces) based on their specific geometric characteristics and positions in the projection layout. Each face receives customized sample location adjustments tailored to its local geometry, ensuring accurate mapping back to the sphere while maintaining manageable processing complexity through localized rather than global adjustments.
Data Source
AI summary
A video processing method includes: decoding apart of a bitstream to generate a decoded frame, where the decoded frame is a projection-based frame that includes projection faces in a projection layout; and remapping sample locations of the projection-based frame to locations on the sphere, where a sample location within the projection-based frame is converted into a local sample location within a projection face packed in the projection-based frame; in response to adjustment criteria being met, an adjusted local sample location within the projection face is generated by applying adjustment to at least one coordinate value of the local sample location within the projection face, and the adjusted local sample location within the projection face is remapped to a location on the sphere; and in response to the adjustment criteria not being met, the local sample location within the projection face is remapped to a location on the sphere.