Extended Reference Frames for 360-Degree Video Border Distortion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding techniques for 360-degree video data face challenges in coding efficiency and distortion due to deformations and discontinuities at the borders between packed faces in cubemap projections, which affect inter-prediction processes.
Innovation Solution
Generating reference frames with extended faces from a cubemap or adjusted cubemap projection of 360-degree video data, where the extended faces are larger than the packed faces, to mitigate distortion and improve coding efficiency by reducing deformations and discontinuities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If standard cubemap projection with packed faces is used for 360-degree video coding, then device complexity and processing requirements are kept manageable, but distortion and coding efficiency deteriorate at the borders between packed faces due to deformations and discontinuities
Solution Approach 1:
The reference frame is segmented into multiple face regions corresponding to the cubemap projection faces. By identifying and separating objects that span across face borders, the system can apply different processing strategies to different segments, improving coding efficiency at face borders without requiring complete restructuring of the entire reference frame
Solution Approach 2:
A border handling mechanism acts as an intermediary between packed faces. The system detects objects near face borders and applies special processing (such as extending reference samples or adjusting motion compensation) to bridge the discontinuities between faces, thereby reducing distortion without fundamentally changing the packed face structure
2Measurement precision
If extended reference frames with larger faces are generated to reduce border distortions, then inter-prediction accuracy improves by keeping objects within the same face, but processing time and computational resources increase
Solution Approach 1:
Instead of extending all reference frame faces uniformly, the system applies face extension or special border handling only to regions where objects are detected near face borders. This partial application of the extended reference frame technique maintains inter-prediction accuracy where needed while avoiding unnecessary processing in regions where standard packing suffices
Solution Approach 2:
The reference frame processing applies different quality levels to different regions: standard packed face processing for most areas, and enhanced processing (such as extended faces or border correction) only locally at regions containing objects near face borders. This local quality approach improves inter-prediction accuracy for affected regions without uniformly increasing processing time across the entire frame
3Device complexity
If objects near face borders are allowed to span multiple packed faces, then device complexity remains low, but distortion increases and coding efficiency deteriorates
Solution Approach 1:
The system performs preliminary detection of objects near face borders during the reference frame processing stage, before actual video coding occurs. By identifying these objects in advance, the system can pre-arrange reference samples or apply border handling techniques, preventing distortion issues from manifesting during encoding without adding complexity during the main processing pipeline
Data Source
AI summary
This disclosure describes techniques for generating reference frames packed with extended faces from a cubemap projection or adjusted cubemap projection of 360-degree video data. The reference frames packed with the extended faces may be used for inter-prediction of subsequent frames of 360-degree video data.