Composite Track Composition for Video Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for deriving composite tracks in video coding do not provide robust track composition, particularly in handling gaps and overlaps between sub-picture tracks, and fail to associate metadata with entire 3D content, leading to inefficient bandwidth usage and user experience issues in viewport-dependent processing.
Innovation Solution
The development of methods and apparatus for encoding and decoding video data that involve composing multiple two-dimensional sub-picture tracks into a canvas, with metadata applied to track groups to specify composition operations, including gap filling and blending, to optimize viewport-dependent processing and reduce bandwidth consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If viewport-dependent processing is implemented to reduce bandwidth consumption, then bandwidth efficiency is improved, but robust track composition and gap/overlap handling are insufficient
Solution Approach 1:
The patent segments the video content into multiple sub-picture tracks, each representing a portion of the spherical content. This segmentation allows selective delivery of only the viewport-relevant sub-pictures, reducing bandwidth consumption while maintaining the ability to compose complete views at the decoder through track composition operations.
Solution Approach 2:
The patent introduces composition operations as intermediary processing steps that combine multiple sub-picture tracks into a complete viewport representation. These composition operations include gap filling and overlap handling mechanisms that serve as mediators to ensure robust track composition and eliminate the harmful effects of incomplete or gapped sub-picture deliveries.
2Reliability
If composition operations are added to handle gaps and overlaps, then track composition robustness is improved, but encoding complexity increases
Solution Approach 1:
The patent applies composition operations and metadata annotations during the encoding phase to pre-establish the composition rules, gap filling strategies, and overlap handling mechanisms. This preliminary action ensures that the decoding process can efficiently compose tracks without requiring complex real-time processing, thereby improving track composition robustness while managing encoding complexity.
3Productivity
If metadata is applied to track groups for composition operations, then composition efficiency is improved, but processing overhead increases
Solution Approach 1:
The patent merges multiple sub-picture tracks into composite tracks with associated metadata that describes composition operations. This merging approach improves composition efficiency by pre-processing and organizing track data with composition instructions, allowing the decoder to efficiently reconstruct complete views. The metadata overhead is managed by consolidating composition information at the track group level rather than individual track level.
Data Source
AI summary
The techniques described herein relate to methods, apparatus, and computer readable media configured to process (including encoding and decoding) composition layouts. The video data includes a plurality of encoded two-dimensional sub-picture tracks associated with a viewport and a composition operation to compose the tracks. The composition operation comprises data indicative of a composition to perform on the plurality of two-dimensional sub-picture tracks to compose the tracks into a canvas associated with the viewport, and a composition layout operation to adjust the composition if the canvas comprises a composition layout created by two or more of the plurality of two-dimensional sub-picture tracks composed on the canvas. The plurality of two-dimensional tracks are composed into the canvas according to the composition, comprising determining two or more of the composed two-dimensional sub-picture tracks comprise the composition layout, and adjusting the composition based on the composition layout operation to compensate for the composition layout.