Mixed NAL Coded Pictures Using Subpicture Track Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in efficiently handling mixed Network Abstraction Layer (NAL) unit types within a coded picture, leading to inefficiencies in decoding and bitstream processing.
Innovation Solution
The implementation of an apparatus that utilizes processors and memory to manage mixed NAL unit types by assigning group identifiers, merging video base tracks, and parsing sample group descriptions to handle subpicture tracks, allowing for the reconstruction of coded pictures with mixed NAL unit types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If video coding systems use mixed NAL unit types within a coded picture, then the versatility and adaptability of video coding is improved, but the decoding complexity and processing difficulty increase
Solution Approach 1:
The coded picture is divided into multiple subpictures, each containing slices with uniform NAL unit types. This segmentation allows the decoder to process each subpicture independently with simplified logic, while the overall system maintains versatility through support for multiple NAL unit types across different subpictures. The extraction reference map further segments the reconstruction process by organizing references from multiple tracks.
Solution Approach 2:
An extraction reference map is introduced as an intermediary data structure that bridges multiple video base tracks and the final reconstructed picture. This map contains extraction references that point to specific subpictures or slices from different tracks, allowing the decoder to efficiently navigate and combine data from multiple sources without increasing fundamental decoding complexity.
2Productivity
If multiple video base tracks are merged to form a coded picture, then the productivity and efficiency of video processing is improved, but the bitstream processing complexity increases
Solution Approach 1:
The extraction reference map is prepared in advance during bitstream parsing, organizing all necessary extraction references before the actual picture reconstruction begins. This preliminary organization allows the merging of multiple video base tracks to proceed efficiently without ad-hoc processing complexity during reconstruction.
Solution Approach 2:
The extraction reference map contains nested structures where extraction references point to subpictures or slices that themselves contain further nested data structures. This nesting allows multiple video base tracks to be merged through hierarchical reference resolution, improving processing efficiency by avoiding flat, complex cross-referencing.
3Loss of time
If extraction references are used to extract NAL unit data by reference from another track, then the loss of time in data retrieval is reduced, but the difficulty of detecting and measuring increases
Solution Approach 1:
Instead of directly accessing and parsing complex track reference structures during reconstruction, the extraction reference map stores copied essential information (track IDs, subpicture parameters, slice information) in a simplified format. This copying approach reduces data retrieval time while avoiding the complexity of real-time track reference parsing.
Data Source
AI summary
An apparatus comprising: at least one processor; and at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: indicate a mixed network abstraction layer unit type pictures sample group used to merge video base tracks having a subpicture track; indicate a sample group description entry of the mixed network abstraction layer unit type pictures sample group, the sample group description entry indicating a group of pairs of mixed network abstraction layer unit type track reference indices, which reference video subpicture tracks or track groups; and wherein when a video bitstream is resolved from a video base track containing a mixed network abstraction layer unit type sample group with merging pairs of video subpicture tracks signaled in a mixed network abstraction layer unit type pictures sample group entry, then there is mixing of different network abstraction layer unit types.