Mixed NAL Unit Type Constraints for VR Sub-Picture Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding systems struggle to efficiently handle mixed intra-random access point (IRAP) and non-IRAP sub-pictures within a single picture, leading to inefficiencies in bandwidth and resource usage, particularly in virtual reality applications where different sub-pictures require varying resolutions.
Innovation Solution
Implementing a flag, referred to as mixed_nalu_types_in_pic_flag, to indicate whether a picture contains both IRAP and non-IRAP sub-pictures, constraining the picture to exactly two NAL unit types, allowing decoders to treat sub-pictures differently based on their types, thereby enabling dynamic resolution changes without impairing user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sub-pictures are coded into separate sub-bitstreams with different resolutions for VR applications, then coding efficiency and bandwidth usage are improved, but the complexity of handling mixed NAL unit types in a single picture increases
Solution Approach 1:
The picture is segmented into multiple sub-pictures, each coded into separate sub-bitstreams with different resolutions. This allows different sub-pictures to be transmitted at appropriate quality levels, improving coding efficiency while the flag mechanism segments the complexity of handling them.
Solution Approach 2:
A flag parameter (mixed_nalu Types in pic flag) is introduced to indicate whether a picture contains both IRAP and non-IRAP sub-pictures. This parameter change enables decoders to adapt their behavior based on the picture type, managing complexity through informed decision-making rather than handling all cases uniformly.
2Reliability
If IRAP pictures are transmitted for all sub-pictures to ensure decoding capability, then reliability of video decoding is improved, but bandwidth consumption increases
Solution Approach 1:
Different sub-pictures are assigned different qualities (resolutions) based on their importance and decoding requirements. IRAP sub-pictures maintain full quality for reliable decoding, while non-IRAP sub-pictures can use lower resolutions, optimizing the balance between reliability and bandwidth usage.
Solution Approach 2:
Instead of transmitting IRAP pictures for all sub-pictures (excessive action), the system transmits IRAP pictures only where necessary (partial action), specifically for sub-pictures that require independent decoding capability, thereby reducing unnecessary bandwidth consumption.
3Adaptability or versatility
If a flag is added to indicate mixed NAL unit types, then adaptability of the video coding system is improved, but the bitstream complexity increases
Solution Approach 1:
The flag acts as an intermediary element that mediates between the complexity of mixed NAL unit types and the need for clear picture identification. By introducing this intermediate indicator, the system achieves better adaptability without requiring complex direct encoding of all picture type combinations.
Data Source
AI summary
A video coding mechanism is disclosed. The mechanism includes receiving a bitstream comprising a flag and a plurality of sub-pictures associated with a picture. The plurality of sub-pictures are contained in a plurality of video coding layer (VCL) network abstraction layer (NAL) units. The mechanism determines that VCL NAL units of one or more of the sub-pictures of the picture all have a first particular value of NAL unit type and other VCL NAL units in the picture all have a different second particular value of NAL unit type based on a value of the flag. One or more of the sub-pictures are decoded based on the first particular value of NAL unit type and the second particular value of NAL unit type. One or more of the sub-pictures are forwarded for display as part of a decoded video sequence.