Mixed NAL Unit Sub-Pictures for VR Dynamic Resolution Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding systems struggle with efficiently handling mixed intra-random access point (IRAP) and non-IRAP sub-pictures in virtual reality (VR) applications, leading to issues with decoding and dynamic resolution changes due to inconsistent NAL unit types within a single picture.
Innovation Solution
Implementing a flag, referred to as mixed_nalu_types_in_pic_flag, to indicate whether a picture contains both IRAP and non-IRAP sub-pictures, allowing decoders to treat these sub-pictures differently during decoding, and supporting dynamic resolution changes in VR video streaming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a picture contains both IRAP and non-IRAP sub-pictures to enable dynamic resolution changes, then coding efficiency is improved, but decoder compatibility deteriorates because some video systems cannot handle mixed pictures
Solution Approach 1:
The patent applies local quality by allowing different sub-pictures within the same picture to have different NAL unit types (IRAP or non-IRAP) based on their specific requirements. Each sub-picture can be independently coded with appropriate NAL unit type, enabling dynamic resolution changes in certain regions while maintaining compatibility in others. The mixed_nalu_types_in_pic_flag signals this local differentiation to the decoder.
2Quantity of substance
If non-IRAP pictures are used to reduce data size, then network and memory resources are reduced, but decoding complexity increases because non-IRAP pictures require reference to other pictures for decoding
Solution Approach 1:
The patent applies dynamics by dynamically selecting the NAL unit type for each sub-picture based on its position and importance. IRAP sub-pictures are used in regions that require independent decoding capability, while non-IRAP sub-pictures are used in regions where reference to other pictures is acceptable, creating a dynamic balance between data size and decoding complexity throughout the picture.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A video coding mechanism is disclosed. The mechanism includes receiving a bitstream comprising a plurality of sub-pictures associated with a picture and a flag. The sub-pictures are contained in a plurality of video coding layer (VCL) network abstraction layer (NAL) units. A NAL unit type value is the same for all VCL NAL units associated with the picture when the flag is set to a first value. When the flag is set to a second value, a first NAL unit type value for VCL NAL units containing one or more of the sub-pictures of the picture is different than a second NAL unit type value for VCL NAL units containing one or more of the sub-pictures of the picture. The sub-pictures are decoded based on the NAL unit type values. The sub-pictures are forwarded for display as part of a decoded video sequence.