Mixed NAL Unit Picture Constraints for VR Video Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding systems face challenges in efficiently decoding mixed NAL unit pictures, particularly in virtual reality (VR) applications, due to constraints that prevent decoders from starting at non-IRAP pictures, leading to increased latency and file size when dynamic resolution changes are required.
Innovation Solution
Implementing constraints at the sub-picture level to ensure that slices in subsequent pictures do not reference preceding pictures when an IRAP NAL unit is present, allowing decoding to start at IRAP sub-pictures while maintaining inter-prediction integrity for non-IRAP sub-pictures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If video data is compressed to reduce file size and bandwidth usage, then network transmission efficiency is improved, but decoding complexity and processing time increase
Solution Approach 1:
The patent divides a picture into multiple sub-pictures, each with its own NAL unit type (IRAP or non-IRAP). This segmentation allows independent decoding of sub-pictures, reducing overall decoding complexity while maintaining compression efficiency. Each sub-picture can be decoded separately based on its NAL unit type, enabling parallel processing and reducing the burden on the decoder.
2Adaptability or versatility
If IRAP pictures are used to enable random access and break inter-prediction chains, then decoding flexibility and error resilience are improved, but file size and bandwidth consumption increase
Solution Approach 1:
The patent applies different NAL unit types to different sub-pictures within the same picture. Specifically, certain sub-pictures are marked as IRAP while others are non-IRAP. This local differentiation allows the system to provide random access and error resilience only where needed (IRAP sub-pictures) while maintaining compression efficiency in other areas (non-IRAP sub-pictures), thus resolving the contradiction between decoding flexibility and file size.
3Quantity of substance
If sub-pictures with different resolutions are transmitted for VR video, then network resource efficiency is improved, but maintaining inter-prediction integrity across resolutions becomes difficult
Solution Approach 1:
The patent dynamically adjusts the NAL unit type of sub-pictures based on their resolution and decoding requirements. Higher resolution sub-pictures that require random access are marked as IRAP, while lower resolution sub-pictures can be non-IRAP to save bandwidth. This dynamic classification maintains inter-prediction integrity within each resolution level while optimizing network resource usage across multiple resolutions.
Data Source
AI summary
A video coding mechanism is disclosed. The mechanism includes receiving a bitstream comprising a current picture including a plurality of video coding layer (VCL) network abstraction layer (NAL) units that do not have a same NAL unit type. Active entries of reference picture lists are obtained for slices positioned in a sub-picture A (subpicA) in subsequent pictures following the current picture in decoding order. The active entries do not refer to any reference picture preceding the current picture in decoding order when the subpicA at the current picture is associated with an intra-random access point (IRAP) NAL unit type. The subsequent pictures are decoded based on the reference picture list active entries. The subsequent pictures are forwarded for display as part of a decoded video sequence.