Video Bitstream Filler Data Removal for Sub-Bitstream Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards face challenges in efficiently managing bandwidth and processing complexity due to the increasing demand for digital video, particularly in multi-layer video coding, where subpictures, tiles, and slices are not adequately addressed, leading to inefficiencies in parallel processing and MTU size matching.
Innovation Solution
The proposed solution involves format rules for video processing that specify the order of messages within access units, allow or disallow certain types of NAL units, and define syntax elements for in-loop filtering, synchronization, and sub-bitstream extraction, ensuring efficient decoding and encoding processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If digital video bandwidth demand increases to accommodate more connected user devices, then video delivery capability improves, but network bandwidth consumption increases
Solution Approach 1:
The video bitstream is segmented into multiple Network Abstraction Layer (NAL) units, each carrying specific video data or filler data. This segmentation allows selective transmission and efficient bandwidth utilization by dividing the video stream into manageable units that can be independently processed and transmitted over the network.
Solution Approach 2:
The patent introduces parameter changes by modifying the bitstream structure to include filler data units with specific NAL unit types and by implementing rules for removing filler data during sub-bitstream extraction. These parameter changes optimize bandwidth usage by controlling the amount and type of data transmitted while maintaining video decoding capability.
2Adaptability or versatility
If video coding standards are enhanced to support multi-layer video coding with subpictures, tiles, and slices, then video processing flexibility improves, but processing complexity increases
Solution Approach 1:
The video picture is divided into subpictures, which are further divided into tiles and slices. This hierarchical segmentation provides processing flexibility by allowing independent encoding and decoding of different regions, enabling parallel processing and reducing the computational burden on single processing units while maintaining overall video quality.
Solution Approach 2:
The patent introduces a new dimension of organization by implementing a multi-layer structure with subpictures, tiles, and slices. This dimensional expansion allows video processing to occur at multiple levels simultaneously, providing flexibility in resource allocation and processing strategies while managing complexity through structured organization.
3Adaptability or versatility
If filler data units are included in video bitstreams for sub-bitstream extraction, then bitstream compatibility improves, but decoding efficiency decreases
Solution Approach 1:
Filler data units act as intermediaries in the bitstream, providing necessary structural elements for sub-bitstream extraction and compatibility with different decoding scenarios. These filler units serve as placeholders that maintain bitstream integrity while allowing selective removal during extraction processes, ensuring compatibility across different video coding applications.
Solution Approach 2:
The patent applies local quality optimization by selectively removing filler data units from specific NAL units during sub-bitstream extraction based on local requirements. This allows the decoding process to efficiently handle only the necessary data for the current application, improving decoding efficiency while maintaining bitstream compatibility where needed.
Data Source
AI summary
A method of video processing includes performing a conversion between a video and a bitstream of the video according to a rule. The rule specifies that a sub-bitstream extraction process is implemented to generate a sub-bitstream for decoding. The sub-bitstream extraction process is configured to extract, from the bitstream, a sub-bitstream with a target highest temporal identifier. The rule specifies that, during the extracting, upon removing a video coding layer (VCL) network abstraction layer (NAL) unit, filler data units and filler supplemental enhancement information (SEI) messages in SEI NAL units that are associated with the VCL NAL unit are also removed.