PES Header Modification for Scalable Video Compatibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current HEVC scalable specification requires the AVC or MPEG-2 base layer to be available to legacy decoders without the special NAL header, which is not supported in existing transport streams, limiting the compatibility and decoding capabilities of scalable video streams.
Innovation Solution
The method involves encoding video data into packetized elementary streams (PES) by adding a constant-length NAL unit header from the enhancement layer to the base layer AUs, modifying the PES header to include this length, and mapping these AUs into transport streams with specific PIDs and program map tables, allowing legacy decoders to decode the base layer without the enhancement layer headers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the base layer is encoded with enhancement layer NAL headers, then the scalable video stream can be decoded by advanced decoders, but legacy decoders cannot decode the base layer without these headers
Solution Approach 1:
The video stream is segmented into base layer AUs and enhancement layer AUs, with separate handling of NAL headers for each layer type. Base layer AUs are processed with modified PES headers that exclude enhancement layer NAL headers, while enhancement layer AUs retain their original headers, allowing different decoder types to process only the layers they support.
Solution Approach 2:
A modified PES header structure acts as an intermediary between the base layer AU data and legacy decoders. This modified header excludes enhancement layer NAL header information, serving as a mediator that allows legacy decoders to correctly parse and decode base layer content without being confused by enhancement layer metadata.
2Manufacturing precision
If the PES packet length includes the NAL unit header length, then the complete AU data is accurately represented, but legacy decoders cannot correctly parse the packet structure
Solution Approach 1:
The PES header structure is made locally adaptive based on the layer type. For base layer AUs, the PES header has a modified form with adjusted length fields that exclude enhancement layer NAL header information. This local structural variation allows legacy decoders to correctly parse base layer packets while advanced decoders can handle both modified and standard packet formats.
Solution Approach 2:
The PES packet length parameter is dynamically adjusted based on the layer type. When encoding base layer AUs, the PES packet length is modified to reflect only the actual base layer data size without including enhancement layer NAL header length, enabling legacy decoders to correctly interpret the packet boundaries and data length.
3Productivity
If a single transport stream carries both base and enhancement layers, then bandwidth efficiency is improved, but decoder compatibility and stream processing complexity increase
Solution Approach 1:
The transport stream structure is designed to be universal, accommodating both legacy and advanced decoders simultaneously. By using modified PES headers for base layer AUs that exclude enhancement layer NAL headers, the same transport stream can be processed by legacy decoders (which ignore the modified header format) and advanced decoders (which recognize and utilize the modified structure), achieving multi-functionality without requiring separate streams.
Data Source
AI summary
Disclosed are herein are methods and systems for encoding and decoding enhanced video streams using a legacy and scalable video components. In one aspect, a method of encoding video data in a packetized elementary stream (PES), is disclosed. The method includes receiving a signal including frames of image data, wherein the frames include base layer and enhancement layer components; encoding the frames of image data to form video access units (AUs), wherein base layer AUs are formed for base layer components and enhancement layers AUs are formed for enhancement layer components, each AU including: an elementary stream header (ELSM) including image data metadata parameters associated with decoding and displaying the image data; a PES header including a presentation time stamp (PTS) and a PES packet length indicating the length of the PES header; one or more codestreams associated with a frame of the image data; adding a network abstraction layer (NAL) unit header defined by an enhancement layer video standard to each base layer AU, the NAL unit header being a constant value and length; and forming a modified PES header by increasing the PES packet length to include the length of the NAL unit header in it.