Partial Video Decoding via NAL Unit Header Rewriting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video transmission methods experience significant decoding and rendering delays, especially in VR and 360 video settings, due to the need to transmit and decode full video streams, which can lead to a decreased user experience and nausea, and are limited by hardware capabilities that struggle with high-resolution content.
Innovation Solution
A method that uses meta data to control the rewriting of header data in NAL units to create a second video stream decodable for partial video images, allowing for efficient decoding and rendering by adjusting slice positions and adding or removing padding bits, enabling decoding of video data in lower resolutions and reducing delay during viewport changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If full video streams are transmitted and decoded, then complete video images are obtained, but decoding and rendering delays increase significantly
Solution Approach 1:
The video stream is divided into independently decodable slices that correspond to different spatial regions of the video image. The decoder selectively decodes only the slices corresponding to the current viewport region, avoiding the need to decode the entire video stream. This segmentation enables partial decoding of video data, significantly reducing decoding and rendering delays while maintaining user experience quality.
2Manufacturing precision
If high-resolution video data is decoded, then video quality is improved, but hardware capabilities are exceeded and processing becomes infeasible
Solution Approach 1:
Instead of decoding the entire high-resolution video stream, the system decodes only the partial video data corresponding to the viewport region. This partial action approach allows the decoder to process a manageable subset of the video data at full quality, avoiding hardware overload while maintaining high video quality for the visible region.
3Adaptability or versatility
If viewport changes are handled by retrieving and decoding additional tiles, then updated video content is provided, but total delay increases and user experience decreases
Solution Approach 1:
The video stream is pre-divided into independently decodable slices that can be selectively decoded based on viewport position. When viewport changes occur, the system can quickly switch to decoding different slice combinations without needing to retrieve and decode entire tiles or frames. This preliminary structuring of the video data enables rapid adaptation to viewport changes, reducing total delay and improving user experience.
Data Source
AI summary
Encoded video data representing partial video images is decoded from a first video stream such as a HEVC bitstream. The first video stream comprises video images divided into a spatial array of independently decodable slices. The first video stream comprises network abstraction layer units that each contain video content data for a respective one of the slices preceded by a header that comprises parameters relating to the respective one of the slices in relation to the first video stream. Previously prepared information such as meta data about positions of parameters in the network abstraction layer units or a plurality of different versions of the network abstraction layer units for different partial image sizes (different picture sizes) is used. The meta data may be used for partially rewriting the network abstraction layer units for a subarray of slices, the network abstraction layer units being rewritten at a location selected on the basis of meta data, to make the network abstraction layer units consistent with a second video stream of encoded video data only for the sub-array of slices. Versions of the network abstraction layer units may be selected for the sub-array of slices, from the plurality of different versions. Subsequently, the partial video images are generated by decoding the encoded video data from the rewritten network abstraction layer units.