Packing Higher-Resolution Chroma Frames for Legacy Codecs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video codecs primarily support YUV 4:2:0 chroma sampling format, limiting the ability to encode and decode higher-resolution formats like YUV 4:4:4, which are necessary for applications with richer color information, such as screen content and animated videos, due to a lack of widespread hardware support for 4:4:4 formats.
Innovation Solution
A method for packing frames of a higher-resolution chroma sampling format, such as YUV 4:4:4, into frames of a lower-resolution format, like YUV 4:2:0, allowing existing encoders and decoders to process higher-resolution content by rearranging and encoding the frames, and subsequently unpacking them to retain chroma information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If YUV 4:2:0 chroma sampling format is used for encoding/decoding, then compression efficiency is improved and hardware compatibility is enhanced, but chroma resolution and color fidelity deteriorate
Solution Approach 1:
The patent segments the chroma information into two separate streams: a base stream containing down-sampled chroma data and an enhanced stream containing the remaining high-resolution chroma information. This segmentation allows the base stream to be processed by standard 4:2:0 hardware while the enhanced stream preserves the full 4:4:4 chroma resolution, resolving the contradiction between compression efficiency and chroma fidelity.
2Manufacturing precision
If YUV 4:4:4 chroma sampling format is used for encoding/decoding, then chroma resolution and color fidelity are improved, but hardware compatibility and widespread support deteriorate
Solution Approach 1:
The patent introduces an intermediary approach by creating a base stream in 4:2:0 format that can be processed by existing hardware decoders. This base stream acts as a mediator that maintains hardware compatibility while an enhanced stream adds the necessary 4:4:4 chroma information, thus resolving the contradiction between chroma resolution and hardware compatibility.
3Adaptability or versatility
If frame packing of higher-resolution chroma sampling format into lower-resolution format is implemented, then compatibility with existing codecs is improved, but complexity of encoding/decoding process increases
Solution Approach 1:
The encoding process is segmented into two independent streams: a base stream processed by standard 4:2:0 codecs and an enhanced stream containing supplementary 4:4:4 chroma information. This segmentation reduces encoding complexity by allowing the use of existing 4:2:0 hardware while adding minimal processing for the enhanced stream, thus resolving the contradiction between codec compatibility and encoding complexity.
Data Source
Figure 1~2b
Figure 3
Figure 4
AI summary
Video frames of a higher-resolution chroma sampling format such as YUV 4:4:4 are packed into video frames of a lower-resolution chroma sampling format such as YUV 4:2:0 for purposes of video encoding. For example, sample values for a frame in YUV 4:4:4 format are packed into two frames in YUV 4:2:0 format. After decoding, the video frames of the lower-resolution chroma sampling format can be unpacked to reconstruct the video frames of the higher-resolution chroma sampling format. In this way, available encoders and decoders operating at the lower-resolution chroma sampling format can be used, while still retaining higher resolution chroma information. In example implementations, frames in YUV 4:4:4 format are packed into frames in YUV 4:2:0 format such that geometric correspondence is maintained between Y, U and V components for the frames in YUV 4:2:0 format.