Chroma Subsampling Handling for ML Picture Coding Trade-Offs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies using machine learning networks face inefficiencies in handling chroma subsampled formats, leading to challenges in bitrate management and picture quality, particularly in applications with limited network resources and high video quality demands.
Innovation Solution
Adaptive selection between luma down-sampling and chroma up-sampling to match resolution, enabling efficient encoding and decoding of picture portions with a trained network, utilizing variational autoencoders (VAEs) for improved bitrate and quality management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If chroma subsampled formats are used to reduce bitrate, then data transmission efficiency is improved, but picture quality deteriorates due to loss of chroma information
Solution Approach 1:
The patent applies parameter changes by transforming the chroma component from subsampled format to full-resolution format through up-sampling operations. This changes the resolution parameter of the chroma component to match the luma component, thereby improving picture quality while maintaining efficient bitrate utilization through the learned transformation process
Solution Approach 2:
The patent replaces traditional mechanical up-sampling methods (such as simple interpolation or replication) with a machine learning-based up-sampling network. This learned mechanism substitutes conventional signal processing approaches, achieving superior chroma reconstruction quality by leveraging patterns learned from training data
2Device complexity
If luma down-sampling is applied to match chroma resolution, then encoding complexity is reduced, but picture quality deteriorates due to loss of luma detail
Solution Approach 1:
Instead of down-sampling the luma component to match the lower-resolution chroma, the patent inverts the approach by up-sampling the chroma component to match the higher-resolution luma. This reversal maintains luma detail integrity while achieving resolution matching through chroma enhancement
Solution Approach 2:
The patent changes the resolution parameter of the chroma component through up-sampling to match the luma resolution, avoiding luma down-sampling entirely. This parameter transformation maintains encoding efficiency while preserving picture quality by keeping the high-resolution luma intact
3Productivity
If resolution matching is performed through traditional methods, then encoding efficiency is improved, but adaptability to different chroma subsampling formats deteriorates
Solution Approach 1:
The patent implements universality by designing a machine learning-based up-sampling network that can handle multiple chroma subsampling formats (such as 4:2:0, 4:2:2, 4:4:4) through a single unified model. The network learns to adapt to different input formats and produces consistent high-quality output, eliminating the need for format-specific encoding paths
Solution Approach 2:
The patent applies self-service by enabling the up-sampling network to automatically adapt to different chroma subsampling formats without requiring manual configuration or pre-processing. The model autonomously learns the appropriate transformation for each format during training and applies it during inference, improving encoding efficiency across diverse formats
4Manufacturing precision
If machine learning networks are used for picture coding, then picture quality is improved, but computational complexity and processing time increase
Solution Approach 1:
The patent applies preliminary action by pre-training the machine learning network on extensive training data to learn optimal chroma up-sampling transformations. This pre-training phase performs the computationally intensive learning work in advance, allowing the trained model to execute efficient inference during actual encoding operations with reduced real-time computational burden
Solution Approach 2:
The patent uses copying by creating a trained model that encapsulates the learned up-sampling knowledge. Once trained, the model can be copied and deployed across multiple encoding systems, amortizing the computational complexity over many uses and reducing the per-encoding computational burden through efficient inference operations
Data Source
AI summary
The present disclosure relates to picture or video encoding and decoding. A picture portion includes a luma component and a chroma component with a resolution lower than a resolution of the luma component. To obtain matching resolution of the luma component and the chroma component, a selection is performed between a down-sampling processing of the luma component and an up-sampling processing of the chroma component. Then, the selected processing is applied to a picture portion. The bitstream is then generated based on the selected processing, and includes encoding the processed picture portion. The selection between chroma up-sampling and luma down-sampling enables adaption of the input picture portion for the desired application. For example, some applications may aim at reducing complexity or bitrate in which case the luma down-sampling may be more favorable, whereas other application may aim at higher quality in which case chroma up-sampling may be more favorable.