Chroma Prediction from Luma with Neural Embeddings in Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in efficiently compressing and decompressing chroma components, leading to suboptimal video quality and increased data requirements.
Innovation Solution
Implementing a chroma prediction model using neural networks to enhance chroma prediction in video coding, particularly for screen content, by leveraging intra and inter prediction techniques and quadtree+multi-type tree partitioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional chroma compression methods are used, then device complexity is reduced, but video quality deteriorates and data requirements increase
Solution Approach 1:
The patent divides the chroma compression process into distinct stages: partitioning the chroma block into multiple sub-blocks, applying different prediction models to each sub-block, and selectively using neural network-based or traditional prediction methods. This segmentation allows the system to achieve high video quality in critical regions while maintaining reasonable device complexity by not applying complex models universally.
Solution Approach 2:
The patent implements local quality enhancement by applying different prediction strategies to different regions of the chroma block. Neural network-based prediction is applied to sub-blocks where it provides significant quality improvement, while traditional prediction methods are used in other regions. This local differentiation optimizes the balance between video quality and computational complexity.
2Manufacturing precision
If chroma prediction is enhanced with neural networks, then video quality improves, but data requirements and processing complexity increase
Solution Approach 1:
The patent applies neural network-based prediction partially rather than universally. Specifically, it uses neural network prediction for certain sub-blocks of the chroma block while relying on traditional prediction methods for others. This partial application achieves improved chroma prediction accuracy where needed while avoiding the excessive data requirements and processing complexity that would result from applying neural networks to the entire chroma block.
3Measurement precision
If multiple prediction modes are used for chroma blocks, then prediction accuracy improves, but device complexity and processing time increase
Solution Approach 1:
The patent implements dynamic selection of prediction modes based on local characteristics of chroma sub-blocks. Rather than using a fixed prediction mode for the entire chroma block or evaluating all possible modes uniformly, the system dynamically adapts the prediction approach for each sub-block, selecting between neural network-based and traditional prediction methods based on local requirements. This dynamic approach improves prediction accuracy while managing device complexity by avoiding exhaustive evaluation of all modes in all regions.
Data Source
AI summary
A video decoder generates a reconstruction of a luma block based on a prediction of the luma block and a residual of the luma block. The decoder further generates a prediction of a chroma block corresponding to the luma block via a chroma prediction model by inputting the reconstruction of the luma block to one or more first embedding layers of the chroma prediction model and inputting one or more coding parameters of the luma block to one or more second embedding layers of the chroma prediction model. The one or more second embedding layers are separate from the one or more first embedding layers. The decoder further determines a reconstruction of the chroma block based on the prediction of the chroma block and a residual of the chroma block.