Joint Luma-Chroma Inter Coding for YUV Video Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning-based video coding techniques, such as DLVC, struggle with inefficiencies in inter-frame coding, particularly in the YUV 4:2:0 domain, due to separate processing of luma and chroma components, which leads to increased complexity and reduced coding performance compared to VVC.
Innovation Solution
Implementing joint luma-chroma motion compensation and residual coding networks, along with attention layers, temporal motion prediction, cross-domain fusion, and improved training procedures to enhance neural network performance in YUV 4:2:0 coding, including weighted motion-compensated inter prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If separate luma and chroma motion compensation networks are used, then the processing is simpler to implement, but the coding efficiency deteriorates
Solution Approach 1:
The patent merges separate luma and chroma motion compensation networks into a unified joint luma-chroma motion compensation network. This network processes both luma and chroma components simultaneously, sharing motion estimation and compensation operations across both color channels, thereby improving coding efficiency while maintaining implementation feasibility through a consolidated architecture.
Solution Approach 2:
The joint motion compensation network is designed to perform multiple functions: it handles both luma and chroma motion compensation within a single network structure, and can adaptively process different color domains (YUV 4:2:0 and RGB). This multi-functional design improves coding efficiency by exploiting correlations between luma and chroma while keeping the system relatively simple through a universal processing framework.
2Productivity
If joint luma-chroma motion compensation and residual coding networks are implemented, then coding efficiency improves, but device complexity increases
Solution Approach 1:
The patent segments the joint luma-chroma processing network into distinct functional modules: a joint motion compensation network for motion estimation and compensation, and a joint residual coding network for residual processing. This segmentation allows the complex joint processing to be organized into manageable, specialized sub-networks that can be implemented efficiently while maintaining the benefits of joint luma-chroma processing.
Solution Approach 2:
The patent introduces domain adaptation mechanisms that operate in an additional dimensional space, enabling the network to handle both YUV 4:2:0 and RGB color domains through a unified architecture. This dimensional approach allows the complex joint processing to be managed by adding an adaptation layer rather than creating entirely separate processing paths for different color domains.
3Manufacturing precision
If deep learning-based inter-frame coding is used, then coding performance can be improved, but complexity increases compared to traditional methods like VVC
Solution Approach 1:
The patent implements self-service mechanisms where the deep learning networks automatically learn and adapt to the specific characteristics of the input video content. The joint luma-chroma motion compensation network and residual coding network perform self-adjustment during processing, reducing the need for complex external control systems and manual parameter tuning, thereby improving coding performance while managing system complexity through autonomous adaptation.
Solution Approach 2:
The patent utilizes parameter changes in the deep learning networks, particularly through domain adaptation parameters that allow the same network architecture to efficiently process different color domains (YUV 4:2:0 and RGB). By changing adaptive parameters rather than restructuring the entire network, the system achieves improved coding performance while keeping the overall system complexity manageable through parameter-based flexibility rather than architectural complexity.
Data Source
AI summary
Methods, systems, and bitstream syntax are described for inter-frame coding using end-to-end neural networks used in image and video compression. Inter-frame coding methods include one or more of: joint luma-chroma motion compensation for YUV pictures, joint luma-chroma residual coding for YUV pictures, using attention layers, enabling temporal motion prediction networks for motion vector prediction, using a cross-domain network which combines motion vector and residue information for motion vectors decoding, using the cross-domain network for decoding residuals, using weighted motion-compensated inter prediction, and using temporal only, spatial only, or both temporal and spatial features in entropy decoding. Methods to improve training of neural networks for inter-frame coding are also described.