Machine Learning Video Coding Luma Chroma Motion Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding techniques face challenges in efficiently compressing high-quality video data, leading to increased burdens on communication networks and storage devices due to large data sizes, and existing deep learning-based architectures are not optimized for subsampled YUV formats like YUV 4:2:0.
Innovation Solution
A machine learning-based system that processes luminance and chrominance components separately to estimate motion information using a luma component of the current frame and a previously reconstructed frame, allowing for efficient determination of chroma motion information without requiring chroma information in the bitstream, and applies space-scale flow warping parameters for inter-frame predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional video coding techniques are used to compress video data, then video quality can be maintained, but the data volume increases placing a burden on communication networks and storage devices
Solution Approach 1:
The patent extracts and utilizes only the luma component from video frames for motion estimation and compensation, separating it from the full YUV 4:2:0 format processing. This extraction approach allows the system to perform complex motion analysis on a single channel while maintaining compatibility with subsampled chroma components, thereby reducing overall data processing requirements while preserving video quality
Solution Approach 2:
The machine learning model serves multiple functions: it performs motion estimation, motion compensation, and chroma prediction simultaneously using the luma component. This multi-functionality allows a single processing pipeline to handle what would traditionally require separate operations on all color channels, reducing computational complexity and data volume while maintaining reliability
2Productivity
If deep learning-based architectures are used for video coding, then coding efficiency can be improved, but the architectures are not optimized for subsampled YUV formats like YUV 4:2:0
Solution Approach 1:
The patent applies different processing strategies to different components of the YUV 4:2:0 format. The luma component receives full machine learning-based motion estimation and compensation processing, while the chroma components utilize the motion information derived from luma. This localized quality approach optimizes coding efficiency for the most important luminance information while adapting to the subsampled nature of chroma data
Solution Approach 2:
The system creates a composite approach by combining traditional video coding concepts (motion estimation, compensation) with machine learning techniques, and further combines processing of multiple color components (luma and chroma) in a unified framework. This composite methodology enables the architecture to achieve high coding efficiency while being natively compatible with subsampled YUV formats
3Measurement precision
If motion information for both luma and chroma components is determined independently, then accuracy can be maintained, but computational complexity and data requirements increase
Solution Approach 1:
The system performs motion estimation on the luma component first, before processing chroma components. This preliminary action on the higher-resolution luma data provides accurate motion vectors that are then applied to the chroma components. This sequential approach maintains measurement precision for both components while significantly reducing computational complexity compared to independent processing
Data Source
AI summary
Systems and techniques are described herein for processing video data. In some aspects, a method can include obtain, by a machine learning system, input video data. The input video data includes one or more luminance components for a current frame. The method can include determining, by the machine learning system, motion information for the luminance component(s) of the current frame and motion information for one or more chrominance components of the current frame using the luminance component(s) for the current frame. In some cases, the method can include determining the motion information for the luminance component(s) based on the luma component(s) of the current frame and at least one reconstructed luma component of a previous frame. In some cases, the method can further include determining the motion information for the chrominance component(s) of the current frame using the motion information determined for the luminance component(s) of the current frame.