YUV Motion Flow Coding Using Luminance-Only Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video compression methods using artificial neural networks fail to efficiently compress videos represented in YUV 4:2:0 format due to the inability to process luminance and chrominance channels at different resolutions, leading to suboptimal motion estimation and coding.
Innovation Solution
The method computes motion information using the luminance component only, warps the chrominance components separately, and down-samples the motion information to match the chrominance resolution, allowing efficient compression of YUV 4:2:0 format videos.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion information is computed using both luminance and chrominance components at full resolution, then motion estimation accuracy is improved, but computational complexity and bitrate increase
Solution Approach 1:
The patent segments the video processing into separate luminance and chrominance channels, computing motion information exclusively from the luminance component while handling chrominance separately. This segmentation allows the system to focus computational resources on the most important motion-carrying channel (luminance) while reducing overall complexity by excluding chrominance from motion estimation.
Solution Approach 2:
The patent extracts and utilizes only the luminance component for motion information computation, deliberately excluding the chrominance components from this specific task. This extraction principle reduces the amount of data processed for motion estimation while maintaining accuracy, since luminance contains the primary motion information.
2Manufacturing precision
If chrominance data is processed at full resolution alongside luminance, then chrominance quality is improved, but bitrate and processing load increase
Solution Approach 1:
The patent applies different processing qualities to different color channels: full-resolution processing for luminance (which carries motion information) and down-sampled processing for chrominance. This local quality differentiation optimizes bitrate usage by allocating higher precision only where necessary for motion estimation while maintaining acceptable chrominance quality at lower resolution.
Solution Approach 2:
The patent merges the motion compensation process for luminance and chrominance by using the same motion vectors derived from luminance for both channels. This combining approach reduces redundancy in the bitstream while maintaining synchronization between color channels, effectively reducing overall bitrate requirements.
3Productivity
If separate processing paths are used for luminance and chrominance, then compression efficiency is improved, but device complexity increases
Solution Approach 1:
The patent creates a universal motion estimation process that serves both luminance and chrominance channels. The motion vectors computed from luminance are universally applied to both channels, making the motion estimation module multi-functional and reducing the need for separate processing paths, thereby limiting complexity increase while maintaining compression efficiency.
4Productivity
If motion information is down-sampled to match chrominance resolution, then processing efficiency is improved, but motion estimation precision deteriorates
Solution Approach 1:
The patent performs motion estimation and vector computation in advance using full-resolution luminance data before any down-sampling operations. This preliminary action ensures that motion information is captured at maximum precision, and subsequent down-sampling only affects the chrominance reconstruction, not the motion estimation itself.
Data Source
AI summary
Video encoding and decoding is implemented with auto encoders using luminance information to derive motion information for chrominance prediction. In one embodiment YUV 4:2:0 video is encoded and decoded in which luminance information is downsampled to generate predictions from chrominance components of a reference frame. In a related embodiment, more than one reference frame is used for predictions. In another embodiment, convolutions and transpose convolutions implement derivation of motion information.