Neural Network Video Encoding for Region-Based Bitrate Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high computational complexity and bit-rate of the Versatile Video Coding (VVC) standard for ultra high-definition video requires improvements in visual quality with reduced complexity and bit-rate without compromising media content quality.
Innovation Solution
Employing a neural network to identify less important regions in video frames, such as textures, and allocating fewer encoding resources or bits to these regions, while maintaining quality by setting inter-picture prediction residuals to zero, thereby reducing bit-rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If VVC standard is used for ultra high-definition video, then video quality and frame rate are improved, but computational complexity increases several times compared to HEVC
Solution Approach 1:
The patent applies local quality by differentiating encoding precision across different regions of the video frame. The neural network identifies important regions (e.g., foreground objects) versus less important regions (e.g., background), and applies different bit allocation strategies to each region type, thereby achieving high video quality where needed while reducing overall computational complexity and bit-rate.
2Manufacturing precision
If higher bit-rate is used, then video quality is improved, but transmission and storage costs increase
Solution Approach 1:
The patent implements local quality by allocating encoding bits non-uniformly across the video frame. The neural network identifies important regions that require higher bit allocation to maintain visual quality, while less important regions receive reduced or zero bit allocation. This selective bit allocation maintains perceived video quality while significantly reducing the overall bit-rate for transmission and storage.
3Manufacturing precision
If more encoding resources are allocated to a region, then encoding quality of that region is improved, but overall bit-rate increases
Solution Approach 1:
The patent applies local quality by making encoding resources dependent on the importance of each region. The neural network classifies regions as important or less important, and the encoder allocates more bits to important regions while using fewer or zero bits for less important regions. This creates a non-uniform bit allocation strategy that optimizes the trade-off between regional encoding quality and overall bit-rate.
Solution Approach 2:
The patent changes the parameter of bit allocation from a uniform value to a region-dependent value. By using the neural network to determine region importance, the system dynamically adjusts the bit allocation parameter for each region, allowing high-quality encoding where necessary and aggressive compression where acceptable, thereby controlling overall bit-rate while maintaining essential video quality.
Data Source
AI summary
An encoder may determine a plurality of coding units associated with a frame of a media file and a plurality of prediction units associated with the frame of the media file. The encoder may determine, based on the plurality of coding units associated with the frame and the plurality of prediction units associated with the frame, and based on a training of the encoder using one or more neural networks, that a particular region of the frame can be encoded using one or more encoding characteristics that are different than the encoding characteristics of one or more other particular regions of the frame. The encoder may allocate one or more encoding resources to the particular region of the frame based on the one or more encoding characteristics of the particular region of the frame in order to reduce the overall media bitrate.