Neural Network Video Quality Model for Bandwidth-Constrained Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image and video compression technologies face challenges in efficiently transmitting and storing high-resolution content over communications channels with limited bandwidth, while maintaining acceptable quality.
Innovation Solution
The use of media compression and processing techniques that incorporate machine-learning-based quality metrics, specifically a neural-network-based video quality model, to optimize encoding parameters and improve reconstructed media quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional compression techniques are used, then bandwidth usage is reduced, but reconstructed media quality deteriorates
Solution Approach 1:
The patent changes the parameter used for quality assessment from traditional compression metrics to machine-learning-based quality models. The neural network model predicts perceived quality by analyzing visual characteristics such as blurriness, artifacts, and structural information, enabling optimization of encoding parameters to achieve better reconstructed quality at lower bandwidths
Solution Approach 2:
The patent replaces traditional mechanical compression algorithms with a hybrid approach that incorporates machine learning models. The neural network-based quality model substitutes for conventional distortion metrics, allowing for more intelligent encoding decisions that prioritize perceptually important regions and maintain quality while reducing bandwidth consumption
2Productivity
If machine-learning-based quality models are integrated into encoding, then encoding efficiency is improved, but device complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-training neural network models offline and storing their parameters for use during encoding. The model weights are prepared in advance and loaded into the encoding system, allowing the actual encoding process to simply involve inference rather than training, thus improving encoding efficiency without requiring complex real-time learning capabilities
Solution Approach 2:
The patent introduces an intermediary layer between the encoder and the quality assessment. The neural network model acts as a mediator that takes encoded data as input and outputs quality predictions, bridging the gap between compression algorithms and quality metrics. This intermediary abstraction allows the encoding system to leverage complex quality models without directly implementing their full complexity in the encoding loop
Data Source
AI summary
Encoding using media compression and processing for machine-learning-based quality metrics includes generating encoded frame data by encoding a current frame from an input video using a neural-network-based video quality model, which includes identifying optimal encoding parameters for encoding a current block, wherein the optimal encoding parameters minimize a rate-distortion optimization cost function, which includes using a gradient value for the current block obtained from a neural-network-based video quality model generated gradient map obtained from the neural-network-based video quality model for the current frame, obtaining a restoration filtered reconstructed frame by restoration filtering a reconstructed frame, obtained by decoding the encoded frame data, using the neural-network-based video quality model generated gradient map obtained for the reconstructed frame.