Hybrid Video Quality Model Combining Deterministic and Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video quality prediction models for HTTP Adaptive Streaming (HAS) are limited in their ability to accurately assess visual quality across various bandwidths and codecs, particularly failing to handle effects beyond compression artifacts and not being adaptable to different video codecs and resolutions used in modern streaming applications.
Innovation Solution
An information-adaptive mixed deterministic/machine-learning-based bit stream video-quality model that predicts video quality by combining a deterministic model with a machine learning model, using a weighted sum of their predictions, and compensating for the deterministic model's residual errors, with the machine learning model's weight increasing with the availability of bit stream parameters, and capable of handling resolutions from 240p to 3840p and frame rates from 15 to 60 fps for H.264, H.265, and VP9 codecs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a deterministic model is used for video quality prediction, then the model is computationally efficient and fast, but the prediction accuracy is insufficient especially for modern codecs and resolutions
Solution Approach 1:
The patent combines a deterministic model with a machine learning model into a hybrid architecture. The deterministic model provides baseline predictions using analytical formulas, while the machine learning model (using Random Forest or Gradient Boosting) corrects residual errors. This merging allows the system to achieve high prediction accuracy for modern codecs like H.265 and VP9 at resolutions up to 3840p, while maintaining computational efficiency through the deterministic component.
Solution Approach 2:
The patent adapts model parameters dynamically based on input characteristics such as codec type, resolution, and frame rate. The machine learning model learns optimal correction parameters from training data across diverse coding conditions, allowing the hybrid model to adjust its prediction behavior to match specific codec and resolution combinations, thereby improving accuracy without requiring separate models for each condition.
2Measurement precision
If a machine learning model is used to improve prediction accuracy, then the prediction accuracy increases, but the computational complexity and processing time increase
Solution Approach 1:
The patent segments the prediction task into two parts: a fast deterministic calculation that provides immediate baseline predictions, and a machine learning-based residual correction that refines the results. This segmentation allows the system to deliver quick initial assessments while applying more computationally intensive machine learning only where needed to correct specific prediction errors, thereby reducing overall processing time compared to using a pure machine learning approach.
Solution Approach 2:
The machine learning model is applied selectively to correct residual errors rather than performing complete predictions from scratch. The deterministic model handles the bulk of the prediction work, and the machine learning component applies partial correction only to the residual difference between deterministic predictions and actual quality metrics, reducing the computational burden of the machine learning portion.
3Adaptability or versatility
If conventional video quality models are used, then the models are simple and easy to implement, but they fail to handle effects beyond compression artifacts and are not adaptable to different codecs and resolutions
Solution Approach 1:
The patent creates a universal hybrid model architecture that can handle multiple video codecs (H.264, H.265, VP9), resolutions (up to 3840p), and frame rates (15-60 fps) through a single unified system. The machine learning component is trained on diverse datasets covering various coding conditions, enabling the model to generalize across different codecs and resolutions without requiring separate specialized models, thereby achieving versatility with moderate complexity increase.
Data Source
Figure 1
Figure 2~3
AI summary
Disclosed herein is a method for predicting the quality Q of a video bit stream. The method comprises obtaining a video segment and a corresponding bit stream, parsing the bit stream by a bit stream parser, and obtaining bit stream parameters. The quality Q is predicted by using a deterministic model and a machine learning model with the obtained bit stream parameters, wherein the predicted video quality is a weighted sum of the prediction of both models. A residuum of the deterministic model is predicted by the machine learning model.