Temporal Distortion Modeling in Video Quality Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional perceptual quality models lack robustness in predicting the perceived quality of re-constructed video content, particularly failing to accurately model complex temporal video distortions such as motion estimation mismatches, flicker, and ghosting.
Innovation Solution
A computer-implemented method that generates frame difference matrices, computes entropy matrices at multiple scales, and uses machine learning models to predict video quality by incorporating temporal features that emulate human visual processing, along with an ensemble prediction engine to combine scores from multiple perceptual quality models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional perceptual quality models are used to predict video quality, then the prediction process is simple and fast, but the models lack robustness and fail to accurately model complex temporal video distortions
Solution Approach 1:
The patent segments the video quality assessment into multiple independent components: spatial quality assessment, temporal quality assessment, and ensemble prediction. Each component uses specialized features (spatial features for static quality, temporal features for motion-related distortions) that are computed separately and then combined, allowing the system to address complex temporal distortions without overwhelming complexity
Solution Approach 2:
The patent creates a composite quality prediction system by combining multiple perceptual quality models (PQM1, PQM2, etc.) into an ensemble. Each model is trained with different feature sets and combines spatial and temporal aspects, creating a robust composite system that leverages the strengths of individual models while mitigating their weaknesses
2Measurement precision
If perceptual quality models are trained based on human assessments, then the models can predict quality scores, but they are unable to reliably predict perceived visual quality for complex temporal distortions
Solution Approach 1:
The patent adds a temporal dimension to traditional spatial quality assessment by introducing temporal features computed from frame differences and optical flow. This dimensional expansion allows the system to capture motion-related distortions (flicker, ghosting, motion estimation errors) that were previously invisible to spatial-only models, significantly improving adaptability to diverse distortion types
Solution Approach 2:
The patent creates universal quality assessment features that work across multiple distortion types. The temporal features (frame difference, optical flow) and the ensemble of multiple PQMs are designed to handle various distortion scenarios uniformly, making the system versatile without requiring separate models for each distortion type
Data Source
AI summary
In various embodiments, a prediction application computes a quality score for re-constructed visual content that is derived from visual content. The prediction application generates a frame difference matrix based on two frames included in the re-constructed video content. The prediction application then generates a first entropy matrix based on the frame difference matrix and a first scale. Subsequently, the prediction application computes a first value for a first temporal feature based on the first entropy matrix and a second entropy matrix associated with both the visual content and the first scale. The prediction application computes a quality score for the re-constructed video content based on the first value, a second value for a second temporal feature associated with a second scale, and a machine learning model that is trained using subjective quality scores. The quality score indicates a level of visual quality associated with streamed video content.