Bootstrapping Perceptual Quality Model Uncertainty Quantification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing perceptual quality models for encoded videos face accuracy issues due to statistical uncertainty from small sample sizes of encoded videos and human subjects, making it difficult to draw valid conclusions and optimize encoding operations.
Innovation Solution
A computer-implemented method that uses bootstrapping techniques to generate resampled datasets and models, allowing for the quantification of accuracy in perceptual quality scores by creating a distribution of bootstrap scores, which reflects the uncertainty associated with baseline scores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a perceptual quality model is trained on a small sample of encoded training videos and human subjects, then the model can be developed and deployed efficiently, but the accuracy of the predicted perceptual quality scores is reduced due to statistical uncertainty
Solution Approach 1:
The patent applies preliminary action by performing bootstrapping resampling operations on the training data before final model training. Multiple resampled datasets are generated and used to train multiple bootstrap models in advance, allowing the system to quantify uncertainty and validate results before deploying the final model, thus improving accuracy while maintaining development efficiency
Solution Approach 2:
The patent uses copying by creating multiple bootstrap models from resampled versions of the original training data. Each bootstrap model is a copy trained on a slightly different subset of the data, and these multiple copies are then used to generate a distribution of predictions that quantifies the uncertainty in the final model's outputs
2Extent of automation
If perceptual quality scores are used to evaluate codec performance and optimize encoding, then encoding operations can be automated, but valid conclusions cannot be drawn because the accuracy of the scores is unknown
Solution Approach 1:
The patent implements feedback by using the distribution of bootstrap perceptual quality scores to validate and inform encoding decisions. The system compares the baseline score against the bootstrap distribution to determine statistical significance, providing feedback that confirms whether observed quality differences are real or due to sampling variability, thus enabling reliable automated conclusions
Solution Approach 2:
The patent performs preliminary validation by generating bootstrap models and their associated score distributions before making encoding decisions. This preliminary analysis establishes confidence intervals and statistical significance thresholds that guide subsequent automated encoding optimizations, ensuring that only statistically valid conclusions are drawn
Data Source
AI summary
In various embodiments, a bootstrapping training subsystem performs sampling operation(s) on a training database that includes subjective scores to generate resampled dataset. For each resampled dataset, the bootstrapping training subsystem performs machine learning operation(s) to generate a different bootstrap perceptual quality model. The bootstrapping training subsystem then uses the bootstrap perceptual quality models to quantify the accuracy of a perceptual quality score generated by a baseline perceptual quality model for a portion of encoded video content. Advantageously, relative to prior art solutions in which the accuracy of a perceptual quality score is unknown, the bootstrap perceptual quality models enable developers and software applications to draw more valid conclusions and/or more reliably optimize encoding operations based on the perceptual quality score.