Video Quality Assessment Using Temporal High-Pass Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video quality assessment metrics, such as PSNR, poorly correlate with subjective impressions, and while alternatives like SSIM and VMAF show better correlation, they are complex and not differentiable, making them unsuitable for perceptual bit-allocation strategies during encoding.
Innovation Solution
A low-complexity extension of the perceptually weighted PSNR (WPSNR) is developed, incorporating motion-aware algorithms that determine visual activity information using temporal and spatial high-pass filters, allowing for adaptive coding quantization across video frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional PSNR metric is used for video quality assessment, then computational complexity is low, but correlation with subjective impressions is poor
Solution Approach 1:
The patent applies local quality assessment by computing visual activity information for each picture block separately using high-pass filtering. This allows the metric to adapt to local characteristics of different regions in the video frame, improving correlation with subjective impressions while maintaining computational efficiency through localized processing rather than global analysis.
Solution Approach 2:
The patent segments the video frame into multiple picture blocks and processes each block independently to compute visual activity information. This segmentation enables parallel computation and reduces overall complexity while capturing local variations in visual content that improve measurement precision.
2Measurement precision
If VMAF algorithm is used for video quality assessment, then correlation with MOS scores is high, but algorithmic complexity is high and it is not differentiable
Solution Approach 1:
The patent replaces the complex, non-differentiable VMAF algorithm with a simplified mathematical model based on high-pass filtering and visual activity computation. This substitution maintains differentiability for optimization purposes while achieving comparable correlation with MOS scores through a more tractable computational approach.
Solution Approach 2:
The patent changes the computational parameters from VMAF's complex multi-stage processing to a streamlined approach using high-pass filter coefficients and visual activity metrics. This parameter transformation simplifies the algorithm while preserving the essential perceptual weighting characteristics needed for high MOS correlation.
3Measurement precision
If block-wise WPSNR metric is used for image quality assessment, then correlation with MOS data is good, but performance on video data is worse than SSIM or VMAF
Solution Approach 1:
The patent introduces temporal dynamics by computing visual activity information across multiple video frames using high-pass filtering in both spatial and temporal domains. This dynamic approach adapts to motion and temporal variations in video content, improving versatility and performance on video data while maintaining the block-wise structure that provides good MOS correlation.
Data Source
AI summary
An apparatus for determining visual activity information for a predetermined picture block of a video sequence including a plurality of video frames, the plurality of video frames including a current video frame and one or more timely-preceding video frames, wherein the one or more timely-preceding video frames precede the current video frame in time, is provided. The apparatus is configured to receive the predetermined picture block of each of the one or more timely-preceding video frames and the predetermined picture block of the current video frame. Moreover, the apparatus is configured to determine the visual activity information depending on the predetermined picture block of the current video frame and depending on the predetermined picture block of each of the one or more timely-preceding video frames and depending on a temporal high-pass filter.