Pixel-Adaptive Video Frame Filtering for Decoding Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies use fixed filters that do not adapt to the specific characteristics of individual frames, leading to suboptimal restoration quality in decoded images.
Innovation Solution
Implement pixel-adaptive filtering using a filter bank combined with pixel-specific combining scalars derived from local pixel information and potentially side-information, trained to minimize errors with machine-learning models or simpler features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If fixed filters are used for decoding video frames, then device complexity is reduced, but restoration quality deteriorates
Solution Approach 1:
The patent implements dynamic filter selection where the filter bank is adaptively chosen based on the characteristics of the current video frame and previous frames. Different filters are selected for different frames rather than using a single fixed filter, allowing the system to adapt to varying content characteristics while maintaining manageable complexity through the use of a predefined filter bank.
Solution Approach 2:
The patent applies different filters from the filter bank to different regions or frames based on their specific characteristics. Each frame or region can utilize the most appropriate filter from the bank, ensuring that local quality requirements are met without requiring a completely custom filter for every location, thus balancing quality and complexity.
2Manufacturing precision
If pixel-adaptive filtering is implemented, then restoration quality is improved, but computational complexity increases
Solution Approach 1:
The patent pre-computes and stores multiple filters in a filter bank during system initialization or offline processing. These pre-computed filters are then directly applied during video decoding without requiring real-time computation, significantly reducing the computational burden during actual video processing while still providing adaptive filtering capabilities.
Solution Approach 2:
The patent changes the parameters of the filtering operation by selecting different filters from the pre-computed bank based on frame characteristics. Instead of computing new filters for each frame, the system varies the filter parameters by selecting from predefined options, maintaining restoration quality while reducing computational complexity during runtime.
3Productivity
If compression encoding is applied to reduce video data, then data transmission efficiency is improved, but artifacts are introduced in decoded frames
Solution Approach 1:
The patent converts the harmful compression artifacts into beneficial information by using them as indicators for filter selection. The filtering process is specifically designed to address and remove the types of artifacts introduced by compression, transforming the problem of artifact introduction into an opportunity for targeted artifact removal and quality enhancement.
Solution Approach 2:
The patent implements a feedback mechanism where the decoded frame quality is assessed and used to select appropriate filters from the bank. The filtering process continuously adapts based on the observed quality and artifact levels, creating a closed-loop system that responds to compression-induced degradation and actively works to mitigate it.
Data Source
AI summary
A filter bank comprising filters is obtained. For pixels of a degraded frame, respective sets of combining scalars for combining the filters of the filter bank are obtained. For the pixels of the degraded frame, respective pixel-specific filters are obtained by combining the filters of the filter bank using the respective sets of combining scalars. A restored frame is obtained by filtering the pixels of the degraded frame using the respective pixel-specific filters. The respective sets of combining scalars may be obtained using a machine-learning model that receives the degraded frame as an input, where the machine-learning model is a convolutional neural network. The machine-learning model may be trained to minimize an error between restored frames and corresponding source frames.