Overlapping Pixel Blending With Decoder-Side Filter Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multimedia transport systems face challenges in efficiently blending extra output pixels and selecting optimal filtering modes for neural networks, particularly in scenarios involving overlapping pixels and varying filter performance.
Innovation Solution
An apparatus and method for blending extra output pixels and decoder-side selection of filtering modes using neural networks, involving multiple filters and weighted sums based on predetermined or optimized weights, uncertainty estimates, and training/finetuning to determine optimal filtering modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple filters are applied to process overlapping pixels, then the quality and accuracy of the filtered output is improved, but the computational complexity and processing time increase
Solution Approach 1:
The patent implements dynamic filter selection where the system adaptively chooses between different filtering modes (neural network filter, traditional filter, or no filter) based on the characteristics of each block and its overlapping regions. This dynamic approach allows the system to apply complex neural network filtering only when necessary, while using simpler methods for other cases, thereby maintaining high accuracy where needed while reducing overall computational complexity.
Solution Approach 2:
The patent applies different filtering strategies to different regions of the video frame based on local characteristics. Specifically, it processes overlapping pixels with neural network filters while non-overlapping pixels may use traditional filters or no filtering. This localized approach ensures high quality processing for critical overlapping regions while reducing unnecessary computational overhead in other areas.
2Manufacturing precision
If neural network filters are used for processing video blocks, then the blending quality of overlapping pixels is improved, but the processing speed and computational efficiency decrease
Solution Approach 1:
The patent applies neural network filtering selectively only to overlapping pixels and regions where it provides the most benefit, rather than applying it to the entire video frame. This partial application approach maintains high blending quality for overlapping regions while significantly reducing the overall computational burden and processing time compared to full-frame neural network filtering.
Solution Approach 2:
The patent divides the video processing task into separate segments: overlapping regions are processed with neural network filters while non-overlapping regions use traditional filtering or no filtering. This segmentation allows the system to optimize quality for critical overlapping areas without incurring the full computational cost of applying neural networks to the entire frame.
3Adaptability or versatility
If decoder-side filtering mode selection is implemented, then the adaptability to different filtering scenarios is improved, but the device complexity and implementation difficulty increase
Solution Approach 1:
The patent implements dynamic filtering mode selection at the decoder side, where the system automatically chooses between neural network filtering, traditional filtering, or no filtering based on the characteristics of each video block and its overlapping regions. This dynamic adaptation provides high versatility for different filtering scenarios while managing decoder complexity through intelligent, context-aware decision-making rather than requiring all filtering modes to be permanently implemented.
Data Source
AI summary
An example method includes: receiving an input extended block comprising an input block and input margins, wherein the input block is derived from an image or a video frame, and wherein the input block and the input margins are input to a filter, and wherein the input margins comprise pixels of the image or the video frame; filtering the input extended block to obtain a filtered extended block comprising a filtered block and filtered margins; receiving other blocks, wherein the other blocks are derived from the image or video frame, and wherein at least one pixel of the other blocks overlaps with at least one pixel of the filtered margins of the filtered extended block; and blending the filtered extended block with the other blocks, wherein an operation to blend the filtered extended block with the other blocks is applied to two or more overlapping pixels.