Overlapping Pixel Blending With Decoder-Side Filter Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multimedia transport systems face challenges in effectively blending extra output pixels and selecting optimal filtering modes for neural networks, particularly in scenarios involving overlapping pixels and varying filter performance.
Innovation Solution
An apparatus and method for blending extra output pixels of a filter and decoder-side selection of filtering modes, utilizing neural networks to process input blocks with overlapping pixels, apply multiple filters, and determine optimal filtering modes based on weights and confidence estimates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple filters are applied to process input blocks with overlapping pixels, then the accuracy of pixel blending is improved, but the computational complexity and processing time increase
Solution Approach 1:
The system dynamically selects filtering modes based on confidence estimates. Different filtering modes (e.g., first filter for certain pixel regions, second filter for other regions) are applied adaptively to different parts of the image or video data. This allows the system to use complex filtering only where necessary while using simpler filtering elsewhere, resolving the contradiction between accuracy and complexity.
Solution Approach 2:
The patent applies different filtering strategies to different regions of the image based on local characteristics. By dividing the processing into regions with different confidence levels or characteristics, the system can apply more complex filtering to specific local areas that require it, while using simpler or no filtering in other areas, thus balancing accuracy requirements with computational complexity.
2Manufacturing precision
If multiple filtering modes are selected and applied, then the quality of processed pixels is improved, but the processing time and computational resources increase
Solution Approach 1:
The system applies multiple filtering modes selectively rather than universally. By using confidence estimates to determine where complex filtering is truly necessary, the system avoids applying excessive filtering to all pixels. This partial application of complex filtering modes maintains high quality where needed while reducing overall processing time and computational resource consumption.
Solution Approach 2:
The system performs preliminary assessment of pixel regions to determine which areas require complex filtering. By evaluating confidence estimates or regional characteristics beforehand, the system can plan and execute filtering operations more efficiently, applying computationally intensive modes only to predetermined regions that benefit from them, thus reducing total processing time.
3Adaptability or versatility
If confidence estimates are used to select filtering modes, then the adaptability of the processing system is improved, but the computational overhead increases
Solution Approach 1:
The system uses confidence estimates that are naturally generated or easily derived from the filtering process itself, rather than requiring separate complex assessment mechanisms. This self-service approach to adaptability allows the system to select appropriate filtering modes using information already available from the processing pipeline, minimizing additional computational energy consumption while maintaining high adaptability.
Data Source
AI summary
An example method includes: receiving an input extended block comprising an input block and input margins, wherein the input block is derived from an image or a video frame, and wherein the input block and the input margins are input to a filter, and wherein the input margins comprise pixels of the image or the video frame; filtering the input extended block to obtain a filtered extended block comprising a filtered block and filtered margins; receiving other blocks, wherein the other blocks are derived from the image or video frame, and wherein at least one pixel of the other blocks overlaps with at least one pixel of the filtered margins of the filtered extended block; and blending the filtered extended block with the other blocks, wherein an operation to blend the filtered extended block with the other blocks is applied to two or more overlapping pixels