In-loop Filter Apparatus for Video Coding Using 3D Auxiliary Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards face limitations in achieving optimal prediction quality and coding efficiency due to constraints in computational complexity and compression efficiency, particularly in suppressing quantization noise through in-loop filtering.
Innovation Solution
An in-loop filter apparatus that performs both local and non-local filtering of reconstructed frames by dividing them into macro-blocks, determining clusters of similar blocks, and generating auxiliary frames to accumulate non-local pixel correlations, allowing for improved averaging and edge preservation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the spatial filter size is increased to improve filtering quality, then the number of pixels used for averaging increases, but the edge directed filtering effect is reduced because edges become curved in larger spatial areas
Solution Approach 1:
The patent transitions from conventional 2D spatial filtering to 3D filtering by incorporating the temporal dimension. Multiple reconstructed frames are stacked to form a 3D volume, allowing filters to operate in three dimensions. This enables the filter to capture non-local correlations across time while maintaining edge directed properties, as the temporal dimension provides additional context without being constrained by spatial curvature issues.
Solution Approach 2:
The patent segments the filtering process into local and non-local components. Local filtering operates on small 3D neighborhoods to preserve edges and local structures, while non-local filtering leverages similarities across different spatial locations and time frames. This segmentation allows the system to benefit from both small filter sizes (for edge preservation) and large effective filter sizes (for noise suppression).
2Object-affected harmful factors
If conventional in-loop filters are used to suppress quantization noise, then filtering is performed on reconstructed frames, but the filtering effectiveness is limited by the local nature of conventional filters
Solution Approach 1:
By stacking multiple reconstructed frames to create a 3D volume, the patent enables non-local filtering operations that can suppress quantization noise more effectively. The temporal dimension provides redundant information from different time points, allowing the filter to distinguish between actual image content and quantization noise through temporal consistency analysis.
Solution Approach 2:
The patent creates copies of the filtering operation across multiple frames and spatial locations by identifying similar blocks throughout the 3D volume. Non-local means filtering computes weighted averages of corresponding pixels from similar blocks, effectively copying and combining information from multiple sources to suppress noise while preserving genuine image features.
3Productivity
If video coding standards prioritize compression efficiency, then computational complexity is constrained, but this limits the ability to implement advanced filtering techniques
Solution Approach 1:
The patent segments the video frame into macro-blocks and further into smaller processing units, allowing parallel processing of independent blocks. This segmentation enables efficient implementation of computationally intensive non-local filtering operations by distributing the workload across multiple processing elements, thereby reducing the computational burden per unit while maintaining overall filtering quality.
Solution Approach 2:
The patent implements a two-stage filtering approach where a fast local filter is applied first to provide immediate noise suppression, followed by a more computationally intensive non-local filter that processes only the filtered output. This partial action strategy achieves enhanced filtering effectiveness without fully committing to the highest computational complexity, balancing quality and performance.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to an in-loop filter apparatus for video coding, wherein the in-loop filter apparatus is configured for processing a reconstructed frame from a decoded reconstructed video stream for generation of a filtered reconstructed frame, wherein the reconstructed frame comprises a plurality of pixels, each pixel being associated with a pixel value. The in-loop filter apparatus comprises a processing unit configured to: generate a set of overlapping or non-overlapping reference blocks of pixels from the reconstructed frame; determine for each reference block of pixels from the reconstructed frame a cluster of spatially or temporally similar blocks of pixels from the reconstructed frame or from another earlier decoded reconstructed frame on the basis of a similarity measure, wherein the cluster includes at least the reference block of pixels from the reconstructed frame; generate a set of auxiliary frames, including at least a first auxiliary frame and a second auxiliary frame, on the basis of the clusters corresponding to all reference blocks of pixels from the reconstructed frame; and generate a filtered reconstructed frame, wherein each pixel from the filtered reconstructed frame is a weighted sum of collocated pixels from auxiliary frames which are located around the position of the filtered pixel in the reconstructed frame.