Video Coding and Decoding Filter Combination for Block Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding and decoding technologies face issues with image smoothing leading to loss of edge information when using deblocking filters, and neural network filters fail to adequately handle multiple transfer functions and color spaces, resulting in visible block noise when strength is set to zero.
Innovation Solution
A video decoding apparatus with a parameter decoder, bS derivation unit, DF unit, and NN filter unit, along with image combining, is employed to derive deblocking filter strength and apply neural network filters based on luma and chroma parameters, ensuring effective processing across various transfer functions and color spaces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If deblocking filter is applied using bS value to reduce block noise, then block noise is reduced, but image is smoothed and edge information is lost
Solution Approach 1:
The patent segments the image processing into two distinct filter paths: a deblocking filter (DF) that reduces block noise while preserving edges through gradient-based processing, and a neural network filter (NN) that handles residual artifacts. By dividing the filtering task into specialized components rather than using a single filter, the system reduces block noise without uniformly smoothing the entire image, thus preserving edge information.
Solution Approach 2:
The patent applies different filtering strategies to different regions and artifacts: the deblocking filter uses gradient calculations to apply strength selectively based on local edge characteristics, while the neural network filter processes residual artifacts with learned patterns. This local quality approach ensures that filtering strength is adapted to local image characteristics, reducing block noise in flat regions while preserving edges at boundaries.
2Productivity
If neural network filter strength is set to 0 to maintain processing speed, then processing complexity is reduced, but block noise becomes visible
Solution Approach 1:
The patent performs preliminary deblocking filter processing before the neural network filter stage. By pre-processing the image with the DF unit to remove major block artifacts, the neural network filter receives pre-conditioned input that requires less intensive processing, allowing the NN filter strength to be reduced or set to 0 in some cases while still achieving acceptable results, thus maintaining processing speed.
Solution Approach 2:
The deblocking filter acts as an intermediary between the decoding process and the neural network filter. It pre-processes the decoded image to remove prominent block noise, creating a better input for the NN filter. This intermediary processing reduces the burden on the NN filter, allowing it to operate with lower strength or be selectively applied, thereby maintaining processing speed while still reducing visible block noise.
3Device complexity
If single neural network filter is used for all color spaces and transfer functions, then device complexity is reduced, but processing accuracy for multiple transfer functions and color spaces deteriorates
Solution Approach 1:
The patent implements a universal neural network filter architecture that can process multiple color spaces (YCbCr, RGB) and transfer functions (SDR, HDR) through a single unified structure. The NN filter unit is designed to accept different input formats and apply appropriate filtering based on the input characteristics, eliminating the need for separate filter structures for each color space or transfer function while maintaining processing accuracy.
Solution Approach 2:
The patent uses parameter-based adaptation where the neural network filter's behavior is adjusted based on input parameters indicating color space and transfer function type. By changing filter parameters (such as filter kernels, strength values, or activation functions) based on the detected input characteristics, a single filter structure can accurately process diverse color spaces and transfer functions without requiring multiple dedicated filters.
Data Source
AI summary
In a case that a scaling value of neural network filter strength is set equal to 0, a decoded image before a deblocking filter is output, and thus block noise occurs. In a case that luma and chroma parameters are not taken into consideration, processing for an image of multiple transfer functions and chroma parameters is not appropriately performed. In a unit of a prescribed block, by using a parameter indicating a degree of application of the NN filter, an image after the deblocking filter and an image after an NN filter are combined using a different ratio of the images. Filter processing of a luma image is switched based on a luma parameter, and filter processing of a chroma image is switched based on a chroma parameter.