Joint-Component Neural Network Filtering for Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face challenges in effectively filtering distorted decoded pictures, particularly in advanced video codecs like ITU-T H.266/Versatile Video Coding (VVC) and subsequent standards, where redundancy reduction and artifact minimization are critical for improved video quality.
Innovation Solution
The implementation of a downsampling convolutional neural network layer for filtering decoded video data, which involves applying the layer to a first color component to downsample it, filtering a second color component, concatenating the downsampled and filtered components, and further filtering the concatenated components to produce a refined output, effectively addressing distortion and redundancy in video data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional filtering methods are used on decoded video data, then processing complexity is low, but video quality and artifact reduction are insufficient
Solution Approach 1:
The filtering process is segmented into multiple sequential operations: downsampling the first color component, filtering the second color component, concatenating them, and performing final filtering. This segmentation allows each step to be optimized independently while achieving overall high video quality with manageable complexity at each stage.
Solution Approach 2:
The patent introduces a dimensional transformation by downsampling the first color component to match the resolution of the second color component, then concatenating them in a joint component space. This dimensional change enables the neural network to process color components together in a unified feature space, improving artifact reduction capability.
2Manufacturing precision
If color components are processed separately, then processing is simpler, but color consistency and overall quality improvement are limited
Solution Approach 1:
The patent merges the processing of multiple color components by downsampling the first color component to the same resolution as the second color component, then concatenating them into a joint tensor. This merging allows the neural network to analyze and filter color components together, improving color consistency and overall quality while the modular architecture keeps processing complexity manageable.
3Productivity
If downsampling is applied to reduce computational load, then processing speed increases, but detail information may be lost
Solution Approach 1:
The downsampling operation is performed as a preliminary action before the main filtering process. By downsampling the first color component early in the pipeline, the patent reduces the computational load for subsequent neural network operations while preserving essential information through the concatenation with the second color component, which retains its original resolution.
Data Source
AI summary
An example device for decoding video data includes a memory configured to store video data; and one or more processors implemented in circuitry and configured to: apply a downsampling convolutional neural network layer to a first color component of a block of video data, the first color component of the block having a first size, wherein applying the downsampling convolutional neural network layer to the first color component generates a downsampled first color component having a second size smaller than the first size; filter a second color component having the second size to form a filtered second color component; concatenate the downsampled first color component with the filtered second color component to form concatenated color components; and filter the concatenated color components to form a filtered concatenated component including a filtered downsampled first color component.