Neural Network Loop Filter Supplementary Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face challenges in reducing computational complexity and memory bandwidth requirements, particularly in neural network-based filtering, which hinders the performance of video codecs like ITU-T H.266/Versatile Video Coding (VVC) and other next-generation standards.
Innovation Solution
The implementation of neural network-assisted loop filtering techniques that combine supplementary data, such as boundary strength and quantization parameters, to reduce computational complexity and memory bandwidth, allowing for increased filter performance under constrained complexity by processing more features or blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network-based filtering is used to improve video coding performance, then filtering quality is improved, but computational complexity and memory bandwidth requirements increase
Solution Approach 1:
The patent extracts and processes only the most essential supplementary data (boundary strength and quantization parameters) rather than all available data, reducing the computational burden while maintaining filtering quality. This selective extraction allows the neural network to focus on critical features that most impact filtering performance.
Solution Approach 2:
The patent performs preliminary processing of supplementary data by combining multiple data sets into a single integrated data structure before feeding it to the neural network. This preprocessing step reduces the complexity of the main filtering operation by organizing data in advance, reducing memory bandwidth requirements, and enabling more efficient neural network execution.
2Manufacturing precision
If more supplementary data is processed to improve filter performance, then filtering quality is improved, but memory bandwidth requirements increase
Solution Approach 1:
The patent merges multiple sets of supplementary data (boundary strength data, quantization parameter data, and other coding information) into a single integrated data structure. This combining approach reduces memory bandwidth requirements by eliminating redundant data access operations and consolidating memory operations, while still providing the neural network with comprehensive information for high-quality filtering.
3Device complexity
If computational complexity is reduced to meet hardware constraints, then hardware requirements are reduced, but filter performance deteriorates
Solution Approach 1:
The patent changes the parameter representation of supplementary data by combining multiple data sets into a unified format with a standardized data structure. This parameter transformation reduces computational complexity by simplifying data access patterns and reducing the number of separate memory operations, while the neural network maintains high filtering performance by receiving all essential information in an optimized format.
Data Source
AI summary
An example device for decoding video data includes: a memory configured to store video data; and a processing system comprising one or more processors implemented in circuitry, the processing system being configured to: decode at least a portion of a picture of video data; combine two or more sets of supplementary data for the at least portion of the picture into a single set of supplementary data; and execute a neural network filter, using the at least portion of the picture and the single set of supplementary data as inputs to the neural network filter, to filter the at least portion of the picture.