Video In-Loop NN Filter Architecture With Separable Convolutions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network (NN)-based filtering techniques for video coding are highly complex and require significant processing power, limiting their applicability to devices with sufficient computational resources.
Innovation Solution
Implementing a video coder configured to perform a plurality of separable convolutions in parallel with a point-wise input convolution for NN-based filtering, reducing processing requirements while maintaining coding quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network-based filtering techniques are implemented for video coding, then coding quality is improved, but processing complexity and power consumption increase significantly
Solution Approach 1:
The patent segments the neural network filtering process into multiple parallel pipelines, each handling specific filtering operations. This segmentation allows the complex NN-based filtering to be divided into manageable parts that can be processed independently and in parallel, reducing the processing complexity burden on single processing units while maintaining overall coding quality improvement.
2Manufacturing precision
If neural network-based filtering techniques are implemented for video coding, then coding quality is improved, but processing complexity and power consumption increase significantly
Solution Approach 1:
The patent segments the neural network filtering process into multiple parallel pipelines, each handling specific filtering operations. This segmentation allows the complex NN-based filtering to be divided into manageable parts that can be processed independently and in parallel, reducing the processing complexity burden on single processing units while maintaining overall coding quality improvement.
3Device complexity
If traditional filtering methods are used, then processing complexity is low, but coding quality is insufficient
Solution Approach 1:
The patent creates a universal filtering architecture that can perform both traditional filtering operations and neural network-based filtering through the same parallel pipeline structure. This multi-functionality allows the system to adapt between different filtering methods based on computational requirements while maintaining high coding quality when NN filtering is applied, eliminating the need for separate dedicated hardware for each filtering type.
4Manufacturing precision
If full neural network filtering is applied, then coding quality is maximized, but processing time increases
Solution Approach 1:
The patent segments the neural network filtering process into multiple parallel pipelines, each handling specific filtering operations. This segmentation allows the complex NN-based filtering to be divided into manageable parts that can be processed independently and in parallel, reducing the processing complexity burden on single processing units while maintaining overall coding quality improvement.
Solution Approach 2:
The parallel pipeline architecture enables continuous processing of video data through multiple concurrent filtering operations. While one pipeline processes a particular frame or block, other pipelines simultaneously process different segments, ensuring that the filtering operation continues without interruption and maximizing throughput, thereby reducing overall processing time for complete video sequences.
Data Source
AI summary
A device for decoding video data receives a picture of video data; reconstructs a block of the picture of video data to generate a reconstructed block; and performs a neural network (NN)-based filter process on the reconstructed block to generate a filtered block, wherein the NN-based filter process includes performing a plurality of separable convolutions in parallel with a point-wise input convolution.