Neural Network Loop Filtering With Multi-Scale Convolutions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network loop filters (NNLFs) struggle to balance filtering performance and complexity, leading to poor picture filtering effects in video compression.
Innovation Solution
Implement a neural network-based picture filtering method using convolution layers processed in multi-level receptive field (MLRF), tensor decomposition, and group convolution (GC) modes to reduce operation complexity while maintaining performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a standard neural network loop filter is used, then filtering performance can be achieved, but the operation complexity becomes too high
Solution Approach 1:
The patent divides the standard convolution layer into multiple sub-convolution layers with different kernel sizes (e.g., 1x1, 3x3, 5x5 convolutions). This segmentation allows the network to capture multi-scale features while reducing the computational burden of any single convolution operation, thereby lowering overall operation complexity while maintaining filtering performance.
Solution Approach 2:
The patent applies different convolution kernel sizes to different regions or features within the neural network loop filter. By using smaller kernels (1x1) for certain features and larger kernels (3x3, 5x5) for others, the system optimizes the balance between capturing local details and reducing computational complexity, achieving high filtering performance with manageable operation complexity.
2Manufacturing precision
If more convolution layers are added to improve filtering, then picture quality improves, but filtering complexity increases
Solution Approach 1:
The patent merges multiple convolution operations with different kernel sizes into a unified neural network loop filter structure. By combining 1x1, 3x3, and 5x5 convolutions in a coordinated manner, the system achieves the filtering effect of many layers while maintaining a more compact and computationally efficient structure, thus improving picture quality without proportionally increasing filtering complexity.
Solution Approach 2:
The patent introduces a multi-scale dimension by incorporating convolution layers with varying kernel sizes (1x1, 3x3, 5x5). This dimensional approach allows the network to process features at different scales simultaneously, achieving high picture quality through multi-scale feature extraction while avoiding the need for excessively deep networks that would increase filtering complexity.
Data Source
AI summary
This application provides neural network-based picture filtering method performed by a computer device. The method includes the following operations: determining input information of a neural network loop filter (NNLF) for a target picture; and filtering the input information through the NNLF to obtain a filtered picture of the target picture, the NNLF including a plurality of convolution layers configured to extract picture feature information, and the plurality of convolution layers being obtained by processing a standard convolution layer in at least two of a multi-level receptive field (MLRF) mode, a tensor decomposition mode, and a group convolution (GC) mode. Thus, the filtering operation complexity of the NNLF is reduced without reducing the filtering performance of the NNLF, thereby improving the picture filtering effect.