Neural Network Loop Filter With Multi-Scale Residual Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding and decoding technologies face challenges in improving filtering effects while reducing the operation complexity of neural network loop filters (NNLFs) to enhance coding and decoding efficiency.
Innovation Solution
Implement a neural network loop filter (NNLF) with residual units containing sequentially-connected residual blocks, each with multiple convolution layers of different kernel sizes to extract multi-scale receptive fields, thereby improving generalization capability and reducing operation complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a neural network loop filter (NNLF) is used to improve filtering effects, then the filtering quality is improved, but the operation complexity increases
Solution Approach 1:
The NNLF is segmented into multiple residual blocks, where each residual block contains parallel convolution layers with different kernel sizes. This segmentation allows the filter to process different spatial scales independently and then combine results, improving filtering quality while managing complexity through modular architecture.
Solution Approach 2:
The patent introduces multi-scale processing by adding the dimension of convolution kernel size variation. Instead of using a single kernel size, the network employs parallel convolution layers with different kernel sizes (e.g., 3x3, 5x5, 7x7), enabling the system to capture features at multiple spatial scales simultaneously, thereby improving filtering effectiveness.
2Adaptability or versatility
If multiple convolution layers with different kernel sizes are used to extract multi-scale receptive fields, then the generalization capability is improved, but the operation complexity increases
Solution Approach 1:
The residual blocks serve multiple functions: they extract features at different scales through parallel convolutions, perform non-linear transformations via activation functions, and enable gradient flow through residual connections. This multi-functionality improves generalization capability while avoiding the need for separate processing units for each function, thereby controlling operation complexity.
Solution Approach 2:
The network employs dynamic feature extraction by adaptively combining features from different kernel sizes. The residual connections allow the network to dynamically adjust the flow of information, enabling it to learn which scale and which residual path are most beneficial for different regions of the image, thus improving generalization without requiring a fixed rigid structure.
Data Source
AI summary
This application provides filtering, coding, and decoding methods performed by a computer device. The filtering method based on a neural network includes the following operations: generating input data of a neural network loop filter (NNLF) based on a target image, the input data containing at least the target image; inputting the input data into the NNLF, the NNLF containing residual units configured to extract image feature information, the residual unit containing a plurality of sequentially-connected residual blocks, and a first residual block among the plurality of residual blocks containing a plurality of convolution layers that are provided in parallel and have different convolution kernel sizes; and processing the target image using the NNLF to obtain a filtered image. According to the embodiments of this application, the filtering effect may be improved while reducing the operation complexity of the NNLF, thereby improving the video coding and decoding efficiency.