Neural Network In-Loop Filtering With Selective Video Coding Inputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current implementations of neural network in-loop filters in video encoders and decoders receive limited inputs, which do not always improve the filtering profile and can lead to complexity concerns.
Innovation Solution
The method involves generating reconstructed video frames and applying neural network in-loop filters with additional inputs such as prediction settings, transform settings, and importance settings to enhance filtering profiles while addressing complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If additional inputs are provided to the neural network in-loop filter, then the filtering performance is improved, but the device complexity increases
Solution Approach 1:
The patent extracts only the most relevant inputs from the available video coding parameters. Instead of feeding all possible parameters to the neural network, the invention selectively extracts prediction settings, transform settings, and importance settings that have the most significant impact on filtering performance, thereby improving filtering quality while controlling complexity
Solution Approach 2:
The patent applies different input selections for different regions or types of video content. The importance setting allows the system to provide different levels of input detail for different blocks or regions, ensuring high filtering quality where needed while reducing complexity in less critical areas
2Device complexity
If selective inputs are provided to the neural network in-loop filter, then the complexity is reduced, but the filtering performance may be degraded
Solution Approach 1:
The patent changes the parameters provided to the neural network by transforming video coding parameters into a simplified set of key inputs. The prediction setting, transform setting, and importance setting are derived from multiple original parameters through specific transformation rules, maintaining the essential information needed for high-quality filtering while reducing the overall parameter count and complexity
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Neural network in-loop filter. One method for filtering a reconstructed video frame includes generating the reconstructed video frame and applying one or more in-loop filters to the reconstructed video frame to generate a filtered reconstructed video frame. The one or more in-loop filters includes a neural network in-loop filter receiving as input a prediction setting used with the reconstructed video frame, a transform setting used with the reconstructed video frame, or an importance setting of the reconstructed video frame.