Video Encoding and Decoding With Adaptive Neural In-Loop Filter Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network-based in-loop filter methods in video encoding and decoding have limited filtering efficiency and may even reduce performance, failing to fully utilize the advantages of neural networks.
Innovation Solution
A method and apparatus for video encoding and decoding that involves determining a target in-loop filter model based on neural networks using relevant syntax elements, inputting reference sample information into the model for filtering, and optimizing the filtering process by selecting the appropriate model based on distortion cost values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a neural network-based in-loop filter method is used, then filtering performance is improved, but filtering efficiency deteriorates
Solution Approach 1:
The patent applies dynamics by making the filter model selection adaptive rather than static. The encoder dynamically selects from multiple candidate filter models (including neural network-based models and traditional models) based on the specific characteristics of each coding tree unit, allowing the system to balance performance and efficiency dynamically across different video regions and scenarios
Solution Approach 2:
The patent segments the video content by applying different filter models to different coding tree units. Instead of uniformly applying a single neural network model to all blocks, the system divides the video into multiple segments (coding tree units) and selects appropriate filter models for each segment based on local characteristics, thereby improving overall efficiency while maintaining performance where needed
2Reliability
If multiple candidate filter models are evaluated, then filtering performance is improved, but device complexity increases
Solution Approach 1:
The patent changes parameters by using distortion cost values as a selection criterion. The encoder evaluates candidate filter models based on distortion cost metrics and selects the model with the optimal balance between filtering performance and complexity. This parameter-based selection simplifies the decision process compared to evaluating multiple complex neural network architectures
Solution Approach 2:
The patent extracts only the necessary filter model information from multiple candidates. Instead of implementing and evaluating all possible neural network models, the system extracts and maintains a limited set of candidate models (including at least one neural network-based model and one traditional model), selecting from these extracted candidates based on distortion cost, thereby reducing device complexity while preserving performance benefits
Data Source
AI summary
A decoding method includes: decoding a bitstream to determine a relevant syntax element of a current coding tree unit; determining, based on the relevant syntax element, a target in-loop filter model of the current coding tree unit from candidate in-loop filter models based on neural network; determining reference sample information of the current coding tree unit; and inputting the reference sample information of the current coding tree unit into the target in-loop filter model for filtering, to output filtered reconstructed sample information.