Video Decoding with CTU-Level NN Filtering and Parallel ALF
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding schemes face challenges in efficiently switching neural network parameters for small blocks like CTUs, leading to increased processing time due to sequential NN filtering and ALF processing.
Innovation Solution
A video decoding apparatus that includes a header decoder, CT information decoder, NN filter unit, and NN layer selection unit to adaptively select NN models and layers for small blocks, allowing parallel processing of ALF class classification and filtering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all network parameters are adaptively switched depending on input image, then filtering accuracy is improved, but device complexity increases and processing time increases
Solution Approach 1:
The patent segments the parameter switching into two levels: NN model index at slice/picture level (coarse granularity) and NN layer index at CTU level (fine granularity). This segmentation allows adaptive filtering at different granularities without overwhelming complexity, resolving the contradiction between filtering accuracy and device complexity.
Solution Approach 2:
The patent applies different levels of adaptivity to different regions: full NN model adaptation at slice/picture level and selective NN layer adaptation at CTU level. This local quality approach optimizes filtering accuracy where needed while controlling overall system complexity.
2Ease of operation
If NN filtering and ALF processing are performed sequentially, then processing order is simplified, but productivity decreases
Solution Approach 1:
The patent performs ALF class classification before NN filtering as a preliminary action. This allows the classification results to be ready when NN filtering completes, enabling immediate parallel execution of ALF filtering without waiting for sequential completion, thus improving productivity while maintaining operational simplicity.
Solution Approach 2:
By overlapping ALF class classification with NN filtering execution, the patent ensures continuous useful action - while NN filtering processes the image, ALF classification simultaneously prepares filter parameters, maximizing processor utilization and improving processing throughput.
3Ease of operation
If NN filtering is performed before ALF class classification, then processing flow is simplified, but loss of time increases
Solution Approach 1:
The patent reorders operations to perform ALF class classification before NN filtering, making it a preliminary action. This reordering enables parallel execution of classification and filtering operations, reducing total processing time while maintaining a logical and manageable processing flow.
Data Source
AI summary
In a case that all network parameters are adaptively switched, it is difficult to switch the parameters for each small block, such as a CTU. There is a problem in that, in a case that ALF class classification and filtering are successively performed in ALF parallel after application of an NN filter, processing requires time. By selecting an NN model in a slice header and selecting a network layer finetuned in a CTU, the NN model can be adaptively selected even for each small block. By performing ALF class classification in an image before being subjected to an NN filter and performing ALF processing using an image subjected to NN filtering processing and ALF class information, the NN filtering processing and the ALF class classification can be performed in parallel.