ML Model Selection Signaling Reduction in Video Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for video decoding using machine learning models incur a signaling penalty due to explicit signaling of model selection, which affects performance and increases bitrate, leading to redundancy in the bitstream.
Innovation Solution
The proposed solution reduces or eliminates the explicit signaling of machine learning model selection by deriving the appropriate model choice in the decoder based on existing information in the bitstream, such as slice information and quantization parameters, rather than explicitly signaling it, thereby reducing redundancy and improving transmission efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If explicit signaling of ML model selection is performed in the bitstream, then the decoder can select the proper ML model for decoding, but the signaling cost increases the bitrate and reduces coding efficiency
Solution Approach 1:
The decoder autonomously selects the ML model by evaluating existing bitstream parameters (quantization parameters, block size, slice type) against predefined criteria, eliminating the need for explicit model selection signaling from the encoder. This self-service mechanism maintains decoding accuracy while removing redundant signaling overhead.
Solution Approach 2:
Existing bitstream parameters that originally served for compression control are repurposed to also guide ML model selection. The quantization parameters, block dimensions, and slice information now serve dual functions: maintaining compression efficiency and determining appropriate ML model choices, thereby eliminating separate signaling channels.
2Reliability
If multiple ML models are provided for different video conditions, then the decoding performance is improved, but the complexity of model selection and signaling increases
Solution Approach 1:
The system changes the parameter space by using existing video coding parameters (quantization level, block size, slice type) as the basis for model selection. Instead of adding complex model selection logic, the solution transforms the selection process into a function of already-available parameters, reducing computational complexity while maintaining performance.
Solution Approach 2:
Different ML models are selectively applied to different local regions (slices, blocks) based on their specific characteristics such as quantization level and block size. This localized approach ensures that each region receives the most appropriate model without requiring global model selection complexity.
3Manufacturing precision
If ML loop filtering is applied per CTU with explicit signaling, then the quality of decoding is improved, but the signaling overhead per CTU increases the bitrate
Solution Approach 1:
The video frame is segmented into slices and further into blocks, with ML model selection decisions made at these segmented levels based on local parameters. This segmentation allows differentiated model application without requiring centralized signaling, as each segment independently determines its optimal model based on local characteristics.
Solution Approach 2:
Each CTU or block independently determines its ML model selection by evaluating its own parameters (quantization level, block size, slice type) against predefined selection criteria. This self-service approach eliminates the need for encoder-to-decoder signaling for each CTU's model selection, removing per-CTU signaling overhead while maintaining quality.
Data Source
AI summary
There is provided a method for decoding a video frame. The method comprises receiving an encoded video frame sequence data associated with a video frame sequence, wherein the video frame sequence includes the video frame. The method comprises deriving from the received encoded video frame sequence data (i) sample value information of one or more blocks and/or (ii) block information about said one or more blocks. The method further comprises, based on (i) the derived sample value information and/or (ii) the derived block information, selecting from a set of machine learning, ML, models at least one ML model to use for decoding the video frame. The method comprises decoding the video frame using the selected at least one ML model.