Neural Intra Prediction Mode Derivation for Video Coding Residuals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding techniques struggle with inefficiencies in generating prediction blocks, particularly when using neural networks or AI/ML models, as they do not determine conventional intra-prediction modes, leading to suboptimal encoding and decoding performance.
Innovation Solution
Employing decoder-side intra mode derivation (DIMD) to determine an equivalent intra-prediction mode that best matches the prediction block generated by a neural network, allowing for improved encoding and decoding of residual blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural network-based prediction block generation is used, then prediction accuracy is improved, but compatibility with conventional video coding standards deteriorates
Solution Approach 1:
The patent introduces an intermediary mechanism (mode derivation unit) that translates between neural network prediction outputs and conventional intra-prediction modes. This mediator enables the system to use advanced neural network-based prediction while maintaining compatibility with existing video coding standards by finding equivalent conventional modes that produce similar prediction blocks.
Solution Approach 2:
The patent changes the parameter representation by deriving an equivalent intra-prediction mode index from neural network predictions. Instead of directly using neural network outputs, the system transforms them into conventional mode parameters (mode indices) that can be processed by existing video coding infrastructure, thus bridging the gap between advanced AI methods and traditional standards.
2Adaptability or versatility
If conventional intra-prediction modes are used, then compatibility with video coding standards is maintained, but encoding efficiency deteriorates
Solution Approach 1:
The system performs self-service by automatically deriving equivalent intra-prediction modes from neural network predictions without requiring manual intervention or complex reconfiguration. The mode derivation unit autonomously matches neural network outputs to the best-fit conventional modes, enabling the system to maintain standard compatibility while leveraging AI-based prediction improvements.
Solution Approach 2:
The patent implements a feedback mechanism where the derived intra-prediction mode is used to guide subsequent encoding decisions. The system continuously refines the mode selection by comparing neural network predictions with conventional mode outputs, creating a feedback loop that improves encoding efficiency while maintaining compatibility with video coding standards.
3Reliability
If neural network models are integrated into video coding, then prediction performance is improved, but system complexity increases
Solution Approach 1:
The patent segments the video coding system into distinct functional modules: a neural network prediction unit, a mode derivation unit, and a conventional encoding unit. This segmentation allows the complex neural network component to be isolated and managed separately, reducing overall system complexity while maintaining prediction performance improvements.
Solution Approach 2:
The system creates a simplified copy or representation of neural network predictions in the form of equivalent intra-prediction mode indices. Instead of directly integrating complex neural network models into the entire video coding pipeline, the patent copies the essential predictive information into a format compatible with conventional encoders, thereby reducing system complexity while preserving performance benefits.
Data Source
AI summary
An example device for decoding video data includes a memory configured to store video data; and a processing system comprising one or more processors implemented in circuitry, the processing system being configured to: generate a prediction block for a current block of video data using a neural network; determine an equivalent intra mode for the prediction block from a set of available intra-prediction modes using decoder-side intra mode derivation (DIMD), the equivalent intra mode representing one of the intra-prediction modes that would generate an intra-prediction block that would best match the prediction block generated using the neural network; decode a residual block for the current block of the video data based on the equivalent intra mode; and combine the prediction block with the residual block to decode the current block of the video data.