Neural Intra Prediction Mode Mapping for Video Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image and video coding schemes face challenges in achieving high compression efficiency due to limitations in leveraging spatial and temporal redundancy, particularly in intra and inter frame correlations.
Innovation Solution
Implementing neural network-based intra prediction modes in conjunction with regular intra prediction modes, utilizing a neural network-based predictor for coding blocks and determining corresponding non-neural network-based intra prediction modes based on block context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural network-based intra prediction modes are introduced, then prediction accuracy and compression efficiency are improved, but device complexity increases
Solution Approach 1:
The patent merges neural network-based prediction modes with regular intra prediction modes into a unified prediction system. The neural network predictor generates predictions that are combined with traditional prediction methods, allowing the system to leverage both the high accuracy of neural networks and the simplicity of regular prediction modes, thereby improving compression efficiency while managing complexity through integration rather than complete replacement
Solution Approach 2:
The patent introduces an intermediary mechanism that maps neural network-based prediction modes to corresponding regular prediction modes. This intermediary mapping allows the system to interpret and utilize neural network predictions within the framework of existing prediction infrastructure, enabling improved prediction accuracy while maintaining compatibility with existing codecs and avoiding excessive complexity
2Measurement precision
If neural network-based predictor is generated for each coding block, then prediction accuracy improves, but processing time and computational load increase
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing neural network predictions for coding blocks during the encoding process. These pre-computed predictions are then available for use during decoding without requiring real-time neural network inference, thereby improving prediction accuracy while reducing the processing time required during actual video decoding operations
Solution Approach 2:
The patent uses copying by creating a mapping or representation of neural network-based prediction modes that can be replicated and applied to multiple coding blocks. Instead of performing complex neural network computations for each block during decoding, the system copies and applies the predicted patterns, significantly reducing computational load and processing time while maintaining high prediction accuracy
Data Source
AI summary
Systems, methods, and instrumentalities are disclosed for performing intra prediction of a luminance coding block and/or a chrominance coding block and related signaling when neural network-based intra prediction is enabled. For a current coding block, information representative of a neural network-based prediction mode and a block context may be obtained. A neural network-based predictor may be generated for the current coding block based on the block context and the neural network-based prediction mode. A non-neural network-based intra prediction mode that corresponds to the generated neural network-based predictor may be determined. A prediction mode for a neighboring block may be determined based on the non-neural network-based intra prediction mode.