GAN-Based Intra Prediction for Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding techniques, such as HEVC and VVC, suffer from limited reference information and fixed prediction patterns in intra prediction, leading to inaccurate predictions and poor coding performance.
Innovation Solution
Implementing Generative Adversarial Network (GAN) based intra prediction, which models intra prediction as an inpainting task to exploit more available information and generate flexible prediction patterns by using a generator network to infer missing image parts based on reconstructed pixels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional HEVC intra prediction with fixed prediction patterns is used, then device complexity is reduced, but prediction accuracy deteriorates due to limited reference information
Solution Approach 1:
The patent introduces a neural network model as an intermediary between the input image and prediction blocks. This neural network processes reference pixels and generates enhanced prediction results that go beyond traditional fixed prediction patterns, improving prediction accuracy while managing complexity through learned representations
Solution Approach 2:
The patent transforms the prediction process by changing from fixed deterministic prediction patterns to learned parameterized transformations through neural networks. The model learns optimal prediction parameters from training data, enabling adaptive prediction that improves accuracy while the trained model structure manages computational complexity
2Measurement precision
If more reference information is exploited for intra prediction, then prediction accuracy improves, but computational complexity increases
Solution Approach 1:
The patent performs preliminary training of the neural network model offline using extensive reference information and ground truth data. This preliminary action pre-computes optimal prediction strategies, allowing the deployed model to achieve high prediction accuracy with reduced runtime computational complexity since the heavy learning process occurs beforehand
Solution Approach 2:
During training, the patent uses excessive reference information and multiple prediction targets to comprehensively train the model. The trained model then captures essential prediction patterns, allowing accurate predictions with manageable computational complexity during actual video encoding
Data Source
AI summary
Systems and methods which provide Generative Adversarial Network (GAN) based intra prediction for video coding are described. GAN based intra prediction as implemented according to embodiments may be modeled as an inpainting task. For example, intra prediction may be formulated as a learning based inpainting task, wherein a latent variable is designed to control different generation modes. GAN based intra prediction provided according to embodiments of the invention may be implemented alone or in combination with one or more other video compression technique, such as a direction intra prediction technique. The intra prediction module of such a HEVC encoder/decoder may be redesigned to also apply GAN based inpainting in intra prediction, wherein Rate-Distortion Optimization (RDO) may be performed to select the best intra prediction mode between the intra prediction approaches.