Neural Intra Prediction for Image Encoding and Block Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image encoding and decoding technologies, such as H.264 AVC and HEVC, struggle with efficiently removing spatial redundancy in images using intra prediction, particularly when employing fixed filter coefficients, and recent AI-based methods lack efficiency in predicting pixel values.
Innovation Solution
An image decoding and encoding method and apparatus utilize neural networks to determine intra prediction feature data, intra flow information, and weight information to predict pixel values more accurately, reconstructing blocks based on these predictions, and encoding the intra prediction feature data efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If fixed filter coefficients are used for intra prediction, then the encoding/decoding process is simple, but the prediction accuracy and efficiency are limited
Solution Approach 1:
The patent applies dynamics by transitioning from fixed filter coefficients to dynamic neural network-based filtering. The neural network adapts its filtering operations based on the specific characteristics of each block and its neighboring pixels, allowing the system to optimize prediction accuracy dynamically rather than using static fixed coefficients for all cases.
Solution Approach 2:
The patent changes the parameters from fixed filter coefficients to learned neural network weights and activation functions. By training the neural network on diverse image data, the system learns optimal filtering parameters adaptively, enabling it to handle various prediction scenarios with higher accuracy than fixed coefficients can provide.
2Measurement precision
If AI-based methods are used for intra prediction, then prediction accuracy improves, but computational efficiency decreases
Solution Approach 1:
The patent applies preliminary action by training the neural network offline before actual encoding/decoding operations. During training, the network learns optimal prediction strategies from large datasets. Once trained, the network structure and parameters are fixed, allowing fast inference during actual video processing without requiring complex real-time training, thus improving computational efficiency.
Solution Approach 2:
The patent uses copying by creating a trained neural network model that can be deployed and reused for multiple encoding/decoding operations. Instead of performing complex AI computations repeatedly, the system copies the learned knowledge into the network structure and applies it efficiently across different video blocks and sequences.
3Device complexity
If traditional intra prediction methods are used, then computational complexity is low, but spatial redundancy removal efficiency is insufficient
Solution Approach 1:
The patent introduces an intermediary neural network layer between the traditional prediction process and the final pixel value generation. This neural network intermediary processes the relationship between current block pixels and neighboring pixels, learning complex spatial dependencies that traditional methods miss, thereby improving spatial redundancy removal efficiency without excessively increasing computational complexity.
Data Source
AI summary
An image decoding method and apparatus obtain intra prediction feature data of a current block from a bitstream, determine an intra flow indicating a reference pixel of a current pixel in the current block, by applying the intra prediction feature data, neighboring pixels of the current block, and coding context information of the current block to a neural network, obtain a predicted pixel of the current pixel based on the intra flow of the current block, and reconstruct the current block based on the predicted pixel.