Neural Network Video Coding Adaptive Parameter Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network-based video/image compression methods primarily focus on enhancing encoding tools but do not fully utilize neural net prediction for adaptive content adaptation, limiting their efficiency in video/image compression.
Innovation Solution
An enhanced encoding method that derives prediction blocks using neural networks, encodes prediction error blocks, and retrains the neural network with training signals to adaptively adjust parameters for improved prediction, allowing the neural net to adapt to encoded content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural network-based prediction is used for video/image compression, then prediction accuracy is improved, but the system cannot adapt to different content types efficiently
Solution Approach 1:
The patent implements dynamic adaptation by allowing the neural network parameters to be updated during the encoding process. The system derives training signals from the actual encoding results (prediction error blocks) and retrains the neural network to adapt to the specific content being encoded, transforming a static prediction system into a dynamic one that evolves during operation.
Solution Approach 2:
The system establishes a feedback loop where the prediction error blocks generated during encoding are used as training signals to retrain the neural network. This feedback mechanism allows the system to learn from its own performance and continuously improve prediction accuracy for the current content type.
2Productivity
If traditional encoding tools are enhanced with neural networks, then encoding performance is improved, but compression efficiency is limited due to lack of adaptive retraining
Solution Approach 1:
The neural network performs self-improvement by using the encoding residuals (prediction error blocks) as training data to update its own parameters. This self-service mechanism enables the system to automatically optimize its prediction capabilities without external intervention, directly improving compression efficiency.
Solution Approach 2:
The system dynamically changes the neural network parameters (weights and biases) during the encoding process based on the specific content characteristics. By adapting parameters to match the current content type, the system achieves better compression efficiency for diverse video and image materials.
Data Source
AI summary
A method includes maintaining a set of parameters or weights derived through online learning for a neural net; transmitting an update of the parameters or weights to a decoder; deriving a first prediction block based on an output of the neural net using the parameters or weights; deriving a first encoded prediction error block through encoding a difference of the first prediction block and a first input block; encoding the first encoded prediction error block into a bitstream; deriving a reconstructed prediction error block based on the first encoded prediction error block; deriving a second prediction block based on an output of the neural net using the parameters or weights and the reconstructed prediction error block; deriving a second encoded prediction error block through encoding a difference of the second prediction block and a second input block; and encoding the second encoded prediction error block into a bitstream.