Neural Video Coding with Partial Layer Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in achieving bitrate reduction without sacrificing picture quality, particularly in limited bandwidth scenarios, and there is a need for improved compression and decompression techniques that reduce training time and signaling overhead.
Innovation Solution
The use of a neural network with updatable layers, where only a subset of layers is trained and updated, reducing training time and signaling overhead by encoding information on these layers into separate bitstreams, and utilizing pre-configured layers as references for delta-weight training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If all layers of the neural network are trained and updated, then picture quality is improved, but training time increases
Solution Approach 1:
The neural network is divided into two distinct segments: pre-configured layers (with fixed parameters) and updatable layers (with trainable parameters). This segmentation allows the system to train only the necessary updatable layers while keeping the pre-configured layers static, thereby reducing training time while still improving picture quality through targeted updates to the flexible layers.
2Measurement precision
If all parameters of the neural network are encoded into the bitstream, then decoding accuracy is improved, but signaling overhead increases
Solution Approach 1:
The patent extracts only the essential updatable parameters from the neural network and encodes them into the bitstream, while the pre-configured parameters are excluded from transmission. This extraction approach ensures that decoding accuracy is maintained through transmission of necessary updates, while signaling overhead is reduced by omitting redundant pre-configured parameters that can be reconstructed at the decoder side.
3Productivity
If the neural network is fully trained on custom data, then compression performance is improved, but complexity increases
Solution Approach 1:
The neural network is pre-configured with initial layers before custom training begins. This preliminary action establishes a foundation that guides the subsequent custom training process, allowing the system to achieve improved compression performance while reducing training complexity by focusing updates only on the updatable layers rather than the entire network.
Data Source
AI summary
The present disclosure relates to encoding and decoding of a picture or a plurality of pictures (e.g. video) using a neural network which is partially trained online. Accordingly, at an encoder, one or more layers are selected which are to be trained. Then, the training of the neural network is performed, in which parameters of the selected layers are updated. The parameters of the remaining layers are maintained and not updated. The updated parameters are provided within a bitstream. The picture(s) is/are also encoded. The decoder receives an indication of the updated parameters, updates only those parameters indicated, and applies the so obtained neural network.