Microdosing Decompression Data for Compact Low-Bitrate Video Decoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video codecs require large decoder models that are computationally intensive and memory-heavy, making them impractical for devices with limited resources, and retraining these models is complicated and time-consuming.
Innovation Solution
Implement a knowledge distillation approach to transfer the knowledge of a large teacher network to a smaller student network, reducing the decoder size and computational requirements while maintaining performance, using microdosing to send sequence-specific decompression data alongside compressed video frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large decoder model is used to maintain visual quality, then the decoding accuracy is improved, but the memory requirements and computational complexity increase
Solution Approach 1:
The patent creates a student network that copies the essential functionality of the teacher network through knowledge distillation. The student network is trained to replicate the teacher's predictions and intermediate representations, achieving similar decoding accuracy with significantly fewer parameters and reduced memory requirements.
Solution Approach 2:
The patent extracts only the necessary knowledge from the large teacher network to train a smaller student network. By distilling knowledge from multiple teacher networks with different architectures and training data, the student network captures essential decoding capabilities without requiring the full complexity of any single large model.
2Measurement precision
If a large decoder model is used to maintain visual quality, then the decoding accuracy is improved, but the decoding speed decreases
Solution Approach 1:
The student network copies the teacher network's decoding capabilities in a more efficient manner. By learning from the teacher's predictions and intermediate features, the student network achieves comparable visual quality while requiring fewer computational operations, thereby increasing decoding speed by approximately 50%.
3Measurement precision
If existing large decoder models are retrained to improve performance, then the decoding accuracy may be improved, but the training time and computational resources increase
Solution Approach 1:
Instead of retraining large models, the patent copies knowledge from multiple pre-trained teacher networks into a smaller student network. This knowledge distillation approach transfers learned representations and predictions without requiring extensive retraining, significantly reducing training time and computational resource requirements.
Solution Approach 2:
The teacher networks are pre-trained on their respective datasets before distillation. This preliminary training allows the student network to learn from already-optimized models, avoiding the need to train from scratch and reducing overall training time while maintaining decoding accuracy.
Data Source
AI summary
A system includes a machine learning (ML) model-based video encoder configured to receive an uncompressed video sequence including multiple video frames, determine, from among the multiple video frames, a first video frame subset and a second video frame subset, encode the first video frame subset to produce a first compressed video frame subset, and identify a first decompression data for the first compressed video frame subset. The ML model-based video encoder is further configured to encode the second video frame subset to produce a second compressed video frame subset, and identify a second decompression data for the second compressed video frame subset. The first decompression data is specific to decoding the first compressed video frame subset but not the second compressed video frame subset, and the second decompression data is specific to decoding the second compressed video frame subset but not the first compressed video frame subset.