Decoder Neural Network Updates for Varying Encoding Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks implemented in decoders may yield unsatisfactory results due to changing encoding constraints, such as bitrate or framerate, necessitating methods for adapting to variable encoding conditions.
Innovation Solution
A method of generating a media bitstream that includes parameters for updating a neural network in a decoder, where the parameters are encoded and transmitted with media data, allowing the decoder to update the neural network based on syntax elements and encoding constraints, ensuring time-alignment with media processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a neural network is trained on specific encoding constraints, then it achieves good processing quality for those constraints, but it yields unsatisfactory results when encoding constraints change
Solution Approach 1:
The patent implements dynamic adaptation by allowing the neural network parameters to be updated incrementally based on current encoding constraints. The decoder receives update parameters in the bitstream that adjust the neural network weights and biases according to actual encoding conditions, transforming the static pre-trained model into a dynamic system that adapts to changing constraints.
Solution Approach 2:
The patent changes the parameters of the neural network by transmitting update parameters (weights and biases) in the bitstream. These parameter updates allow the neural network to adjust its internal state according to varying encoding constraints, enabling the system to maintain processing quality across different encoding conditions without complete retraining.
2Adaptability or versatility
If the neural network is updated with multiple sets of parameters, then it can adapt to different encoding constraints, but the bitstream complexity increases
Solution Approach 1:
The patent segments the parameter updates into distinct sets, each corresponding to specific encoding constraints. The syntax structure divides parameter information into organized groups with clear identification, allowing the decoder to selectively apply appropriate parameter sets based on current encoding conditions, thereby managing complexity through structured segmentation.
Solution Approach 2:
The patent creates a universal syntax structure that can handle multiple types of parameter updates for different encoding constraints. The same bitstream mechanism and syntax elements serve multiple functions: identifying parameter sets, indicating applicable constraints, and transmitting update data, thereby reducing overall system complexity through multi-functional design.
3Reliability
If the neural network parameters are updated frequently, then the processing quality improves under varying constraints, but the bitrate increases
Solution Approach 1:
The patent applies partial updates by transmitting only the necessary parameter changes (weights and biases) rather than complete parameter sets. This partial action approach updates only what is needed to maintain processing quality under varying constraints, reducing the bitrate overhead compared to transmitting full parameter sets frequently.
Data Source
AI summary
Described herein is a method of generating a media bitstream to transmit parameters for updating a neural network implemented in a decoder, wherein the method includes the steps of: (a) determining at least one set of parameters for updating the neural network; (b) encoding the at least one set of parameters and media data to generate the media bitstream; and (c) transmitting the media bitstream to the decoder for updating the neural network with the at least one set of parameters. Described herein are further a method for updating a neural network implemented in a decoder, an apparatus for generating a media bitstream to transmit parameters for updating a neural network implemented in a decoder, an apparatus for updating a neural network implemented in a decoder and computer program products comprising a computer-readable storage medium with instructions adapted to cause the device to carry out said methods when executed by a device having processing capability.