Decoder Neural Network Updates for Variable Bitrate Media Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks implemented in decoders may yield unsatisfactory results due to changing encoding constraints such as bitrate or framerate, necessitating methods and apparatus for adapting to variable encoding conditions.
Innovation Solution
A method of generating a media bitstream that includes encoding parameters for updating a neural network in a decoder, allowing time-aligned transmission and decoding of these parameters with media data to adapt the neural network to current encoding constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a neural network is trained on specific task data and encoding constraints, then it achieves good processing quality for that specific task, but it cannot adapt to changing encoding constraints such as bitrate or framerate variations
Solution Approach 1:
The patent makes the neural network dynamically adaptable by transmitting update parameters in the bitstream that allow the decoder's neural network to adjust its weights and processing behavior based on current encoding constraints such as bitrate and framerate, transforming a static trained network into a dynamic adaptive system
Solution Approach 2:
The patent changes the parameters of the neural network by transmitting update parameters (weight updates, bias updates, or activation function parameters) in the bitstream, allowing the decoder to modify its neural network parameters according to current encoding conditions rather than using fixed pre-trained parameters
2Adaptability or versatility
If update parameters for the neural network are transmitted in the bitstream, then the neural network can adapt to current encoding constraints, but the bitrate overhead increases
Solution Approach 1:
The patent extracts only the essential update parameters needed for neural network adaptation (such as weight updates, bias updates, or activation function parameters) and transmits them separately from the main media data, allowing adaptation functionality to be added with minimal bitrate overhead by transmitting only what is necessary
Solution Approach 2:
The patent transmits partial update parameters rather than complete neural network weight matrices, sending only the portions of parameters that are necessary for adapting to current encoding constraints, thereby reducing bitrate overhead while maintaining adaptability
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Described herein is a method of generating a media bitstream to transmit parameters for updating a neural network implemented in a decoder, wherein the method includes the steps of: (a) determining at least one set of parameters for updating the neural network; (b) encoding the at least one set of parameters and media data to generate the media bitstream; and (c) transmitting the media bitstream to the decoder for updating the neural network with the at least one set of parameters. Described herein are further a method for updating a neural network implemented in a decoder, an apparatus for generating a media bitstream to transmit parameters for updating a neural network implemented in a decoder, an apparatus for updating a neural network implemented in a decoder and computer program products comprising a computer-readable storage medium with instructions adapted to cause the device to carry out said methods when executed by a device having processing capability.