Neural Video Decoding With Dynamic GPU/TPU Configuration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video decoding methods using parallelized processing units, such as GPUs or TPUs, lack flexibility and efficiency in adapting to different types of audio or video content, leading to suboptimal decoding performance.
Innovation Solution
A method and device that utilize a processor and a parallelized processing unit to decode data streams by processing a first set of data to configure the parallelized processing unit, using an artificial neural network to adaptively process a second set of data, allowing for flexible and efficient decoding based on the first set's characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a parallelized processing unit (GPU/TPU) is used for video decoding, then decoding speed and productivity are improved, but adaptability to different content types deteriorates
Solution Approach 1:
The patent applies dynamics by making the neural network configuration dynamic rather than static. The system dynamically selects and configures neural network parameters (such as convolution kernels, activation functions, and network depth) based on the characteristics of the input video content. This allows the parallelized processing unit to maintain high decoding speed while adapting to different content types by changing its processing configuration in real-time.
Solution Approach 2:
The patent implements parameter changes by modifying neural network parameters according to content characteristics. Different video content types (e.g., high-motion scenes, low-motion scenes, different resolutions) trigger different parameter configurations in the neural network. This enables the system to optimize decoding performance for each content type while utilizing the parallelized processing unit's high-speed capabilities.
2Device complexity
If a fixed neural network configuration is used for decoding, then device complexity is reduced, but decoding performance deteriorates
Solution Approach 1:
The patent applies segmentation by dividing the neural network configuration into multiple independent parameter sets, each optimized for specific content types. Instead of using a single complex configurable system, the patent segments the configuration space into discrete, pre-defined parameter sets that can be selectively applied. This reduces the complexity of real-time configuration while maintaining high decoding performance through appropriate parameter selection.
3Adaptability or versatility
If content-specific processing is implemented, then adaptability is improved, but processing time increases
Solution Approach 1:
The patent implements preliminary action by pre-configuring multiple neural network parameter sets during the off-line phase, each optimized for specific content characteristics. During actual decoding, the system only needs to select from these pre-prepared configurations rather than creating them in real-time. This eliminates the time penalty associated with content-specific processing while maintaining high adaptability to different video types.
Data Source
AI summary
In a method for decoding a data stream by way of an electronic device (10) including a processor (14), and a parallelized processing unit (16) designed to perform a plurality of operations of the same type in parallel at a given time, the data stream includes a first dataset (Fet) and a second dataset (Fnn) representative of audio or video content. The decoding method includes the processor (14) processing data from the first dataset (Fet), obtaining the audio or video content by processing (E70) data from the second dataset (Fnn) using a process depending at least partially on the data from the first set (Fet) and using an artificial neural network (18) implemented by the parallelized processing unit (16).