GPU Video Decoding With Multithreaded Command Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video decoding technologies face challenges with high computational complexity and resource consumption, particularly in decoding high-quality, high-bit rate video, which can be exacerbated by recent standards like H.264 and VC-1, and lack effective multithreading implementations that leverage GPU capabilities for parallel processing.
Innovation Solution
Implement a video decoder framework with layered data structures, picture extent discovery, a picture command queue, and task scheduling to facilitate multithreading, utilizing both CPU and GPU for asynchronous decoding operations, including intra prediction and loop filtering, and optimizing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If video decoding is performed using traditional single-threaded CPU methods, then decoding operations can be executed sequentially with simple architecture, but decoding speed is slow and computational complexity is high
Solution Approach 1:
The video decoding process is divided into multiple independent threads, each handling different aspects of decoding (e.g., one thread for entropy decoding, another for inverse transform, another for motion compensation). This segmentation allows parallel execution and significantly increases decoding speed while managing complexity through modular design
Solution Approach 2:
The patent transitions from single-threaded sequential processing to multi-threaded parallel processing, adding the dimension of concurrency. By utilizing multiple CPU threads and integrating GPU acceleration, the system exploits temporal and spatial parallelism to dramatically improve decoding throughput
2Productivity
If GPU acceleration is implemented for video decoding, then decoding speed increases and computational burden is reduced, but synchronization complexity between CPU and GPU increases
Solution Approach 1:
A command buffer is introduced as an intermediary mechanism between CPU and GPU. The CPU writes decode commands to the command buffer, which the GPU reads and executes asynchronously. This intermediary abstraction simplifies the interaction protocol and reduces synchronization complexity while enabling high-throughput decoding
Solution Approach 2:
The system performs preliminary actions by pre-processing decoding commands and preparing data in the command buffer before GPU execution. This allows the GPU to execute decoding operations asynchronously without waiting for CPU readiness, improving throughput while managing complexity through advance preparation
3Loss of time
If multithreading is implemented to process multiple pictures simultaneously, then decoding time is reduced, but memory resource management becomes more complex
Solution Approach 1:
Memory resources are segmented into thread-specific buffers and shared buffers, with each thread managing its own memory region. This segmentation enables simultaneous processing of multiple pictures without memory conflicts, reducing decoding time while simplifying memory management through clear ownership boundaries
Solution Approach 2:
The memory management system dynamically allocates and deallocates memory regions based on the actual decoding needs of each thread. This dynamic approach allows efficient utilization of memory resources across multiple threads, reducing overall decoding time while adapting to varying memory requirements without complex static allocation schemes
4Quantity of substance
If lossy compression is applied to reduce bit rate, then storage and transmission costs decrease, but video quality deteriorates
Solution Approach 1:
The system dynamically adjusts compression parameters (quantization step size, transform coefficient thresholds) based on the balance between bit rate reduction and quality preservation requirements. By changing these parameters adaptively during decoding, the system optimizes the trade-off between reduced bit rate and maintained video quality according to specific application needs
Data Source
AI summary
Video decoding innovations for multithreading implementations and graphics processor unit (“GPU”) implementations are described. For example, for multithreaded decoding, a decoder uses innovations in the areas of layered data structures, picture extent discovery, a picture command queue, and/or task scheduling for multithreading. Or, for a GPU implementation, a decoder uses innovations in the areas of inverse transforms, inverse quantization, fractional interpolation, intra prediction using waves, loop filtering using waves, memory usage and/or performance-adaptive loop filtering. Innovations are also described in the areas of error handling and recovery, determination of neighbor availability for operations such as context modeling and intra prediction, CABAC decoding, computation of collocated information for direct mode macroblocks in B slices, reduction of memory consumption, implementation of trick play modes, and picture dropping for quality adjustment.