Gang Scheduling Onboard GPU With User-Based Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Dependencies between commands in different queues in a graphics processing unit (GPU) lead to serialization of virtual pipelines, reducing performance due to delayed execution of command buffers, and while reducing buffer sizes increases concurrency, it comes at the cost of increased context switching overhead.
Innovation Solution
Implementing gang scheduling, where multiple queues are allocated and scheduled together for concurrent execution on virtual pipelines, allowing applications to manage queue groups, modify allocations, and introduce delays to resolve dependencies without waiting for entire command buffers to complete.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If command buffers are executed with full completion waiting for dependencies, then correctness is maintained, but execution speed decreases due to serialization
Solution Approach 1:
The patent segments command execution into independent command buffers that can be scheduled individually. Each command buffer is treated as a separate executable unit that can be dispatched to different virtual pipelines independently, allowing parallel execution of independent commands while maintaining dependency correctness through proper synchronization barriers.
Solution Approach 2:
The patent performs preliminary dependency analysis and command buffer preparation before execution. The system pre-processes command buffers to identify independent executable units and prepares their execution schedules in advance, allowing the GPU to execute commands as soon as their dependencies are resolved without waiting for entire command buffer completion.
2Productivity
If command buffer sizes are reduced to increase concurrency, then parallel execution increases, but context switching overhead increases
Solution Approach 1:
The patent implements dynamic command buffer management where the system adaptively adjusts command buffer dispatching based on current GPU state and pipeline availability. Command buffers are dynamically allocated and dispatched to virtual pipelines based on real-time resource availability, allowing optimal concurrency without excessive context switching by matching buffer execution to actual pipeline readiness.
3Adaptability or versatility
If multiple queues are scheduled independently, then queue autonomy is maintained, but pipeline serialization occurs due to command dependencies
Solution Approach 1:
The patent merges multiple independent queues into a unified scheduling system that manages command buffers from multiple queues together. The scheduler combines commands from different queues into a unified execution schedule, allowing it to optimize for parallel execution across virtual pipelines while respecting queue boundaries and dependency relationships, thus eliminating serialization without sacrificing queue autonomy.
Data Source
AI summary
A processing unit such as a graphics processing unit (GPU) includes a set of queues that stores command buffers prior to execution in a corresponding plurality of pipelines. The processing unit also implements a kernel mode driver that allocates a first subset of the set of queues to a first application in response to receiving registration requests from the first application. The processing unit further includes a scheduler that schedules command buffers in the first subset of the set of queues for concurrent execution on a first subset of the set of pipelines. In some cases, an interrupt is generated in response to execution of a first command in a first command buffer in the first queue or the second queue. The interrupt includes an address indicating a location of a routine to be executed by a second subset of the plurality of pipelines.