Deferred Preemption Tokens for GPU Command Stream Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In graphics processing systems, the arbitration of GPU resources between applications with varying priority levels is challenging, particularly when high-priority UI command streams need to preempt lower-priority command streams, leading to inefficiencies due to costly context switching overhead.
Innovation Solution
Implementing deferred preemption techniques by placing tokens in the GPU command stream to indicate specific locations where preemption is allowed, allowing the GPU to defer switching until these locations are reached, thereby reducing the overhead of context switching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the GPU allows preemption at any location in the command stream, then high-priority command streams can quickly access GPU resources, but context switching overhead increases significantly
Solution Approach 1:
The command stream is segmented into discrete locations, and preemption tokens are placed at specific segments where preemption is permitted. This segmentation allows the system to enable preemption only at optimized points rather than anywhere, reducing unnecessary context switching overhead while maintaining fast access capability when needed.
Solution Approach 2:
Different locations in the command stream have different preemption characteristics. By placing preemption tokens selectively at specific locations rather than uniformly throughout, the system applies local quality - allowing preemption where it benefits performance most while avoiding it where context switching overhead would be excessive.
2Loss of energy
If the GPU defers preemption until specific locations are reached, then context switching overhead is reduced, but the time to access GPU resources for high-priority streams increases
Solution Approach 1:
Preemption tokens are placed in advance at predetermined locations in the command stream before preemption is actually needed. This preliminary action ensures that when a high-priority stream needs to preempt, the GPU can immediately switch at the next token location without delay, balancing reduced overhead with fast access capability.
3Adaptability or versatility
If preemption tokens are placed frequently in the command stream, then preemption capability is enhanced, but the complexity of command stream management increases
Solution Approach 1:
Instead of placing preemption tokens at every possible location (excessive action), the system uses partial action by placing tokens only at selected locations where preemption is beneficial. This partial placement provides sufficient preemption capability while avoiding the unnecessary complexity that would result from ubiquitous token placement.
Data Source
AI summary
This disclosure is directed to deferred preemption techniques for scheduling graphics processing unit (GPU) command streams for execution on a GPU. A host CPU is described that is configured to control a GPU to perform deferred-preemption scheduling. For example, a host CPU may select one or more locations in a GPU command stream as being one or more locations at which preemption is allowed to occur in response to receiving a preemption notification, and may place one or more tokens in the GPU command stream based on the selected one or more locations. The tokens may indicate to the GPU that preemption is allowed to occur at the selected one or more locations. This disclosure further describes a GPU configured to preempt execution of a GPU command stream based on one or more tokens placed in a GPU command stream.