GPU Work Graph Scheduler With Independent Local Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional parallel processors suffer from increased latency due to local schedulers residing outside shader engines, necessitating communication through additional hierarchy levels, leading to longer scheduling times for work items in work queues.
Innovation Solution
Implementing a hierarchical scheduler system where a global scheduler communicates with local schedulers through a shared cache, allowing each local scheduler to operate independently without direct communication, reducing latency by enabling direct access to local caches and facilitating finer-grained scheduling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If local schedulers reside outside shader engines, then hierarchical work scheduling can be implemented, but scheduling latency increases due to additional communication hierarchy levels
Solution Approach 1:
The scheduling function is segmented into two independent levels: a global scheduler that manages work streams at the GPU level, and local schedulers that manage work items at the shader engine level. Each local scheduler operates independently within its shader engine, eliminating the need for cross-hierarchy communication for work item scheduling. This segmentation resolves the contradiction by maintaining hierarchical structure while removing latency-causing communication paths.
Solution Approach 2:
The patent introduces a new dimensional organization by placing local schedulers inside shader engines rather than outside them. This spatial reorganization creates a parallel scheduling dimension where local schedulers can independently access and dispatch work items without traversing the traditional hierarchical communication path, thereby reducing scheduling latency while preserving hierarchical management capabilities.
2Reliability
If local schedulers communicate through additional hierarchy levels, then centralized control is maintained, but work items experience longer scheduling times
Solution Approach 1:
The global scheduler performs preliminary action by allocating work streams to shader engines in advance. Once allocated, local schedulers independently manage work items within their assigned shader engines without requiring further centralized communication. This preliminary allocation maintains centralized control over resource distribution while enabling fast local scheduling execution.
Solution Approach 2:
The patent introduces shader engines as intermediary entities that host local schedulers. These intermediaries bridge the global scheduler and processing units, allowing work items to be scheduled locally without direct communication between the global scheduler and individual processing units. This intermediary structure preserves centralized oversight while enabling rapid local scheduling operations.
Data Source
AI summary
Systems, apparatuses, and methods for implementing a hierarchical scheduler. In various implementations, a processor includes a global scheduler, and a plurality of independent local schedulers with each of the local schedulers coupled to a plurality of processors. In one implementation, the processor is a graphics processing unit and the processors are computation units. The processor further includes a shared cache that is shared by the plurality of local schedulers. Each of the local schedulers also includes a local cache used by the local scheduler and processors coupled to the local scheduler. To schedule work items for execution, the global scheduler is configured to store one or more work items in the shared cache and convey an indication to a first local scheduler of the plurality of local schedulers which causes the first local scheduler to retrieve the one or more work items from the shared cache. Subsequent to retrieving the work items, the local scheduler is configured to schedule the retrieved work items for execution by the coupled processors. Each of the plurality of local schedulers is configured to schedule work items for execution independent of scheduling performed by other local schedulers.