Hierarchical Thread Scheduling for Hardware Resource Conservation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-threaded processors face inefficiencies when scheduling large numbers of thread groups due to the need for proportional increases in hardware resources and scheduler size, leading to increased costs and resource wastage.
Innovation Solution
Implementing a hierarchical scheduling system that allocates limited hardware resources only to thread groups ready for execution, using a macro-scheduling policy to categorize thread groups into pre-staging and staging areas, and a micro-scheduling policy to select and allocate resources efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of thread groups is increased to support more parallel execution, then processing capability is improved, but hardware resources (RAM space, transistor budget) must be scaled proportionally, increasing cost and complexity
Solution Approach 1:
The patent divides the scheduler into hierarchical levels (e.g., coarse-grain scheduler and fine-grain schedulers), where the coarse-grain scheduler manages a large number of thread groups at a higher level, and fine-grain schedulers manage smaller subsets at lower levels. This segmentation allows the system to support many thread groups without requiring a single large, resource-intensive scheduler, thus improving processing capability while controlling hardware resource requirements.
Solution Approach 2:
The patent introduces a hierarchical dimension to the scheduling architecture, organizing schedulers in multiple levels rather than a single flat layer. This dimensional change allows the system to manage a large number of thread groups by distributing scheduling responsibilities across hierarchical levels, reducing the resource burden at any single level while maintaining overall processing capability.
2Adaptability or versatility
If the scheduler size is increased to arbitrate between more thread groups, then scheduling capability is improved, but transistor budget and cost increase proportionally
Solution Approach 1:
The scheduler is segmented into hierarchical levels with different responsibilities. The coarse-grain scheduler performs high-level arbitration among many thread groups using fewer resources, while fine-grain schedulers handle detailed scheduling for smaller subsets. This segmentation enables the system to maintain scheduling capability for a large number of thread groups without requiring a single large scheduler that would consume excessive transistor budget.
Solution Approach 2:
Instead of having every scheduler manage all thread groups equally, the hierarchical structure allows partial specialization where each scheduler level focuses on a specific subset or aspect of scheduling. This partial action approach reduces the complexity and transistor requirements at each level while collectively providing comprehensive scheduling capability across all thread groups.
3Reliability
If hardware resources are allocated to all thread groups, then resource availability is improved, but resource wastage increases when threads are not ready for execution
Solution Approach 1:
The hierarchical scheduling structure enables preliminary preparation of thread groups at higher levels before detailed resource allocation at lower levels. Thread groups can be pre-organized and prepared in the hierarchical structure without committing full hardware resources until they are actually ready for execution, ensuring resource availability when needed while avoiding premature resource allocation that would cause wastage.
Solution Approach 2:
The hierarchical scheduler dynamically allocates hardware resources based on the readiness state of thread groups. Resources are allocated flexibly at different hierarchical levels according to actual execution needs, allowing the system to maintain resource availability for ready threads while releasing or avoiding allocation for threads that are not yet ready, thereby reducing resource wastage while maintaining reliability.
Data Source
AI summary
One embodiment of the present invention is a computer-implemented method for scheduling a thread group for execution on a processing engine that includes identifying a first thread group included in a first set of thread groups that can be issued for execution on the processing engine, where the first thread group includes one or more threads. The method also includes transferring the first thread group from the first set of thread groups to a second set of thread groups, allocating hardware resources to the first thread group, and selecting the first thread group from the second set of thread groups for execution on the processing engine. One advantage of the disclosed technique is that a scheduler only allocates limited hardware resources to thread groups that are, in fact, ready to be issued for execution, thereby conserving those resources in a manner that is generally more efficient than conventional techniques.