Two-Level Thread Scheduler for Latency Hiding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-threaded processing systems face complexity and increased power consumption due to the need to manage a large number of hardware threads to hide latency, leading to inefficient scheduling and higher resource requirements.
Innovation Solution
A two-level scheduler is implemented to partition threads into active 'strands' and pending threads based on latency characteristics, reducing the number of concurrently active threads and using a smaller register file cache to improve energy efficiency and simplify scheduling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a large number of hardware execution threads are used to hide latency, then thread processing efficiency is improved, but device complexity and power consumption increase
Solution Approach 1:
The patent segments threads into two distinct categories: strands (active threads with short latency) and pending threads (threads with long latency). This segmentation allows the scheduler to manage different thread types with different priorities and resource allocations, reducing overall system complexity while maintaining high throughput by processing strands independently from pending threads.
Solution Approach 2:
The patent implements dynamic thread state management where threads can transition between active and pending states based on their latency characteristics. The scheduler dynamically adjusts which threads are executed and which are waited on, allowing flexible resource allocation that reduces scheduler complexity while maintaining productivity.
2Productivity
If a large number of hardware execution threads are used to hide latency, then thread processing efficiency is improved, but power consumption increases
Solution Approach 1:
By segmenting threads into strands and pending threads, the patent enables selective resource allocation. Only strands require full register file cache resources, while pending threads can use reduced resources. This segmentation directly reduces power consumption by limiting active resource usage to only the necessary threads.
Solution Approach 2:
The patent applies local quality by providing different resource levels to different thread types. Strands receive full resource allocation (register file cache) while pending threads receive reduced allocation. This differentiated resource distribution reduces overall power consumption while maintaining high productivity for the critical strand execution path.
3Use of energy by moving object
If threads are separated into strands and pending threads, then power consumption is reduced, but scheduling complexity increases
Solution Approach 1:
The segmentation into strands and pending threads actually simplifies scheduling by creating clear separation of concerns. Strands are executed immediately with high priority, while pending threads are scheduled separately. This segmentation reduces scheduling complexity compared to managing all threads uniformly.
4Use of energy by moving object
If a smaller register file cache is used for strands, then power consumption is reduced, but thread processing capacity is limited
Solution Approach 1:
The segmentation allows the system to use a smaller register file cache for strands (which are the critical path threads) while maintaining high productivity. The smaller cache is sufficient for strands because they represent only a subset of threads, and pending threads can be managed with reduced resources.
Solution Approach 2:
The patent applies local quality by allocating full register file cache resources only to strands, which are the threads requiring maximum processing capacity. Pending threads receive reduced resource allocation. This differentiated approach maintains high productivity for critical threads while reducing overall power consumption.
Data Source
AI summary
One embodiment of the present invention sets forth a technique for scheduling thread execution in a multi-threaded processing environment. A two-level scheduler maintains a small set of active threads called strands to hide function unit pipeline latency and local memory access latency. The strands are a sub-set of a larger set of pending threads that is also maintained by the two-leveler scheduler. Pending threads are promoted to strands and strands are demoted to pending threads based on latency characteristics. The two-level scheduler selects strands for execution based on strand state. The longer latency of the pending threads is hidden by selecting strands for execution. When the latency for a pending thread is expired, the pending thread may be promoted to a strand and begin (or resume) execution. When a strand encounters a latency event, the strand may be demoted to a pending thread while the latency is incurred.