Stream Multiprocessor Warp Scheduling for GPU Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing GPUs face challenges in efficiently scheduling stream processors to execute thread blocks of different kernels, leading to suboptimal hardware utilization and computation efficiency due to resource competition among warps.
Innovation Solution
A stream multiprocessor with a local dispatcher that includes a warp state table, warp resource detection unit, and warp launching unit, which selectively dispatches warps to idle stream processors based on available hardware resources, allowing for parallel execution of warps from different thread blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If stream multiprocessors simultaneously execute thread blocks of different kernel codes, then computation efficiency is enhanced, but hardware resource competition among warps makes scheduling difficult and reduces hardware utilization rate
Solution Approach 1:
The patent segments the scheduling process into two distinct levels: a global thread block dispatcher that assigns thread blocks to stream multiprocessors, and a local warp dispatcher within each stream multiprocessor that manages warps. This segmentation allows each level to focus on specific scheduling tasks, reducing overall scheduling complexity while maintaining high computation efficiency through parallel execution of multiple thread blocks.
Solution Approach 2:
The patent implements preliminary action by having the global thread block dispatcher pre-assign thread blocks to stream multiprocessors before execution begins. This preliminary assignment prepares the system in advance, allowing local warp dispatchers to subsequently manage warp-level scheduling without the overhead of making scheduling decisions during execution, thus reducing scheduling complexity while enabling efficient parallel computation.
2Productivity
If warps of different kernels compete for hardware resources, then hardware utilization rate decreases, but allowing simultaneous execution of different kernels is necessary to enhance computation efficiency
Solution Approach 1:
The patent implements feedback mechanisms where the local warp dispatcher continuously monitors hardware resource availability and adjusts warp scheduling decisions accordingly. This feedback loop allows the system to dynamically respond to resource competition, selecting warps that can execute with available resources, thereby maintaining high hardware utilization rate while enabling simultaneous execution of multiple thread blocks from different kernels.
Solution Approach 2:
The patent introduces dynamic scheduling at the warp level within each stream multiprocessor. Instead of static allocation, the local warp dispatcher dynamically selects which warps to execute based on current hardware resource availability. This dynamic approach allows the system to adapt to changing resource conditions, ensuring high hardware utilization while maintaining the ability to execute multiple thread blocks simultaneously for improved computation efficiency.
Data Source
AI summary
A stream multiprocessor, a GPU, and related methods are provided. The stream multiprocessor executes thread blocks. Each thread block includes warps. The stream multiprocessor includes stream processors and a local dispatcher. Each stream processor executes one or more warps. The local dispatcher includes a warp state table, a warp resource detection unit and a warp launching unit. The warp state table records dispatching states and processing states of warps of the thread blocks. The warp resource detection unit selects all the first warps of a first thread block and at least one second warp of a second thread block according to hardware resources available to the stream multiprocessor and hardware resources required for thread blocks. The warp launching unit dispatches the first warps to idle stream processors and at least one second warp to at least one idle stream processor.