GPU Variable Wavefront Execution for SIMD Scheduling Mismatch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of scheduling instructions for wavefronts in graphics processing units (GPUs) is exacerbated when the number of work-items does not match the number of execution units in SIMD cores, leading to inefficiencies in processing.
Innovation Solution
Implementing a variable wavefront size execution mechanism in GPUs, allowing for two operating modes: one where instructions are executed on multiple portions of a wavefront before proceeding to the next, and another where instructions are executed on a portion of the wavefront before moving to the next, with register sharing based on cache miss rates and software indications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of work-items in a wavefront is increased to improve parallel processing throughput, then productivity is improved, but device complexity increases due to challenging instruction scheduling when work-items do not match execution units
Solution Approach 1:
The wavefront is segmented into multiple portions, each containing a subset of work-items. The scheduler independently manages each portion, assigning work-items from different portions to different execution units. This segmentation allows the system to handle wavefronts of any size without requiring the total number of work-items to match the number of execution units, thereby maintaining high throughput while simplifying scheduling complexity.
2Adaptability or versatility
If fixed wavefront size is used to simplify hardware design, then device complexity is reduced, but adaptability deteriorates when handling varying workloads
Solution Approach 1:
The system employs dynamic wavefront sizing where the wavefront size can be adjusted based on workload requirements. The scheduler dynamically determines how to divide work-items into portions and assigns them to execution units. This dynamic approach allows the GPU to adapt to varying workload sizes and types without requiring fixed hardware configuration, achieving high adaptability while managing complexity through software-based scheduling flexibility.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system includes a processor configured to operate in at least a first mode and a second mode. In the first mode the first processor operates to execute an instruction for an entire wavefront before executing a next instruction for the entire wavefront. In the second mode the processor operates to execute a set instructions for a portion of a wavefront before executing the set instructions for another portion of the same wavefront. The system further includes a memory coupled to the processor. The memory is configured to store a shader program for execution by the processor, wherein the shader program includes at least one indication associated with one of the first mode or the second mode. The processor is further to implement one of the first mode or the second mode while executing the shader program responsive to the at least one indication present in the first shader program.