GPU Split Workgroup Dispatch via Load-Rated Wavefront Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU hardware limits wavefront level parallelism (WLP) and thread level parallelism (TLP) due to resource unavailability, leading to workgroup stalls and reduced performance, as it cannot dispatch a workgroup to a compute unit unless all resources are available for the entire workgroup, even if individual wavefronts could fit on different units.
Innovation Solution
Implementing a split workgroup dispatch mechanism where the control logic monitors resource contention, calculates load-ratings for compute units, and splits workgroups into wavefronts, allocating them to multiple compute units based on load-ratings to minimize resource contention, using a scoreboard to track and synchronize the execution of wavefronts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a workgroup is dispatched only when all resources are available on a single compute unit, then resource availability is ensured, but workgroup stalls increase and productivity decreases
Solution Approach 1:
The workgroup is segmented into individual wavefronts that can be dispatched independently to different compute units. Instead of requiring all resources to be available for the entire workgroup on a single unit, each wavefront can be allocated to available compute units individually, eliminating stalls caused by resource unavailability and improving overall productivity.
2Reliability
If a workgroup is dispatched when all resources are available on a single compute unit, then resource contention is minimized, but wavefront level parallelism and thread level parallelism are reduced
Solution Approach 1:
By segmenting the workgroup into dispatchable wavefronts, the system can distribute parallel work across multiple compute units simultaneously. This maintains resource contention management while enabling both wavefront level and thread level parallelism to operate at full capacity across the GPU architecture.
Solution Approach 2:
The invention introduces a new dimension of resource allocation by mapping wavefronts to multiple compute units in parallel rather than sequentially. This dimensional change in the dispatching approach enables simultaneous execution of multiple wavefronts across different compute units, thereby increasing parallelism capability while maintaining controlled resource contention.
3Productivity
If workgroups are dispatched to multiple compute units with load-ratings, then productivity and parallelism are improved, but device complexity increases
Solution Approach 1:
The control logic uses feedback mechanisms to monitor resource availability and contention across compute units, dynamically adjusting wavefront dispatch decisions based on real-time conditions. This feedback-driven approach improves productivity by optimizing resource utilization while managing complexity through automated adaptive control rather than static rigid rules.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, apparatuses, and methods for performing split-workgroup dispatch to multiple compute units are disclosed. A system includes at least a plurality of compute units, control logic, and a dispatch unit. The control logic monitors resource contention among the plurality of compute units and calculates a load-rating for each compute unit based on the resource contention. The dispatch unit receives workgroups for dispatch and determines how to dispatch workgroups to the plurality of compute units based on the calculated load-ratings. If a workgroup is unable to fit in a single compute unit based on the currently available resources of the compute units, the dispatch unit divides the workgroup into its individual wavefronts and dispatches wavefronts of the workgroup to different compute units. The dispatch unit determines how to dispatch the wavefronts to specific ones of the compute units based on the calculated load-ratings.