GPU Fine-Grained DVFS Using Workload Hints and CU Masks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional dynamic voltage and frequency scaling (DVFS) policies in GPUs are inefficient due to reactive determinations based on outdated observations, failing to adapt quickly to changing workload characteristics and environmental conditions, leading to suboptimal power usage and performance.
Innovation Solution
Implementing a system management unit (SMU) that uses software hints in the form of compute unit (CU) masks and workload characteristics to proactively determine optimal voltage and frequency settings for subsets of GPU compute units, allowing fine-grained DVFS without reliance on past observations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If DVFS managers use observation-based metrics to make DVFS policy decisions, then they can adjust voltage and frequency based on past performance, but the decisions are based on outdated observations that may not reflect current workload characteristics and the observation process consumes additional resources
Solution Approach 1:
The system performs preliminary characterization of workloads using hints provided by the workload itself before making DVFS decisions. This allows the DVFS manager to have advance knowledge of workload properties (compute-bound vs. memory-bound) and prepare appropriate voltage/frequency settings in advance, eliminating the need to wait for observations and enabling proactive optimization rather than reactive adjustment
2Reliability
If DVFS managers rely on observation-based metrics, then they can make policy decisions based on actual performance data, but the observation process consumes resources of the DVFS manager and introduces delays
Solution Approach 1:
The workload itself provides the necessary characterization information through hints embedded in the workload structure. Instead of the DVFS manager expending resources to observe and measure workload properties, the workload self-describes its characteristics (compute-bound or memory-bound) through its own structure, effectively making the workload serve its own characterization needs and eliminating the observation overhead
3Device complexity
If conventional DVFS policies are used, then they can maintain simple implementation, but they fail to adapt quickly to changing workload characteristics and environmental conditions
Solution Approach 1:
The system changes the fundamental parameter of how workload characteristics are obtained - from indirect observation to direct parameter provision through hints. The workload provides explicit parameters about its nature (compute-bound or memory-bound), allowing the DVFS manager to directly translate these into appropriate voltage/frequency settings without complex observation and analysis mechanisms
Data Source
AI summary
A processing system dynamically scales at least one of voltage and frequency at a subset of a plurality of compute units of a graphics processing unit (GPU) based on characteristics of a kernel or workload to be executed at the subset. A system management unit for the processing system receives a compute unit mask, designating the subset of a plurality of compute units of a GPU to execute the kernel or workload, and workload characteristics indicating the compute-boundedness or memory bandwidth-boundedness of the kernel or workload from a central processing unit of the processing system. The system management unit determines a dynamic voltage and frequency scaling policy for the subset of the plurality of compute units of the GPU based on the compute unit mask and the workload characteristics.