GPU Scratch Cache Scheduling for Wavefront Contention Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cache contention and capacity evictions in memory hierarchies lead to increased latencies and decreased performance in highly parallel data applications due to the increase in local data values used by iterative function calls, which exceed the capacity of local caches.
Innovation Solution
Implementing a control circuit to monitor and manage cache usage by adjusting the dispatch and issue rates of wavefronts based on reservation data sizes, priority levels, and available cache space, pausing or reducing the execution of wavefronts when cache space is insufficient, and using priority-based scheduling to maintain efficient cache utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the amount of local data values used by iterative function calls increases, then the processing capability and functionality of the application is improved, but the cache capacity is exceeded causing capacity evictions and increased cache contention
Solution Approach 1:
The patent implements dynamic reservation of cache space for wavefronts based on their priority levels and data requirements. The system dynamically adjusts cache allocation by reserving specific portions of the cache for high-priority wavefronts while allowing lower-priority wavefronts to access remaining space, thereby adapting cache usage to varying processing needs without fixed capacity constraints
Solution Approach 2:
The system changes the parameter of cache allocation from a fixed static model to a dynamic model based on wavefront priority levels. Different priority levels receive different reservation quotas, and the system monitors actual cache usage to adjust allocations, transforming the cache management approach to match varying data value requirements of iterative functions
2Reliability
If cache capacity is increased to store all local data values, then cache contention is reduced, but the memory hierarchy performance and throughput are decreased due to slower non-volatile memory access
Solution Approach 1:
The patent segments the cache into multiple reserved portions, each allocated to specific priority levels of wavefronts. This segmentation allows the system to manage cache space efficiently by dividing it into manageable sections that can be dynamically assigned based on wavefront requirements, avoiding the need for a single large cache that would degrade performance
3Speed
If wavefronts are dispatched at high rates to maximize throughput, then processing speed is improved, but cache space becomes insufficient causing evictions and reduced performance
Solution Approach 1:
The system performs preliminary reservation of cache space for wavefronts before they are fully dispatched for execution. By calculating and reserving the required cache portions based on wavefront priority levels and data requirements in advance, the system ensures that sufficient cache space is available when wavefronts need to access local data values, preventing evictions and maintaining high processing speeds
4Productivity
If priority-based scheduling is implemented to manage cache usage, then cache utilization efficiency is improved, but the system complexity increases
Solution Approach 1:
The patent applies local quality by assigning different cache reservation quotas to different priority levels of wavefronts. Each priority level receives a customized portion of the cache based on its specific requirements, allowing fine-grained control of cache allocation that improves utilization efficiency while managing complexity through structured priority-based categories
Data Source
AI summary
An apparatus and method for efficiently scheduling wavefronts for execution on an integrated circuit. In various implementations, a computing system includes a parallel data processing circuit with multiple, replicated compute circuits. Each compute circuit executes one or more wavefronts. Each compute circuit includes a cache configured to store temporary data that cannot fit in the vector general-purpose register file of the compute circuit. Each wavefront requests a corresponding amount of storage space in the cache for storing the temporary data. When the available data storage space in the cache is less than a data size requested by a wavefront waiting to be dispatched, a control circuit of the compute circuit reduces a dispatch rate of wavefronts. The control circuit also reduces an issue rate of instructions of one or more dispatched wavefronts to assigned execution circuits of the compute circuit.