Range-Based Cache Flushing for GPU Memory Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cache flush operations in GPUs are becoming increasingly costly due to larger shared graphics cache capacity, with range-based flushing being slow and inefficient, particularly due to high latency and cache miss rates, especially at synchronization points and kernel boundaries.
Innovation Solution
Optimizing cache flushes by triggering a single cache walk for contiguous physical address ranges, reducing the number of address translations and cache walks, and leveraging contiguous virtual and physical address mappings to minimize flushing overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If range-based flushing is implemented to avoid discarding useful data, then cache hit rates for subsequent kernels are improved, but the latency of the flush operation increases significantly
Solution Approach 1:
The patent segments the address translation process by identifying contiguous physical address ranges that can be flushed together. Instead of translating and flushing every page individually, the system groups contiguous pages into ranges and performs a single address translation for each range, significantly reducing the number of translation operations while maintaining cache coherency.
Solution Approach 2:
The patent merges multiple individual page flush operations into a single range-based flush operation. By combining contiguous pages into a unified address range, the system performs one address translation and one cache walk for the entire range instead of multiple separate operations, reducing overhead and latency.
2Quantity of substance
If shared cache capacity is increased to support larger GPU sizes, then processing capability is improved, but the overhead of flushing shared caches at synchronization points increases
Solution Approach 1:
The patent changes the parameter of address translation granularity from page-level to range-level. By translating addresses at the range level rather than individual page level, the system reduces the number of translation operations required when flushing larger caches, making the flush overhead scale more efficiently with increased cache capacity.
3Reliability
If address translation is performed for every page in the address range, then cache coherency is maintained, but the flushing operation becomes slow for large ranges
Solution Approach 1:
The patent segments the address translation process by identifying contiguous physical address ranges that can be flushed together. Instead of translating and flushing every page individually, the system groups contiguous pages into ranges and performs a single address translation for each range, significantly reducing the number of translation operations while maintaining cache coherency.
Solution Approach 2:
The patent performs address translation only for the boundaries of contiguous address ranges rather than for every page within the ranges. This partial action approach translates only the necessary addresses (range start and end) while assuming contiguity for intermediate pages, dramatically reducing translation overhead while maintaining sufficient coherency for the flush operation.
Data Source
AI summary
Systems and methods of cache flushing include receiving, from a software application, a first cache flush request to perform a range-based cache flush of a contiguous virtual address range within a virtual memory that maps to a physical memory. A single cache walk is triggered via a second cache flush request to a cache. The single cache walk performs the range-based cache flush for the contiguous physical address range from a beginning address of the contiguous physical address range to an ending address of the contiguous physical address range in response to the first cache flush request.