Dispatch Walk Reordering for Cache Locality Across Successive Accesses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In parallel processing units, data evicted from the cache before subsequent dispatches can lead to increased resource consumption and latency due to conventional cache access patterns not leveraging data locality, particularly in read-after-read and read-after-write scenarios.
Innovation Solution
The cache access pattern is translated from a typewriter order to a space-filling curve or alternating ascending/descending order based on data locality, using a translation module to maximize cache residency and reduce evictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional cache access patterns are used, then cache capacity is limited, but data eviction occurs before subsequent dispatches can access the data
Solution Approach 1:
The patent reorders cache access patterns in advance (before actual access) to ensure that data needed for subsequent dispatches is accessed first. By translating the access pattern from conventional order to a reordered pattern, the system proactively keeps frequently needed data in cache, preventing eviction and reducing latency for future accesses.
2Quantity of substance
If conventional cache access patterns are used, then cache size is constrained, but resource consumption increases due to repeated data loading
Solution Approach 1:
By reordering the cache access pattern before execution, the system ensures that data is accessed in an optimal sequence that maximizes cache residency. This preliminary reordering prevents the need to reload evicted data, reducing resource consumption for data loading operations while maintaining the same constrained cache size.
3Device complexity
If conventional typewriter order access pattern is used, then implementation is simple, but data locality is not leveraged leading to increased evictions
Solution Approach 1:
The system performs a preliminary translation of the access pattern from conventional typewriter order to a reordered pattern that exploits data locality. This advance reordering ensures that spatially and temporally related data is accessed together, maximizing cache hit rates and reliability without requiring complex runtime decisions during actual data access.
Data Source
AI summary
A processing system is configured to translate a first cache access pattern of a dispatch of work items to a cache access pattern that facilitates consumption of data stored at a cache of a parallel processing unit by a subsequent access before the data is evicted to a more remote level of the memory hierarchy. For consecutive cache accesses having read-after-read data locality, in some embodiments the processing system translates the first cache access pattern to a space-filling curve. In some embodiments, for consecutive accesses having read-after-write data locality, the processing system translates a first typewriter cache access pattern that proceeds in ascending order for a first access to a reverse typewriter cache access pattern that proceeds in descending order for a subsequent cache access. By translating the cache access pattern based on data locality, the processing system increases the hit rate of the cache.