Locally Biased Cache Replacement for Clustered GPU Architectures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) face inefficiencies due to data replication in clustered cache architectures, which reduces effective cache capacity and throughput, particularly in parallel processing environments.
Innovation Solution
Implementing a locally biased cache replacement policy to optimize cache usage and minimize data replication, thereby maintaining an effective cache footprint and improving performance in parallel processing systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a clustered cache architecture is used to support parallel processing, then processing throughput is improved, but data replication occurs which reduces effective cache capacity
Solution Approach 1:
The patent applies local quality by implementing a cache replacement policy that treats different cache partitions differently based on their access patterns. The policy identifies locally-accessed data (data accessed by the same partition that loaded it) and applies preferential retention to these partitions, while allowing non-locally-accessed data to be replaced more aggressively. This creates non-uniform cache retention behavior across partitions, optimizing for local access patterns while maintaining overall cache efficiency.
2Speed
If data is replicated across multiple cache partitions, then access speed is improved, but cache capacity is reduced
Solution Approach 1:
The patent changes the replacement policy parameter from a uniform global LRU approach to a locally-biased selective LRU approach. By modifying the replacement decision logic to consider locality information (whether data was loaded by the same partition accessing it), the system dynamically adjusts retention behavior. This parameter change enables the cache to retain replicated data longer when it provides local access benefits, while still making room for new data when replication is not beneficial.
Data Source
AI summary
Locally biased cache replacement for a clustered cache architecture is described. An example of an apparatus includes clusters of cores; a clustered cache including multiple cache partitions for the clusters of cores, each cache partition including multiple cachelines; and a computer memory including memory partitions, each of the cache partitions being associated with a respective local memory partition, wherein each cacheline of the cache partitions includes a cacheline tag, each cacheline tag including a local tag to indicate whether data stored in the cacheline is local data stored in the local memory partition or remote data stored in a remote memory partition, and a used tag to indicate whether data stored in the cacheline is recently accessed; and wherein the clustered cache includes circuitry to select cachelines for cache replacement in a cache partition based on values of the tags of the cachelines.