Lightweight Cache Coherence for Limited Data Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional cache coherence mechanisms in modern computing systems with multiple processors and accelerators, such as GPUs and PIMs, face scalability issues and high overhead costs due to the need for extensive probe and response messages and directory structures, which do not efficiently manage memory bandwidth and locality.
Innovation Solution
A lightweight cache coherence scheme that allocates cache lines based on local and remote states, allowing processors to cache data locally and remotely while maintaining coherence through write-through policies and remote state encoding, eliminating the need for explicit tracking and coherence messages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional broadcast cache coherence mechanism is used, then any memory access is visible to all coherent processors, but it does not scale to large numbers of processors or bandwidth-intensive devices due to high bandwidth requirements
Solution Approach 1:
The patent extracts and eliminates the broadcast mechanism from the cache coherence system. Instead of broadcasting all memory accesses to all processors, the invention uses a lightweight protocol where only the requesting processor and the target memory location are involved, removing the unnecessary participation of other processors and thus reducing bandwidth consumption significantly
Solution Approach 2:
The patent introduces a memory address space organization as an intermediary that implicitly defines coherence domains. By organizing memory addresses into regions associated with specific processors or processor groups, the system automatically determines which processors need to participate in coherence operations without requiring explicit broadcast messages
2Productivity
If directory-based cache coherence mechanism is used, then scalability is improved, but storage overhead and design complexity increase to track which processor is caching what data
Solution Approach 1:
The patent implements a self-service approach where each processor independently tracks its own cached data through simple metadata in its cache structure. Instead of a centralized directory that requires complex probe and response protocols, each processor maintains its own coherence information, eliminating the need for complex directory structures and reducing overall system complexity
Solution Approach 2:
The patent removes the centralized directory structure entirely from the cache coherence mechanism. By distributing the coherence tracking functionality to individual processors and using the memory address space organization to implicitly define coherence relationships, the invention eliminates the complex directory infrastructure while maintaining scalability
3Device complexity
If shared data is made uncacheable to simplify coherence, then coherence management is simplified, but performance degrades due to inefficient memory bandwidth use and long load latencies
Solution Approach 1:
The patent changes the state parameters of cached data to include coherence state information directly in the cache line metadata. By encoding whether data is local or remote and its coherence state, the system enables efficient caching of shared data while maintaining correctness through simple state transitions, avoiding the need to make shared data uncacheable
4Device complexity
If cache flushes are used for coherence in systems with relaxed memory models, then low-cost coherence is achieved, but performance degrades for fine-grain data sharing because the entire cache contents are evicted
Solution Approach 1:
The patent applies local quality by differentiating between local and remote data in the cache and applying different coherence management strategies to each. Local data benefits from full caching with simple invalidation, while remote data uses write-through policies. This selective approach avoids the performance penalty of flushing entire caches while maintaining coherence for fine-grain shared data
Data Source
AI summary
A data processing system includes a plurality of processors, local memories associated with a corresponding processor, and at least one inter-processor link. In response to a first processor performing a load or store operation on an address of a corresponding local memory that is not currently in the local cache, a local cache allocates a first cache line and encodes a local state with the first cache line. In response to a load operation from an address of a remote memory that is not currently in the local cache, the local cache allocates a second cache line and encodes a remote state with the second cache line. The first processor performs subsequent loads and stores on the first cache line in the local cache in response to the local state, and subsequent loads from the second cache line in the local cache in response to the remote state.