Shared Data Prefetch Hints for Multi-Core Cache Coherence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processors optimize for single-threaded applications, leading to suboptimal handling of multi-threaded data access, particularly for read-shared data, resulting in increased latency, mesh traffic, and cache pressure due to inefficient cache coherence protocols.
Innovation Solution
Implementing a shared prefetch instruction (PREFETCHS) that hints to hardware that data will be shared by multiple threads, allowing for optimized placement in shared caches, reducing coherence transactions and improving cache hit rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is installed into the requesting core's L2 cache in exclusive state for load misses, then cache hit rate for that core is improved, but mesh traffic and cache pressure increase due to coherence protocols
Solution Approach 1:
The patent changes the cache state parameter from exclusive to shared when installing data into the LLC cache. This allows multiple cores to access the same data simultaneously without requiring coherence protocol interventions, thereby reducing mesh traffic while maintaining cache hit rates for read-shared data access patterns
2Productivity
If cache hierarchy is optimized for single-threaded applications, then single-threaded performance is improved, but multi-threaded performance deteriorates due to suboptimal data handling
Solution Approach 1:
The patent introduces dynamic behavior in the cache hierarchy by allowing the cache controller to adapt its installation policy based on the access pattern. For read-shared data, data is installed in shared state in the LLC; for other cases, existing policies apply. This dynamic adaptation enables the system to maintain single-threaded optimization while gaining multi-threaded performance benefits
3Device complexity
If traditional load instructions are used without hints, then instruction simplicity is maintained, but cache coherence transactions increase leading to higher latency
Solution Approach 1:
The patent applies preliminary action by having the software explicitly indicate read-shared access intent before the data is actually accessed. This hint allows the hardware to pre-configure the cache installation policy, avoiding subsequent coherence transactions and reducing latency when the data is needed
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processor for prefetching data with hints comprises: a plurality of cores to process instructions; a first core of the plurality of cores comprising: decoder circuitry to decode instructions indicating memory operations including load operations of a first type with shared data hints and load operations of a second type without shared data hints; execution circuitry to execute the instructions to perform the memory operations; data prefetch circuitry to store tracking data in a tracking data structure responsive to the memory operations, a portion of the tracking data associated with the first type of load operations; and the data prefetch circuitry to detect memory access patterns using the tracking data, the data prefetch circuitry to responsively issue one or more prefetch operations using shared data hints based, at least in part, on the portion of the tracking data associated with the first type of load operations.