Configurable Cache Sub-Domains for Multi-Core Coherency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-core, multi-threaded processors face challenges in cache coherency management due to concurrent thread operations across multiple cores and dies, leading to significant latency and performance issues, especially in graph workloads with poor cache-line utilization and low temporal locality.
Innovation Solution
A multi-core, multi-threaded processing system with configurable cache sub-domains and a MOESI-F cache coherency protocol, allowing for selective caching and dynamic reconfiguration of cache coherency domains to optimize performance across various programming models, using shadow tags for efficient state tracking and reduced latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple threads operate concurrently across multiple cores and dies, then processing capability is improved, but cache coherency management complexity increases
Solution Approach 1:
The patent divides the cache coherency management into segments by introducing configurable sub-domains that partition the cache hierarchy into smaller, manageable regions. Each sub-domain can be independently configured and managed, reducing the complexity of tracking coherence across the entire multi-core, multi-threaded system. This segmentation allows the system to maintain coherence within smaller groups while scaling across multiple cores and dies.
Solution Approach 2:
The patent implements dynamic reconfiguration of cache coherency domains, allowing the sub-domain boundaries and memberships to be changed at runtime. This dynamic capability enables the system to adapt coherence domains to different workloads and programming models, optimizing performance while managing complexity through flexible, rather than fixed, organization.
2Productivity
If direct memory operations are used to move data, then data transfer is achieved, but latency increases significantly
Solution Approach 1:
The patent employs preliminary actions through proactive cache preloading and prediction mechanisms within sub-domains. By anticipating data access patterns and pre-loading data into appropriate cache levels before actual access occurs, the system reduces the latency of direct memory operations. This is particularly effective in graph workloads where access patterns can be predicted based on graph structure and query patterns.
Solution Approach 2:
The patent introduces intermediary cache levels and sub-domain boundaries that act as mediators between memory requests and actual data storage. Rather than direct memory operations, data flows through multiple intermediate cache tiers within sub-domains, which can serve data from local caches rather than requiring direct memory access, thereby reducing latency for frequently accessed data within the same sub-domain.
3Reliability
If cache coherency is maintained across all threads, then data consistency is ensured, but performance overhead increases
Solution Approach 1:
The patent segments the coherence domain into sub-domains, maintaining strict coherence within each sub-domain while allowing more flexible memory access patterns across sub-domain boundaries. This segmentation reduces the scope of coherence tracking and invalidation operations, lowering performance overhead while ensuring data consistency within the segmented regions where most concurrent operations occur.
Solution Approach 2:
The patent applies different coherence characteristics to different parts of the memory hierarchy and sub-domains. Local coherence is maintained within sub-domains where data consistency is critical, while cross-sub-domain accesses can use relaxed memory ordering. This local quality approach ensures data consistency where needed without imposing coherence overhead across the entire system.
4Adaptability or versatility
If configurable sub-domains are introduced, then cache organization flexibility is improved, but system complexity increases
Solution Approach 1:
The patent implements dynamic configuration of sub-domains where boundaries, sizes, and memberships can be adjusted at runtime based on workload characteristics. This dynamic adaptability allows the same hardware to be optimized for different programming models and workloads without requiring complex static reconfiguration, managing system complexity through software-controllable rather than hardware-hardcoded flexibility.
Solution Approach 2:
The patent designs a universal sub-domain configuration mechanism that can serve multiple functions: supporting different programming models (shared memory, distributed memory), adapting to various workload patterns (graph analytics, general-purpose computing), and providing both performance optimization and flexibility. This multi-functionality reduces overall system complexity by using a single configurable framework rather than multiple specialized mechanisms.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system with configurable cache sub-domains and cross-die memory coherency includes R racks, each rack housing N nodes, each node incorporating D dies (200), each die containing C cores (204, 206, 208 ...) and a die shadow tag (220), each core including P pipelines and a core shadow tag (205, 207, 209 ...), each pipeline associated with a data cache (D$) and data cache tags (D$ Tags) and being either non-coherent or coherent and associated with one of X coherency domains. Each pipeline, when needing to read a cache line, issues a read request to its associated data cache, then, if need be, issues a read request to its associated core-level cache, then, if need be, issues a read request to its associated die-level cache, then, if need be, issues a no-cache remote read request to a target die being mapped to hold the cache line.