Cache Directory Look-up Re-use for Speculative Memory Conflict Checking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large supercomputer systems, the directory lookup process in cache memories becomes energy inefficient due to the need for extensive conflict checking across thousands of caches, especially during sequential or nearly sequential memory access requests, leading to significant power consumption.
Innovation Solution
Implementing a method to queue and store results of memory access requests in a cache memory system, allowing subsequent requests to use stored results instead of rechecking the cache directory, and utilizing a sub-cache to retain address or data related to prior requests, thereby reducing directory SRAM accesses and enhancing throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If directory lookup is performed for every memory access request in large supercomputer systems, then cache coherence and correctness are maintained, but power consumption increases significantly
Solution Approach 1:
The patent performs directory lookup in advance for speculative memory requests before actual data access is needed. By pre-checking the directory and caching the results, the system determines cache validity ahead of time, avoiding the need to perform expensive directory lookups for every subsequent access while maintaining cache coherence.
Solution Approach 2:
The patent creates copies of directory lookup results and stores them in a separate caching structure. Instead of repeatedly accessing the main cache directory for each memory request, the system copies relevant directory information into a faster, smaller storage structure that can be queried efficiently, reducing power consumption while maintaining accuracy.
2Reliability
If extensive conflict checking is performed across thousands of caches, then cache coherence is ensured, but processing time increases
Solution Approach 1:
The patent segments the conflict checking process by dividing it into speculative pre-checking and actual data access phases. The directory lookup is separated from the data access operation, allowing conflict detection to occur independently in advance. This segmentation enables parallel processing and reduces the critical path delay for actual data access.
Solution Approach 2:
The patent performs conflict checking in advance during the speculative execution phase, before actual data access occurs. By pre-determining which cache lines are valid and which would cause conflicts, the system eliminates the need for time-consuming conflict checking during actual data access operations, significantly reducing processing time while maintaining coherence.
3Measurement precision
If sequential memory access requests are handled with full directory lookup, then accuracy is maintained, but energy efficiency deteriorates
Solution Approach 1:
The patent copies directory lookup results into a dedicated caching structure that is optimized for sequential access patterns. For sequential memory requests, the system retrieves pre-cached directory information rather than performing full directory lookups, maintaining access accuracy while dramatically improving energy efficiency for sequential workloads.
Solution Approach 2:
The patent implements periodic directory lookup caching where results are cached and reused for a period, especially beneficial for sequential access patterns where the same directory entries are accessed repeatedly. This periodic caching strategy maintains accuracy by invalidating cached results when necessary while reducing energy consumption during the caching period.
Data Source
AI summary
In a cache memory, energy and other efficiencies can be realized by saving a result of a cache directory lookup for sequential accesses to a same memory address. Where the cache is a point of coherence for speculative execution in a multiprocessor system, with directory lookups serving as the point of conflict detection, such saving becomes particularly advantageous.