3D Access Filter for Cache Miss Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern computing systems face increased memory access latency and power consumption due to interconnect delays and parasitic capacitance in two-dimensional planar layout chips, especially when accessing off-chip DRAM, which is exacerbated by the growing number of processor cores and the need for efficient memory bandwidth management.
Innovation Solution
Implementing a three-dimensional (3D) integrated circuit architecture with a last-level cache (LLC) that uses 3D DRAM, PCM, memristors, or STT-MRAM, where an access filter determines whether data is stored within the on-chip LLC before sending memory requests to off-chip memory, thereby reducing unnecessary access to the LLC and optimizing memory latency and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a large on-chip last-level cache (LLC) is implemented to reduce memory access latency, then cache hit latency is reduced, but cache miss latency increases and power consumption increases due to unnecessary accesses
Solution Approach 1:
A hardware filter is introduced as an intermediary component between the processor and the large on-chip LLC. The filter receives memory requests and determines whether to forward them to the LLC based on predicted cache hit probability. This intermediary prevents unnecessary accesses to the LLC, reducing power consumption while maintaining low latency for actual cache hits.
Solution Approach 2:
The hardware filter performs preliminary analysis of memory requests before they reach the LLC. By evaluating address patterns and predicting cache hit probability in advance, the filter decides whether to forward the request to the LLC. This preliminary action avoids unnecessary LLC accesses, reducing both power consumption and miss latency.
2Use of energy by stationary object
If the memory request is sent to off-chip memory only after LLC hit/miss result is known, then power consumption is reduced, but overall memory latency increases
Solution Approach 1:
The hardware filter performs preliminary evaluation of memory requests using address pattern analysis and historical cache behavior data before the LLC access is initiated. This early prediction allows the system to forward requests to off-chip memory in parallel with LLC access when the filter predicts a miss, thereby reducing overall latency without increasing power consumption from unnecessary LLC accesses.
Solution Approach 2:
The system dynamically adjusts memory request routing based on real-time cache performance metrics and address patterns. The hardware filter continuously learns from cache hit/miss patterns and adapts its prediction algorithm, enabling flexible decision-making that optimizes both power consumption and latency based on current system state.
3Use of energy by stationary object
If a hardware filter is implemented to predict cache hits, then unnecessary LLC accesses are reduced, but device complexity increases
Solution Approach 1:
The hardware filter is segmented into multiple independent functional units: address pattern analysis units, historical cache behavior storage units, prediction logic units, and control units. Each segment performs a specific function and can be independently optimized or configured. This segmentation reduces overall circuit complexity while maintaining effective prediction capability.
Solution Approach 2:
The hardware filter utilizes self-service mechanisms by leveraging existing processor address generation logic and cached historical hit/miss data. The filter autonomously learns from past cache behavior patterns and automatically adjusts its prediction algorithm without requiring external control or complex configuration, thereby reducing overall system complexity.
Data Source
AI summary
A system and method for efficiently determining whether a requested memory location is in a large row-based memory of a computing system. A computing system includes a processing unit that generates memory requests on a first chip and a cache (LLC) on a second chip connected to the first chip. The processing unit includes an access filter that determines whether to access the cache. The cache is fabricated on top of the processing unit. The processing unit determines whether to access the access filter for a given memory request. The processing unit accesses the access filter to determine whether given data associated with a given memory request is stored within the cache. In response to determining the access filter indicates the given data is not stored within the cache, the processing unit generates a memory request to send to off-package memory.