Dynamic Last Level Cache Architecture for DRAM Bandwidth Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current last level cache (LLC) architectures fail to optimize memory efficiency, leading to inefficiencies in DRAM bandwidth usage due to read-to-write turnarounds and row buffer locality, which limits overall system bandwidth, especially in throughput-oriented workloads.
Innovation Solution
A dynamic memory-efficient LLC architecture that detects bandwidth demand and dynamically switches between read and write phases, utilizing a large buffer and adjusting replacement policies to prioritize memory efficiency over hit rate, thereby improving memory bank-level-parallelism and reducing access latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is cached in LLC to improve accessibility, then access speed improves, but memory efficiency deteriorates due to read-to-write turnarounds and row buffer locality
Solution Approach 1:
The patent implements dynamic phase switching between read phase and write phase in the LLC controller. The system monitors bandwidth demand and dynamically transitions between phases: in read phase, the LLC prioritizes fast data retrieval for throughput workloads; in write phase, it performs background writes of dirty lines to memory. This dynamic adaptation resolves the contradiction by optimizing for speed during read-intensive operations and for memory efficiency during write operations, preventing read-to-write turnaround penalties.
Solution Approach 2:
The patent changes operational parameters by switching replacement policies based on phase. During read phase, it uses policies optimized for hit rate; during write phase, it uses policies optimized for memory efficiency and bank-level parallelism. This parameter change allows the system to achieve both fast access speeds (through high hit rate in read phase) and good memory efficiency (through optimized write scheduling in write phase), resolving the fundamental trade-off.
2Productivity
If LLC hit rate is prioritized to improve performance, then accessibility improves, but system bandwidth deteriorates due to reduced memory efficiency
Solution Approach 1:
The patent performs preliminary writes of dirty lines from LLC to memory during the write phase, before they are absolutely needed. By proactively writing dirty lines during background write phase, the system prepares memory in advance, improving subsequent read operations and overall system bandwidth without sacrificing LLC hit rate during throughput workloads. This preliminary action resolves the contradiction by anticipating memory needs.
Solution Approach 2:
The patent implements periodic switching between read phase and write phase. During read phase, the system maximizes LLC hit rate for fast access; during write phase, it performs batch writes to improve memory efficiency and bandwidth utilization. This periodic alternation allows the system to achieve both high productivity (through maintained hit rate) and high system bandwidth (through efficient periodic writes), resolving the trade-off between the two parameters.
3Device complexity
If conventional LLC architecture is used to maintain simplicity, then device complexity remains low, but memory efficiency deteriorates due to lack of dynamic optimization
Solution Approach 1:
The patent segments the LLC operation into distinct read phase and write phase, with a cache controller that independently manages each phase. This segmentation allows simple, dedicated logic for each phase while achieving complex dynamic optimization overall. The read phase handles fast access with simple lookup logic, while the write phase handles background synchronization with simple write-back logic. This segmentation resolves the contradiction by achieving high memory efficiency through phase separation without requiring complex unified control logic.
Data Source
AI summary
A memory-efficient last level cache (LLC) architecture is described. A processor implementing a LLC architecture may include a processor core, a last level cache (LLC) operatively coupled to the processor core, and a cache controller operatively coupled to the LLC. The cache controller is to monitor a bandwidth demand of a channel between the processor core and a dynamic random-access memory (DRAM) device associated with the LLC. The cache controller is further to perform a first defined number of consecutive reads from the DRAM device when the bandwidth demand exceeds a first threshold value and perform a first defined number of consecutive writes of modified lines from the LLC to the DRAM device when the bandwidth demand exceeds the first threshold value.