Multi-Rank Memory Controller Arbitration for Fewer Rank Switches
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern DDR memory systems with multiple ranks face inefficiencies due to complex timing constraints and increased write-to-read turnaround times, particularly in high-performance computing systems, leading to reduced memory bus efficiency and frequent rank switching.
Innovation Solution
A memory controller with a staging buffer and picker mechanism that optimizes rank switching by increasing the window of same-rank accesses before switching, using a multi-rank arbitration system with sub-arbiters to manage write-to-read turnarounds and improve efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple ranks are used to increase memory density and capacity, then memory capacity and density are improved, but timing constraints become more complex and memory bus efficiency decreases
Solution Approach 1:
The patent segments the memory access management into separate components: a command queue for storing pending requests, a picker for selecting requests, and sub-arbiters for managing each rank independently. This segmentation allows complex multi-rank timing constraints to be handled by simpler, dedicated units rather than a monolithic controller, reducing overall system complexity while maintaining high capacity.
Solution Approach 2:
The patent introduces an intermediary picker mechanism between the command queue and the sub-arbiters. This picker acts as a mediator that selects commands from the queue and distributes them to appropriate sub-arbiters, simplifying the interaction between multiple ranks and reducing the complexity of timing constraint management across the multi-rank system.
2Speed
If faster memory clock signals are used to increase performance, then memory speed is improved, but write-to-read turnaround times increase proportionately
Solution Approach 1:
The patent implements preliminary action by pre-loading commands into the command queue before they are needed, and by pre-positioning data in buffer memory. This allows the memory controller to prepare for upcoming read operations after write cycles, effectively reducing the perceived write-to-read turnaround time even when clock speeds increase. The staging buffer and picker mechanism enable commands to be staged in advance, smoothing out timing variations.
Solution Approach 2:
The patent ensures continuity of useful action by maintaining a command queue that continuously processes pending requests without idle gaps. The picker continuously selects and distributes commands to sub-arbiters, keeping the memory bus actively utilized. This continuous operation minimizes idle wait time between write and read operations, effectively reducing turnaround time despite higher clock speeds.
3Adaptability or versatility
If frequent rank switching is performed to access different memory ranks, then memory access flexibility is improved, but memory bus efficiency is reduced
Solution Approach 1:
The patent implements dynamics by making the rank selection process adaptive and flexible. The picker dynamically chooses which rank to access based on current command queue contents and memory state, allowing the system to adapt to varying access patterns. Sub-arbiters dynamically manage their respective ranks, switching between ready and busy states as needed. This dynamic behavior enables flexible access to any rank while optimizing bus usage by avoiding unnecessary switches.
Solution Approach 2:
The patent changes parameters in the command queue management system, specifically tracking and modifying the state of each rank (ready, busy, precharging) to optimize access patterns. By monitoring these parameters and adjusting the picker's selections based on current rank states, the system achieves flexible access while minimizing bus inefficiencies from frequent switching.
4Productivity
If queues are used to store pending memory access requests to enable out-of-order retrieval, then memory access efficiency is improved, but device complexity increases
Solution Approach 1:
The patent segments the queue management into specialized components: a command queue for storing pending requests, a picker for selecting requests, and separate sub-arbiters for each rank. This segmentation allows the out-of-order retrieval capability to be achieved through simple, dedicated structures rather than complex integrated logic, improving access efficiency while keeping individual components manageable and the overall system clear.
Solution Approach 2:
The picker serves as an intermediary between the command queue and the sub-arbiters, simplifying the complexity of coordinating out-of-order requests across multiple ranks. This intermediary layer abstracts the complexity of managing pending requests, allowing each sub-arbiter to operate independently with simpler logic while the picker handles the coordination and timing.
Data Source
AI summary
A data processing system includes a memory accessing agent for generating first memory access requests, a first memory system, and a first memory controller. The first memory system includes a first three-dimensional memory stack comprising a first plurality of stacked memory dice, wherein each memory die of the first three-dimensional memory stack includes a different logical rank of a first memory channel. The first memory controller picks second memory access requests from among the first memory access requests that access a given logical rank of the first memory channel, arbitrates between the second memory access requests, and generates memory access commands to the given logical rank in response to the arbitrating.