Fine-Grained DRAM Dual-Mode I/O for Lower Regular Access Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Fine-grained DRAM architectures experience increased access latency for regular workloads due to narrower interfaces and proportional latency increases with the number of grains per bank, which is detrimental for latency-sensitive operations.
Innovation Solution
Implement a dual-mode I/O circuit in each grain of the DRAM bank that operates in a first mode for individual commands of a first data width and a second mode for parallel execution of commands across multiple grains, using multi-cast column-address-strobe (CAS) commands to overcome latency constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the DRAM die is partitioned into smaller independent grains with dedicated DQ pins, then memory-level parallelism and bandwidth are increased, but access latency increases proportionately to the number of grains per bank
Solution Approach 1:
The patent segments the memory system into multiple grains with dedicated DQ pins for parallel access. Each grain can be accessed independently, enabling simultaneous data transfers across multiple grains, thereby increasing overall memory bandwidth while managing latency through parallelism.
Solution Approach 2:
The patent merges multiple grain accesses into a single transaction by allowing the memory interface to initiate accesses to multiple grains simultaneously. This combining of parallel access operations maintains high bandwidth utilization while reducing the effective latency for aggregate data transfers.
2Temperature
If each grain has a narrower interface to the DRAM channel, then thermal design power limits are respected, but access latency increases for regular workloads
Solution Approach 1:
The patent divides the memory interface into multiple independent grain interfaces, each with its own DQ pins. This segmentation allows parallel access to multiple grains through separate narrow interfaces, distributing the thermal load across multiple channels while maintaining high aggregate bandwidth and reducing effective latency through concurrent operations.
Solution Approach 2:
The patent introduces parallelism across the grain dimension, transforming the single-channel access model into a multi-grain parallel access model. By accessing multiple grains simultaneously through separate DQ pin sets, the system achieves high bandwidth without concentrating thermal load on a single interface, thus respecting TDP limits while maintaining low latency.
3Productivity
If more grains per bank are implemented, then memory-level parallelism is increased, but the access latency of a DRAM atom via column access command increases
Solution Approach 1:
The patent segments the bank into multiple grains, each with dedicated DQ pins and independent data paths. This segmentation enables parallel column access commands to be issued to multiple grains simultaneously, increasing memory-level parallelism while the narrow interface per grain is compensated by the parallel execution of multiple grain accesses.
Solution Approach 2:
The patent ensures continuous useful action by allowing the memory interface to pipeline column access commands across multiple grains. While one grain is being accessed, the interface can initiate accesses to other grains, maintaining continuous data flow and hiding the latency of individual grain accesses through overlapping parallel operations.
Data Source
AI summary
A fine-grained dynamic random-access memory (DRAM) includes a first memory bank, a second memory bank, and a dual mode I/O circuit. The first memory bank includes a memory array divided into a plurality of grains, each grain including a row buffer and input/output (I/O) circuitry. The dual-mode I/O circuit is coupled to the I/O circuitry of each grain in the first memory bank, and operates in a first mode in which commands having a first data width are routed to and fulfilled individually at each grain, and a second mode in which commands having a second data width different from the first data width are fulfilled by at least two of the grains in parallel.