High-Bandwidth Flash Memory Layout for Low-Power DRAM Replacement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional NAND memory devices lack the high bandwidth and low power consumption required for applications like large language models, making them an inadequate alternative to volatile DRAM memory devices.
Innovation Solution
A high bandwidth flash (HBF) package is designed with multiple memory dies, each having arrays of non-volatile memory cells, optimized for high bandwidth and low power efficiency, featuring supply voltages below 1.5 V and a power efficiency of no greater than 1.1 pJ/bit, and integrated with a control die to enhance peripheral circuitry capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional NAND memory devices are used, then cost is reduced compared to DRAM, but bandwidth is too low and power consumption is too high
Solution Approach 1:
The patent changes key operating parameters including reducing supply voltage to below 1.5V (specifically 1.2V in some embodiments), optimizing read/disturb ratio to at least 1:10, and adjusting physical page size to no greater than 4kB. These parameter changes enable non-volatile memory to achieve HBM-level bandwidth (at least 2.7 TB/s) and power efficiency (no greater than 1.1 pJ/bit) simultaneously
Solution Approach 2:
The memory system is divided into multiple independently operable memory dies (at least two), each with its own array of non-volatile memory cells. This segmentation allows parallel operation of multiple dies to achieve high aggregate bandwidth while each die operates at low power, resolving the contradiction between system-level bandwidth and device-level power consumption
2Use of energy by moving object
If supply voltage is reduced to improve power efficiency, then power consumption decreases, but bandwidth may be limited
Solution Approach 1:
The patent transitions from single-die operation to multi-die parallel operation, adding a dimensional aspect to the system architecture. By operating multiple low-voltage dies in parallel, the system achieves high bandwidth (at least 2.7 TB/s) while each die maintains low power consumption (no greater than 1.1 pJ/bit), effectively resolving the bandwidth-power efficiency tradeoff
3Productivity
If physical page size is reduced to improve read performance, then bandwidth increases, but storage capacity per page decreases
Solution Approach 1:
The patent segments the memory array into multiple smaller physical pages (no greater than 4kB each, with some embodiments using 2kB or 1kB pages). This segmentation enables finer-grained parallel read operations across multiple pages simultaneously, achieving high bandwidth (at least 2.7 TB/s) while maintaining efficient storage utilization through the multiplication of page count and parallel operation capacity
Data Source
AI summary
A computer system is provided that includes a single processing unit and a plurality of high bandwidth flash (HBF) packages that are in electrical communication with the single processing unit. Each of the HBF packages has a plurality of memory dies with arrays of memory cells. The HBF packages have a combined bandwidth during read with the single processing unit of at least 2.7 TB/s. The dies have a power efficiency of no greater than 1.1 pJ/bit.