3D NAND Memory Stack With Parallel TSVs for High-Bandwidth Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional non-volatile memory devices lack the high bandwidth and low power consumption required for applications like large language models, with NAND memory devices being too slow and DRAM devices being too expensive.
Innovation Solution
A non-volatile memory apparatus with a stack of memory dies and parallel through silicon vias (TSVs) connected to a memory controller, allowing for high bandwidth reads across multiple memory dies, and including error correction and refresh mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional NAND memory devices are used, then cost is reduced compared to DRAM, but bandwidth is too low and power consumption is too high
Solution Approach 1:
The memory system is divided into multiple independent memory dies stacked vertically, with each die processed separately. This segmentation allows parallel operation of multiple dies through separate TSV paths, dramatically increasing bandwidth while maintaining cost-effectiveness of NAND technology
Solution Approach 2:
The patent transitions from planar 2D memory architecture to 3D stacked architecture with vertical TSV interconnects. This dimensional change enables multiple memory layers to be accessed simultaneously through parallel vertical paths, achieving high bandwidth without increasing footprint area
2Productivity
If high bandwidth volatile memory devices (HBM) are used, then bandwidth requirement is met, but cost is too high compared to non-volatile memory
Solution Approach 1:
The patent employs conventional NAND flash memory dies which are inexpensive to manufacture at scale. By stacking multiple low-cost dies with simple TSV interconnects rather than using expensive HBM technology, the system achieves high bandwidth at a fraction of the cost of volatile memory solutions
3Speed
If conventional NAND memory devices are used, then cost is reduced, but read speed is too slow for high bandwidth applications
Solution Approach 1:
The memory array is segmented across multiple independent dies, each with its own read path through separate TSVs. This allows simultaneous read operations on multiple dies, achieving high aggregate bandwidth even if individual die read speeds remain at conventional NAND levels
Solution Approach 2:
The patent implements preliminary data transfer from memory dies to volatile cache buffers before final data delivery to the host. This preprocessing and buffering approach overlaps data preparation with data transfer, effectively hiding latency and improving perceived read speed
Data Source
AI summary
A non-volatile memory apparatus includes a stack of memory dies with multiple layers. Each layer has multiple memory die, and the stack includes separate parallel through silicon vias (TSVs) for each memory die. The non-volatile memory apparatus also includes a memory controller in electrical communication with the separate parallel TSVs for each memory die and configured to perform a high bandwidth read process for data stored in the stack across all or multiple of the memory dies.