Hybrid High Bandwidth Memory with DRAM and Non-Volatile Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory systems in AI applications face limitations in bandwidth and power consumption due to the separation of dynamic random-access memory (DRAM) and non-volatile memory, which restricts compute performance and energy efficiency.
Innovation Solution
Integration of dynamic random-access memory (DRAM) and non-volatile memory devices, such as phase-change memory (PCM) and resistive random-access memory (ReRAM), on the same die, along with logic devices, to form a hybrid high bandwidth memory (HBM) that includes a protective spacer layer for electrical insulation, enabling improved compute performance and reduced power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If DRAM and non-volatile memory are separated into different memory systems, then each memory type can be optimized independently, but bandwidth is limited and power consumption increases due to data transfer between remote memory locations
Solution Approach 1:
The patent combines DRAM and non-volatile memory (ReRAM/PCM) into a single hybrid memory die, allowing both memory types to coexist and operate together. This merging eliminates the need for data transfer between separate memory systems, thereby increasing bandwidth and reducing power consumption associated with inter-memory data movement.
Solution Approach 2:
The patent implements a 3D-stacked memory architecture where DRAM and non-volatile memory are stacked vertically on the same die using through-silicon vias (TSV). This vertical integration creates short interconnect paths between memory types, enabling high-speed data access and reducing the energy required for data transfer compared to traditional planar or separate memory architectures.
2Productivity
If DRAM and non-volatile memory are integrated on the same die, then compute performance improves and energy efficiency increases, but device complexity and manufacturing difficulty increase
Solution Approach 1:
The patent divides the memory die into distinct regions: a first region for DRAM devices, a second region for non-volatile memory devices (ReRAM or PCM), and a third region for logic devices. This segmentation allows each memory type to be optimized independently while maintaining their benefits through close proximity, reducing the complexity of integration compared to a fully homogenized structure.
Solution Approach 2:
The patent introduces a protective spacer layer as an intermediary between the DRAM region and the non-volatile memory region. This spacer layer provides electrical insulation and physical separation, preventing interference between the two memory types during fabrication and operation, thereby simplifying the integration process and reducing manufacturing complexity.
3Reliability
If a protective spacer layer is added to electrically insulate memory regions, then manufacturing precision and device reliability improve, but device complexity and fabrication steps increase
Solution Approach 1:
The protective spacer layer serves multiple functions simultaneously: it provides electrical insulation between DRAM and non-volatile memory regions, acts as a physical barrier during fabrication processes, and maintains structural integrity of the stacked architecture. This multi-functionality reduces the need for additional specialized components, offsetting the increased fabrication steps with consolidated design benefits.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This integration enhances compute performance by reducing data fetching from remote DRAM and boosts energy efficiency by keeping data local, thereby overcoming bandwidth restrictions and improving system performance in AI systems.
Implementation Method 1
a protective spacer layer that electrically insulates the non-volatile memory region from the dynamic random access memory region and the logic region from the dynamic random access memory region
Data Source
AI summary
A high bandwidth memory is provided. The high bandwidth memory includes a region of dynamic random access memory devices, a region of non-volatile memory devices adjacent to the region of dynamic random access memory devices, and a region of logic devices adjacent to both the region of dynamic random access memory devices and the region of non-volatile memory devices.