Combined HBM Memory Stack With Non-Volatile Backup Dies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current High Bandwidth Memory (HBM) devices suffer from limited storage capacity and power dependency of volatile memory, leading to bottlenecks in data transfer and power management, especially in high-performance computing applications like AI/ML, due to the separation of volatile memory dies and non-volatile storage devices.
Innovation Solution
Integration of both volatile and non-volatile memory dies within a combined HBM device using through-silicon vias (TSVs) to create a high-bandwidth communication path, allowing for efficient data transfer and power management by using non-volatile memory as a memory extension and data backup solution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If volatile memory dies are separated from non-volatile storage devices, then power management is simplified, but data transfer bandwidth is limited and power consumption increases
Solution Approach 1:
The patent combines volatile memory dies and non-volatile storage devices into a single integrated HBM package, allowing them to share common TSV interconnects and control logic. This merging enables high-bandwidth data transfer between volatile and non-volatile memory while reducing overall power consumption through shared infrastructure and coordinated power management.
Solution Approach 2:
The patent introduces an intermediary controller that manages data transfer between volatile and non-volatile memory dies through shared TSVs. This intermediary coordinates access patterns, optimizes data movement, and enables the system to achieve high bandwidth while maintaining efficient power management through intelligent control.
2Ease of manufacture
If volatile memory dies are stacked separately from non-volatile storage, then manufacturing processes are simpler, but storage capacity and data transfer speed are limited
Solution Approach 1:
The patent merges multiple memory dies of different types (volatile and non-volatile) into a single stacked package, achieving high storage capacity through vertical integration while maintaining manufacturing feasibility through standardized TSV processes and modular die design.
Solution Approach 2:
The patent transitions from planar memory organization to three-dimensional stacking, enabling significantly increased storage capacity by utilizing the vertical dimension. Multiple dies are stacked and interconnected through TSVs, creating a high-capacity storage system that overcomes the limitations of two-dimensional memory layouts.
3Device complexity
If external storage devices are used, then device complexity is reduced, but data transfer speed and processing efficiency decrease
Solution Approach 1:
The patent integrates storage functionality directly into the HBM package by combining volatile and non-volatile memory dies, eliminating the need for separate external storage devices. This integration maintains relatively simple device architecture while achieving high data transfer speeds through short TSV interconnects and direct coupling to the memory interface.
Data Source
AI summary
System-in-packages (SiPs) having combined high bandwidth memory (HBM) devices, and associated systems and methods, are disclosed herein. In some embodiments, the SiP includes a base substrate (e.g., a silicon interposer), a processing unit carried by the base substrate, and a HBM device carried by the base substrate. The combined HBM device can be electrically coupled to the processing unit through one or more traces. Further, the combined HBM device can include an interface die, one or more volatile memory dies carried by the interface die (e.g., a volatile, main memory component), and one or more non-volatile memory dies carried by the one or more memory dies. The combined HBM device can also include a shared bus that is electrically coupled to the interface die, the volatile memory dies, and the non-volatile memory dies to establish communication paths therebetween.