Functional HBM Device Neural Network Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D memory devices, such as HBM, face challenges in power consumption and heat generation during neural network computations due to the need for extensive data communication between processing devices and memory dies.
Innovation Solution
The integration of functional HBM devices that include both volatile memory dies and computational dies, connected via through-silicon vias (TSVs), allows for high-bandwidth communication and the execution of neural network computations within the device itself, reducing the need for external data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is transferred between processing devices and memory dies through external communication interfaces, then data can be accessed, but power consumption and heat generation increase significantly
Solution Approach 1:
The patent merges computational functionality directly into the memory die structure by integrating compute units within the HBM device. This allows neural network computations to be performed inside the memory die itself rather than requiring data to be transferred to external processing devices, thereby reducing power consumption while maintaining data access reliability through internal communication paths.
Solution Approach 2:
The patent transitions from traditional planar data storage to three-dimensional vertically stacked memory dies connected through TSVs. This vertical stacking enables high-bandwidth communication paths directly between memory layers, allowing computations to occur close to data storage locations and reducing the energy required for data transfer compared to external interfaces.
2Reliability
If data is transferred between processing devices and memory dies through external communication interfaces, then data can be accessed, but heat generation increases significantly
Solution Approach 1:
The patent merges computational functionality directly into the memory die structure by integrating compute units within the HBM device. This allows neural network computations to be performed inside the memory die itself rather than requiring data to be transferred to external processing devices, thereby reducing power consumption while maintaining data access reliability through internal communication paths.
Solution Approach 2:
The patent transitions from traditional planar data storage to three-dimensional vertically stacked memory dies connected through TSVs. This vertical stacking enables high-bandwidth communication paths directly between memory layers, allowing computations to occur close to data storage locations and reducing the energy required for data transfer compared to external interfaces.
3Productivity
If vertically stacked memory dies are used with through-silicon vias, then high-bandwidth communication is achieved, but device complexity increases
Solution Approach 1:
The patent divides the HBM device into multiple separate memory dies stacked vertically, with each die containing specific computational or storage functions. This segmentation allows for modular design and independent optimization of each layer while achieving high-bandwidth communication through the TSV interconnects between layers.
Solution Approach 2:
The patent creates a multi-functional HBM device where memory dies serve dual purposes as both data storage and computational processing units. The integration of compute units within memory dies allows the same structure to perform both storage and computation functions, reducing the need for separate dedicated components and managing overall device complexity.
Data Source
AI summary
System-in-package (SiP) having functional high bandwidth memory (HBM) devices, and associated systems and methods are disclosed herein. In some embodiments, the functional HBM devices can include a controller die, one or more volatile memory dies, a flash memory die, and an HBM bus communicably coupled to each of the controller, volatile memory, and flash memory dies. The flash memory die can include one or more word lines that each have multiple programmable memory cells, as well as multiple bit lines. Each of the bit lines is coupled to a corresponding programmable memory cell from each of the one or more word lines. During operation, the controller die is configured to control the volatile memory dies and the flash memory die, through a shared bus therebetween, to implement one or more neural network computing operations within the functional HBM device.