HBM Memory Lookup Engine for In-Memory Deep Learning Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning applications, including deep neural networks, require significant computational and memory resources due to large numbers of operations, which existing technologies struggle to efficiently address.
Innovation Solution
A storage device utilizing a high-bandwidth memory (HBM) stack with a memory die and logic die, incorporating a computation lookup table to perform operations like convolution and matrix multiplication directly in memory, reducing the need for CPU computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If computation is performed in CPU rather than in memory, then device complexity is reduced, but computational efficiency and throughput deteriorate
Solution Approach 1:
The patent merges computation and memory functions into a single integrated structure by embedding computation lookup tables directly within the memory array. This allows multiplication operations to be performed within the memory device itself, eliminating the need to transfer data between memory and CPU, thereby improving computational efficiency while maintaining a unified device architecture.
Solution Approach 2:
The memory device is designed to perform multiple functions: it serves as both storage (memory array) and computation (via embedded lookup tables and row/column decoders). This multi-functionality allows the same hardware structure to handle both data storage and arithmetic operations, improving productivity without requiring separate dedicated computation units.
2Loss of time
If computation operations are performed externally to memory, then ease of operation is maintained, but loss of time due to data transfer increases
Solution Approach 1:
By combining memory storage and computation lookup tables into a single integrated memory array, the patent eliminates the need for data transfer between separate memory and computation units. The row decoder and column decoder work together with the embedded lookup tables to perform multiplication operations in-place, significantly reducing data transfer time while maintaining ease of operation through a unified interface.
3Use of energy by moving object
If large numbers of computational operations are performed, then measurement precision of results improves, but use of energy increases
Solution Approach 1:
The patent introduces computation lookup tables as an intermediary structure within the memory array that pre-stores multiplication results. This allows the system to retrieve pre-computed values through simple address decoding rather than performing full multiplication operations, significantly reducing energy consumption while maintaining computation accuracy for machine learning operations.
Data Source
AI summary
A storage device and method of controlling a storage device are disclosed. The storage device includes a host, a logic die, and a high bandwidth memory stack including a memory die. A computation lookup table is stored on a memory array of the memory die. The host sends a command to perform an operation utilizing a kernel and a plurality of input feature maps, includes finding the product of a weight of the kernel and values of multiple input feature maps. The computation lookup table includes a row corresponding to a weight of the kernel, and a column corresponding to a value of the input feature maps. A result value stored at a position corresponding to a row and a column is the product of the weight corresponding to the row and the value corresponding to the column.