Stacked Computational Memory Using MRAM Weights for Neural MAC
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The performance of vector matrix multiplication operations, such as in machine learning and neural network processing, is limited by frequent data movement between computational and memory devices, leading to inefficiencies in power consumption and resource utilization.
Innovation Solution
A computational memory device is designed with a stacked structure, where a computational memory block performs MAC operations on a weight memory block, minimizing data movement through in-memory computing, using non-volatile MRAM for weight data and volatile SRAM for input data, and employing a communication interface for efficient data exchange.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If data is frequently moved between computational and memory devices for MAC operations, then computation can be performed, but power consumption increases and efficiency decreases
Solution Approach 1:
The patent combines memory and computation functions into a single integrated device. The computational memory block includes both storage elements (for holding weight and input data) and computation elements (for performing MAC operations), eliminating the need for separate memory and computational devices and the data movement between them.
Solution Approach 2:
The patent introduces a computational memory block as an intermediary component between the weight memory block and additional memory. This intermediary performs MAC operations on data retrieved from weight memory and additional memory, reducing the need for frequent data movement between separate computational and memory devices.
2Quantity of substance
If data is stored in separate memory and computational devices, then data storage is achieved, but data movement increases resource usage
Solution Approach 1:
The patent merges storage and computation capabilities within the computational memory block. The block can store both weight data and input data, and perform MAC operations internally, thereby maintaining data storage capacity while eliminating energy-intensive data movement between separate devices.
3Loss of energy
If in-memory computing is implemented, then data movement is minimized, but device complexity increases
Solution Approach 1:
The patent segments the computational memory device into distinct functional blocks: a weight memory block for storing weight data, a computational memory block for performing MAC operations, and additional memory for storing input data. This segmentation allows each block to be optimized for its specific function while working together to minimize data movement.
Solution Approach 2:
The computational memory block serves as an intermediary that receives weight data from the weight memory block and input data from additional memory, performs MAC operations, and outputs results. This intermediary structure consolidates computation functionality, reducing the need for complex data movement between separate memory and computational devices.
Data Source
AI summary
A computational memory device and a method using the computational memory device are provided. The computational memory device includes memory banks configured to store weight data of a neural network model and a weight memory block configured to provide at least some of the weight data from memory banks in response to a weight request, a computational memory block physically stacked on the weight memory block such faces of the respective blocks face each other, the computational memory block configured to perform a multiply-accumulate (MAC) operation between the at least some of the weight data and at least some of input data by using a bit cell array including bit cells, and a communication interface configured to perform communication between the weight memory block and the computational memory block.