Stacked Computational Memory for In-Place Neural Network MAC
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The performance of vector matrix multiplication operations, such as those used in neural network processing, is limited by frequent data movement between computational and memory devices, leading to inefficiencies in power consumption and processing speed.
Innovation Solution
A computational memory device is designed with a stacked structure, where a computational memory block is physically stacked on a weight memory block, allowing for in-memory computing through a bit cell array that performs MAC operations directly on stored weight and input data, minimizing data transmission and optimizing power efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If data is frequently moved between computational and memory devices for MAC operations, then processing can be performed, but power consumption increases and processing speed decreases
Solution Approach 1:
The patent combines memory storage functionality with computational MAC operation functionality into a single integrated device. The memory cell array stores weight data and input data, while the same array performs multiplication and accumulation operations in-place, eliminating the need for separate memory and computational devices and the data movement between them.
Solution Approach 2:
The patent introduces sense amplifiers and column decoders as intermediary components that enable direct computation within the memory array. These intermediaries facilitate the MAC operations by detecting and amplifying signals from memory cells and performing bitwise operations without requiring data to be moved to external computational units.
2Productivity
If data is frequently moved between computational and memory devices, then MAC operations can be performed, but the area footprint increases
Solution Approach 1:
The patent merges memory storage and computational processing into a single integrated structure. The memory cell array serves dual purposes: storing weight and input data, and performing MAC operations on this data. This eliminates the need for separate memory devices and computational devices, reducing the total area footprint while maintaining full processing capability.
Solution Approach 2:
The memory cell array is designed to serve multiple functions: storing weight data, storing input data, performing multiplication operations, and performing accumulation operations. This multi-functionality allows the same physical structure to handle both data storage and data processing, reducing the overall device area while maintaining complete processing capability.
3Productivity
If data is frequently moved between computational and memory devices, then neural network processing can be performed, but power efficiency decreases
Solution Approach 1:
The patent combines memory and computation in a single integrated device, eliminating data movement between separate memory and computational devices. This integration dramatically reduces the energy consumed by data transmission while maintaining full neural network processing capability through in-memory MAC operations.
Solution Approach 2:
The memory array performs MAC operations on its own stored data without requiring external computational devices. The weight data and input data stored in the memory cell array are processed directly within the array through self-contained multiplication and accumulation operations, eliminating the energy cost of moving data to external processors.
Data Source
AI summary
A computational memory device and a method using the computational memory device are provided. The computational memory device includes memory banks configured to store weight data of a neural network model and a weight memory block configured to provide at least some of the weight data from memory banks in response to a weight request, a computational memory block physically stacked on the weight memory block such faces of the respective blocks face each other, the computational memory block configured to perform a multiply-accumulate (MAC) operation between the at least some of the weight data and at least some of input data by using a bit cell array including bit cells, and a communication interface configured to perform communication between the weight memory block and the computational memory block.