Compute-in-Memory Memory Array for Neural Network MAC Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The bottleneck in machine learning applications is the slower data transfer between processors and memory devices, which limits computational throughput and increases energy consumption in massively parallel neural network computations.
Innovation Solution
A compute-in-memory (CIM) device using a memory array with bit cells arranged in rows and columns to perform multiply-and-accumulate (MAC) operations, where read word lines provide input activations and read-bit lines receive multiplication products, enabling in-memory computation with reduced data movement and energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred between processor and memory devices for computation, then computation can be performed using standard architecture, but data transfer speed becomes a bottleneck limiting computational throughput
Solution Approach 1:
The patent merges the memory array and computation logic into a unified compute-in-memory device. The memory array performs MAC operations directly within the memory structure by utilizing the natural floating behavior of bit cells during read operations, eliminating the need for separate processor-memory data transfer and significantly improving computational throughput.
Solution Approach 2:
The patent introduces read word lines and read bit lines as intermediary elements that facilitate in-memory computation. These lines serve dual purposes: traditional memory read operations and computation operations, acting as mediators between stored data and processing logic without requiring external data transfer.
2Productivity
If data is transferred between processor and memory devices, then computation can be performed, but energy consumption increases due to repeated data movement
Solution Approach 1:
The patent combines memory storage and computation functions into a single integrated structure. By performing MAC operations within the memory array using the existing read word lines and read bit lines, the system eliminates energy-consuming data transfer between separate memory and processor components.
Solution Approach 2:
The memory array performs computation operations on its own stored data without requiring external processor intervention. The bit cells naturally float during read operations, and this floating behavior is harnessed to perform multiplication and accumulation operations directly within the memory structure, making the memory self-sufficient for computation.
3Adaptability or versatility
If fixed number of filters are used in memory array, then hardware design is simplified, but flexibility for different neural network configurations is reduced
Solution Approach 1:
The patent implements a dynamic filter configuration system where the number of active filters can be programmatically adjusted. Control logic selectively activates or deactivates specific columns of bit cells based on the required neural network configuration, allowing the same hardware to adapt to different numbers of filters without physical reconfiguration.
Solution Approach 2:
The memory array is designed with universal functionality to support variable filter configurations. The same read word lines and read bit lines can be used regardless of how many filters are active, and the control logic can dynamically reconfigure the array to accommodate different neural network requirements, making the hardware universally applicable to multiple configurations.
Data Source
AI summary
Aspects of the present disclosure are directed to devices and methods for performing MAC operations using a memory array as a compute-in-memory (CIM) device that can enable higher computational throughput, higher performance and lower energy consumption compared to computation using a processor outside of a memory array. In some embodiments, an activation architecture is provided using a bit cell array arranged in rows and columns to store charges that represent a weight value in a weight matrix. A read word line (RWL) may be repurposed to provide the input activation value to bit cells within a row of bit cells, while a read-bit line (RBL) is configured to receive multiplication products from bit cells arranged in a column. Some embodiments provide multiple sub-arrays or tiles of bit cell arrays.