Memory Cell Pre-Compensation for Matrix Vector Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Limited memory bandwidth in machine learning systems, particularly in deep neural networks, leads to latency, bandwidth limitations, and increased power consumption due to the bottleneck at the interface between processor chips and DRAM, necessitating efficient storage and computation of large weights and activations.
Innovation Solution
The integration of memory and processing in an integrated circuit device, where memory cells adjust their programming to compensate for predicted conditions during multiplication operations, such as threshold voltage shifts, to maintain accurate output currents and reduce errors, allowing for efficient matrix vector multiplication within the memory device itself.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in external DRAM for neural network computations, then storage capacity is sufficient, but memory bandwidth is limited and power consumption increases
Solution Approach 1:
The patent combines memory storage and processing functions into a single integrated circuit device. Memory cells are configured to perform multiplication operations directly within the memory array, eliminating the need to transfer data between external DRAM and separate processing units. This merging of storage and computation reduces memory bandwidth requirements and lowers power consumption associated with data movement.
2Productivity
If data is frequently transferred between external DRAM and processor, then computation can be performed, but latency and bandwidth limitations increase
Solution Approach 1:
The memory device performs computations autonomously within the memory array without requiring external processor intervention for each operation. Weight values are stored in memory cells and multiplication operations are executed in-place using the stored data, allowing the memory system to serve its own computational needs and eliminate transfer latency.
3Ease of manufacture
If memory cells are programmed with standard threshold voltages, then programming is simple, but output current accuracy decreases due to threshold voltage shifts during multiplication
Solution Approach 1:
The patent applies pre-compensation to the threshold voltage programming process. Before performing multiplication operations, memory cells are programmed with adjusted threshold voltages that anticipate and compensate for expected voltage shifts during computation. This preliminary adjustment ensures that output currents remain accurate despite subsequent threshold voltage drift, without requiring complex real-time correction circuits.
Data Source
AI summary
Systems, methods, and apparatus related to memory devices that perform matrix vector multiplication using memory cells. In one approach, a memory cell array has memory cells used to perform matrix vector multiplication based on summing output currents from the memory cells. A context of memory cells is determined by a controller (e.g., a memory controller internal or external to a memory chip having the array). The context can include, for example, a physical location of memory cells, weight patterns being programmed, and/or neighboring cell interference, etc. Based on the determined context, the controller dynamically determines adjustments (e.g., adjusted target threshold voltages or currents) for programming the memory cells to store weights prior to performing the matrix vector multiplication.