Memory Device Wordline Calibration Matrix Vector Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Limited memory bandwidth in machine learning systems, particularly in deep neural networks, leads to significant latency, power consumption, and inefficiencies due to the bottleneck between processor chips and DRAM, necessitating improved methods for matrix vector multiplication and data storage.
Innovation Solution
Integration of memory and processing in an integrated circuit device with wordline calibration, allowing for efficient matrix vector multiplication within the memory device, reducing the need for data transfer between processing units and memory by performing computations directly in the memory cell array using analog weight-stationary architectures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in external DRAM for machine learning processing, then storage capacity is improved, but memory bandwidth is limited and latency increases
Solution Approach 1:
The patent merges storage and processing functions by implementing in-memory matrix multiplication within the memory device itself. The memory array performs computational operations directly on stored data, eliminating the need to transfer data between external DRAM and processor. This combines the storage capacity of DRAM with the processing capability of the memory array, resolving the bandwidth bottleneck while maintaining large storage capacity.
Solution Approach 2:
The memory device performs self-computation by using its own array to execute matrix multiplication operations on stored weight and activation data. The processing function is embedded within the memory structure itself, allowing the memory to serve both storage and computation purposes without requiring external processing units to access the data.
2Quantity of substance
If data is constantly loaded and saved between GPU and external DRAM, then storage requirements are met, but training time increases and power consumption increases
Solution Approach 1:
The patent combines storage and processing in a single integrated memory device, eliminating the need for frequent data transfers between external DRAM and processor. By performing matrix multiplication in-memory, the system reduces the total data movement volume, which directly lowers power consumption while maintaining the required storage capacity for large neural network models.
3Productivity
If processor chips are specialized for serial processing and DRAM is optimized for high density, then each component is optimized, but the interface between them creates a bottleneck
Solution Approach 1:
The patent merges the processing function into the memory device itself, creating an integrated system where the memory array performs matrix multiplication operations. This eliminates the complex interface between separate processor and memory components, as the computation occurs within the memory structure using its own address and data lines, simplifying the overall system architecture while maintaining high processing efficiency.
Data Source
AI summary
Systems, methods, and apparatus related to memory devices that perform multiplication using memory cells. In one approach, a memory cell array has memory cells used to perform matrix vector multiplication based on summing output currents from the memory cells. A context of the memory cell array is determined by a controller (e.g., a memory controller internal or external to a memory chip having the array). The context can include, for example, memory cell conditions related to data retention stress, quick charge loss, back-pattern effects, and/or cross-temperature variations. Based on the determined context, the controller dynamically determines adjustments to wordline and/or other memory cell bias voltages used during the multiplication.