3D Memory Device for Matrix Vector Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limited memory bandwidth and power consumption issues in deep learning applications, particularly in edge AI systems, due to the bottleneck of inter-chip data movement and the inefficiency of conventional Von-Neumann computer architecture, hinder the performance of matrix vector multiplication operations.
Innovation Solution
An integrated memory device that combines memory and processing, using a 3D memory cell array with analog capabilities to perform matrix vector multiplication efficiently by programming memory cells to represent weights and applying voltages for inference computations, reducing the need for digital logic and minimizing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted from image sensors to microprocessors for processing, then processing capability is provided, but memory bandwidth and power consumption increase due to inter-chip data movement
Solution Approach 1:
The patent merges memory and processing functions into a single integrated circuit device. The memory cell array performs multiplication operations directly during data readout, eliminating the need to transmit data to a separate microprocessor. This integration reduces inter-chip data movement and associated power consumption while maintaining processing capability.
Solution Approach 2:
The memory device performs self-computation by using its own memory cells to execute multiplication operations. The memory cell array multiplies input data by weights internally during the readout process, so the system serves its own processing needs without requiring external processing units, thereby reducing overall power consumption.
2Productivity
If data is transmitted from image sensors to microprocessors for processing, then processing capability is provided, but memory bandwidth is limited due to inter-chip data movement
Solution Approach 1:
By combining memory and processing in one integrated circuit, the patent eliminates the bandwidth bottleneck of inter-chip data movement. The memory cell array performs computations locally during readout, so no data needs to be transmitted across chip boundaries, effectively maximizing the available memory bandwidth for processing operations.
3Productivity
If conventional Von-Neumann architecture is used, then processing can be performed, but latency increases due to inter-chip data movement
Solution Approach 1:
The integration of memory and processing functions eliminates the latency associated with inter-chip data movement. The memory cell array performs multiplication operations during the readout process itself, so data processing occurs immediately without waiting for transmission to a separate processor, significantly reducing latency.
4Productivity
If specialized multiplier-accumulator circuits are used, then multiplication performance is improved, but device complexity increases
Solution Approach 1:
The patent uses the memory cells themselves to perform multiplication operations through their electrical characteristics. The memory cells function as both storage and computation elements, eliminating the need for separate multiplier-accumulator circuits. This approach achieves high multiplication performance while keeping the overall device structure simpler.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables efficient matrix vector multiplication and accumulation operations, reducing latency and power consumption, and enhancing the performance of AI applications by integrating memory and processing within the same integrated circuit device.
Implementation Method 1
Each memory cell in the array is programmed to perform a multiplication operation in response to an applied voltage. The memory cells are operated in a sub-threshold mode such that the multiplication operation is performed without causing the memory cells to threshold.
Data Source
AI summary
Systems, methods, and apparatus related to memory devices that perform signed multi-bit to multi-bit multiplication using sets of memory cells. In one approach, a memory cell array has sets of memory cells. Each set is programmable to store a multi-bit signed weight. Voltage drivers apply voltages to each set. The voltages correspond to multi-bit signed inputs. One or more common lines are coupled to each set for summing output currents from the sets during the multiplication. A digitizer provides signed results based on summing the output currents. The signed results are added with adjustment for the bit significance of each signed result to provide a final accumulation result for the multiplication.