Flash-Based AI Accelerator In-Memory MAC Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning neural networks are computationally intensive, leading to significant data transfer requirements when reading and transferring weight values between memory and processing units, which can result in inefficiencies and accuracy issues due to parasitic resistance and sneak currents in large-scale non-volatile memory cell arrays.
Innovation Solution
A computing apparatus that includes a memory device with non-volatile memory cells, a word line driving circuitry, source line circuitry, and bit line circuitry, configured to perform multiplication and accumulation calculations directly on the memory cells, reducing the need for data transfer by quantizing floating-point weight values and storing them in flash memory for later use, thereby minimizing data exchange with the host processor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If neural network weight values are stored in large-scale non-volatile memory cell arrays, then storage capacity is improved, but parasitic resistance and sneak currents increase causing accuracy issues
Solution Approach 1:
The memory array is divided into multiple blocks, with each block containing a subset of weight values. This segmentation allows parallel processing of different blocks, reducing the overall computation time while maintaining accuracy by limiting the number of cells actively involved in each MAC operation.
Solution Approach 2:
Different blocks of the memory array are configured with different read reference voltages optimized for their specific weight value ranges. This local optimization compensates for parasitic effects in different regions of the array, maintaining computational accuracy across the entire large-scale storage capacity.
2Adaptability or versatility
If weight values are transferred between memory and processing units, then computational flexibility is improved, but data transfer time and power consumption increase
Solution Approach 1:
The memory device is integrated with MAC circuitry to form a unified processing-in-memory architecture. This merging eliminates the need for separate data transfer between memory and processing units, significantly reducing transfer time and power consumption while maintaining computational flexibility through on-memory operations.
Solution Approach 2:
The memory device performs multiplication and accumulation operations internally using its own circuitry rather than relying on external processing units. This self-service capability reduces data transfer requirements while maintaining the ability to perform diverse neural network computations.
3Quantity of substance
If floating-point weight values are quantized for flash memory storage, then storage efficiency is improved, but precision is reduced
Solution Approach 1:
The precision of quantization is dynamically adjusted based on the specific neural network layer and computational requirements. Critical layers use higher precision quantization while less critical layers use lower precision, optimizing the balance between storage efficiency and computational accuracy.
Solution Approach 2:
Full precision floating-point values are stored for a subset of critical weight values that have the greatest impact on computational accuracy, while other weight values use compressed quantization. This partial high-precision approach maintains overall accuracy while improving storage efficiency.
Data Source
AI summary
A computing apparatus comprises a host circuit; and a computing device that includes a memory device for facilitating a neural network, the computing device configured to: read weight values from respective non-volatile memory cells in the memory device by biasing the non-volatile memory cells; perform a multiplication and accumulation calculation on the non-volatile memory cells using the read weight value; and output a result of the multiplication and calculation operation to the host system.