In-Memory Convolution Using Analog Sensing Circuitry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNN) operations in machine learning consume significant power and slow down performance due to frequent data movement between GPUs and DRAMs for convolutional operations.
Innovation Solution
A device with memory cells configured for in-place convolution, where a filter array and input array are stored in addressable memory and convolved without being moved, using sensing and writing circuitry to generate and store output values as analog levels, reducing the need for verify cycles and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If data is moved between GPUs and DRAMs for convolutional operations, then the operations can be performed using standard memory architecture, but power consumption increases and performance slows down
Solution Approach 1:
The patent combines the memory storage function with the convolution computation function into a single integrated structure. Memory cells store both input data and filter weights, while sensing circuitry performs convolution operations directly on the stored data, eliminating the need for separate data movement between memory and processing units.
Solution Approach 2:
The sensing circuitry acts as an intermediary that performs the convolution operation directly within the memory array. Instead of moving data to a separate processor, the sensing circuitry compares electrical differences between memory cells to compute convolution results in-place, reducing data movement overhead.
2Ease of operation
If data is moved between GPUs and DRAMs for convolutional operations, then standard computing workflows can be maintained, but performance speed decreases
Solution Approach 1:
The patent merges storage and computation functions into a single memory array, allowing convolution operations to be executed directly on stored data without external data movement. This in-place computation maintains simplicity of operation while dramatically improving performance speed.
Solution Approach 2:
The memory array is segmented into different functional blocks including input data storage, filter weight storage, and output storage, with sensing circuitry distributed throughout to perform local convolution operations. This segmentation enables parallel processing of multiple convolution operations simultaneously.
3Reliability
If verify cycles are used to ensure digital values are correctly stored, then data accuracy is maintained, but storage time increases
Solution Approach 1:
The patent changes the data representation parameter from digital values to analog levels. Memory cells store analog voltage levels that directly represent convolution results, eliminating the need for verify cycles. The analog nature of the stored values provides sufficient accuracy for machine learning applications without requiring additional verification steps.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances performance and reduces power consumption by eliminating the need for data movement during convolution operations, allowing for efficient generation and storage of output values directly in memory.
Implementation Method 1
Sensing circuitry is coupled to the first block of memory cells and the second block of memory cells to compare electrical differences between the memory cells in the first block and the memory cells in the second block to generate the array of output values
Implementation Method 2
As used herein, an analog level can be stored without verify cycles to verify that the cell has been changed to the target resistance or threshold range corresponding to a particular digital value
Implementation Method 3
Writing circuitry operatively coupled to the third block can store the array of output values in the third block of memory cells
Data Source
AI summary
A device comprises a first block of memory cells, a second block of memory cells to store a feature array, and a third block of memory cells to store an array of output values. Sensing circuitry is coupled to the first block of memory cells and the second block of memory cells to compare electrical differences between the memory cells in the first block and the memory cells in the second block to generate the array of output values. Writing circuitry is operatively coupled to the third block to store the array of output values in the third block of memory cells.