In-Memory Convolution Accelerator for CNN Data Movement Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutions in convolutional neural networks (CNNs) require significant computer resources and data movement, making them computationally intensive and costly.
Innovation Solution
In-memory computing is used to perform convolutions by storing kernel matrix elements in memory cells, allowing input vectors to be applied in parallel to these cells for simultaneous computation of output matrix elements, reducing data movement and resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional von Neumann architecture is used for convolution computation, then computational flexibility is maintained, but data movement between memory and processor increases significantly
Solution Approach 1:
The patent merges memory and computation functions into a single integrated structure. Memory cells store kernel weights and simultaneously perform analog multiplication with input signals, eliminating the need for separate memory and processing units. This integration directly reduces data movement and energy consumption while accepting increased device structural complexity.
Solution Approach 2:
The patent replaces digital mechanical computation (sequential data movement and processing) with analog electrical computation. Electrical currents flow directly through memory cells to perform multiplication and accumulation operations, substituting the mechanical von Neumann data movement cycle with continuous electrical signal processing.
2Productivity
If kernel matrix elements are stored in separate memory locations, then data organization is simplified, but the amount of data movement required for convolution increases
Solution Approach 1:
The patent transitions from one-dimensional sequential data access to two-dimensional spatial parallel processing. Multiple kernel elements are arranged in a memory array where each cell can be accessed simultaneously through word lines and bit lines, enabling parallel computation across multiple data points and dramatically increasing productivity while reducing total data movement.
Solution Approach 2:
The patent performs preliminary organization of kernel data in the memory array before computation begins. Kernel weights are pre-positioned in specific memory locations corresponding to their spatial relationships in the convolution operation, allowing input signals to be applied once and have all necessary multiplications performed simultaneously without subsequent data shuffling.
3Loss of time
If multiple input vectors are processed sequentially, then computation accuracy is maintained, but processing time increases
Solution Approach 1:
The patent enables continuous parallel processing of multiple input vectors through the analog nature of the memory array. While digital systems must complete one vector before starting the next, the analog memory array can receive and process multiple input vectors simultaneously through continuous electrical signal flow, dramatically reducing processing time while maintaining accuracy through the inherent linearity of analog computation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach significantly decreases the computational resources and data movement needed for convolutions, enhancing operational speed and reducing power consumption.
Implementation Method 1
In-memory computing is an approach in which memory cells, organized in an in-memory computing device, can be used for both computation and storage. The weights for sum-of-products functions, for example, can be stored in memory cells of the in-memory computing device. The electrical characteristics of the memory cells of the array effectuate the sum-of-products function.
Implementation Method 2
The weights for sum-of-products functions, for example, can be stored in memory cells of the in-memory computing device. The electrical characteristics of the memory cells of the array effectuate the sum-of-products function.
Data Source
AI summary
A method for accelerating a convolution of a kernel matrix over an input matrix for computation of an output matrix using in-memory computation involves storing in different sets of cells, in an array of cells, respective combinations of elements of the kernel matrix or of multiple kernel matrices. To perform the convolution, a sequence of input vectors from an input matrix is applied to the array. Each of the input vectors is applied to the different sets of cells in parallel for computation during the same time interval. The outputs from each of the different sets of cells generated in response to each input vector are sensed to produce a set of data representing the contributions of that input vector to multiple elements of an output matrix. The sets of data generated across the input matrix are used to produce the output matrix.