In-Memory Computing Macros for Neural Network Convolution Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face inefficiencies in performing multiplication and accumulation operations due to the large number of matrices involved, which can lead to high operation quantities and complexity, especially when executed on general-purpose processors.
Innovation Solution
The method involves dividing input vectors into chunks to match the structure of in-memory computing (IMC) macros, performing first and second convolution operations, and applying batch normalization and activation functions, with the option for frame-wise incremental computation to reduce memory usage and operation quantity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If convolutional neural network operations are performed using general-purpose processors with in-memory computing, then multiplication and accumulation operations can be executed, but the large number of matrices involved leads to high operation quantity and complexity
Solution Approach 1:
The patent divides the input vector into multiple chunks (first chunks, second chunks) and performs convolution operations in stages (first convolution operation, second convolution operation). This segmentation reduces the complexity of handling large matrices by breaking them into smaller manageable portions that can be processed through multiple IMC macros in parallel
Solution Approach 2:
The patent transforms the channel direction dimension by dividing the input vector into chunks along the channel dimension and processing them through multiple IMC macros. This dimensional transformation enables parallel processing of convolution operations, reducing overall operation complexity while maintaining productivity
2Productivity
If a large number of matrices are used in convolutional neural network operations, then convolution operations can be performed, but data movement and power consumption increase
Solution Approach 1:
The patent merges multiple convolution operations into a unified process that leverages in-memory computing macros. By combining the first and second convolution operations with batch normalization and activation functions in a coordinated manner, the system reduces redundant data movements and optimizes power consumption while maintaining full convolutional processing capability
Solution Approach 2:
The in-memory computing macros perform computation directly within memory structures, eliminating the need for frequent data transfers between memory and processing units. This self-service approach reduces energy consumption by minimizing data movement while maintaining high productivity through efficient in-place operations
3Productivity
If input vectors are divided into chunks to match IMC macro structure, then operation efficiency improves, but the process requires additional division and allocation steps
Solution Approach 1:
The patent performs preliminary division of the input vector into first chunks and second chunks before executing the convolution operations. This preliminary action organizes the data in advance to match the IMC macro structure, which虽然 adds division steps but enables highly efficient parallel execution that overall improves productivity
Solution Approach 2:
The patent changes the parameter organization by transforming the input vector into chunks with specific size constraints (size of each channel ≤ number of rows of IMC macros). This parameter transformation optimizes the data structure for IMC macro processing, improving execution efficiency despite the additional processing steps required
Data Source
AI summary
A processor-implemented method includes receiving an input vector comprising a plurality of channels, performing a first convolution operation by allocating first chunks, obtained by dividing the input vector, to a plurality of first in-memory computing (IMC) macros, and performing a second convolution operation by allocating second chunks obtained by dividing a result of the first convolution operation to a plurality of second IMC macros.