Compute-In-Memory Convolution via Vector-Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolution operations in learning networks are inefficient due to the use of nested loops for dot product calculations, and the differences in kernel and input tensor sizes across layers can further impact efficiency.
Innovation Solution
The proposed solution involves using compute engines with compute-in-memory (CIM) hardware modules that store weights corresponding to a kernel and perform vector-matrix multiplications (VMMs). The method includes quantizing the activation, performing VMMs, and then dequantizing the product, with a GP processor controlling the process and setting clock frequencies based on the number of VMMs performed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If nested loops are used to calculate dot products in convolution operations, then the implementation is simple and straightforward, but the computation time increases significantly
Solution Approach 1:
The patent replaces traditional software-based nested loop implementations with hardware-based compute-in-memory (CIM) modules that perform dot product calculations through physical electrical operations. The CIM hardware directly computes dot products by multiplying input activations with stored kernel weights and summing the results in parallel, eliminating the sequential nature of software loops and dramatically reducing computation time.
Solution Approach 2:
The patent introduces compute engines as intermediary components between the general-purpose processor and memory systems. These compute engines contain CIM modules that act as specialized intermediaries for convolution operations, bridging the gap between general-purpose computing and specialized hardware acceleration, thereby improving both simplicity and speed.
2Speed
If AI accelerators are used to perform convolution operations, then the computation speed improves, but the device complexity increases
Solution Approach 1:
The patent designs compute engines with universal CIM modules that can be configured to perform different convolution operations through software control. The same hardware module can handle various kernel sizes, input tensor dimensions, and convolution configurations by loading different weight sets and adjusting operational parameters, thereby achieving high speed without proportionally increasing hardware complexity.
Solution Approach 2:
The patent implements dynamically reconfigurable compute engines where the CIM modules can adapt their operation based on the specific convolution task. The hardware can adjust its configuration, clock frequencies, and operational modes to match the requirements of different layers and operations, providing high performance across diverse workloads without requiring dedicated hardware for each specific operation.
3Adaptability or versatility
If different kernel and input tensor sizes are used across layers, then the model can capture diverse features, but the efficiency of the learning network decreases
Solution Approach 1:
The patent implements dynamically reconfigurable compute engines that can adapt their internal configuration to match the specific requirements of each convolution layer. The CIM modules can adjust their operational parameters, such as the number of MAC operations, clock frequencies, and data precision, based on the kernel size and input tensor dimensions of the current layer, thereby maintaining high efficiency across diverse feature extraction requirements.
Solution Approach 2:
The patent changes operational parameters of the compute engines based on the specific convolution task. The system adjusts clock frequencies, precision levels, and computational depth according to the kernel size and input tensor dimensions, allowing the same hardware to efficiently handle varying layer requirements without sacrificing adaptability for diverse feature extraction.
Data Source
AI summary
A method for performing a convolution is described. The method includes providing an activation to a general purpose (GP) processor. The GP processor is coupled with compute engines. Each of the compute engines includes a compute-in-memory (CIM) hardware module. The CIM hardware module stores weights corresponding to a kernel and is configured to perform vector-matrix multiplications (VMMs) for the kernel. The method also includes performing, by the GP processor or at least one of the compute engines, a quantization of the activation to provide a quantized activation. The compute engine(s) perform the VMMs for the quantized activation and the kernel to provide a product. Dequantization of the product is performed by the GP processor or the compute engine(s).