CNN Partial Sum Quantization for CIM Hardware Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) using Computation-in-Memory (CIM) hardware face calculation errors due to partial sum outputs not being in full precision, leading to reduced prediction accuracy.
Innovation Solution
A quantization method involving a probability-based and margin-based approach, which includes training the network, generating quantization levels, quantizing partial sums, retraining the network, and adjusting quantization edges to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If quantization is applied to partial sums in CIM hardware, then energy efficiency and computational speed are improved, but calculation errors increase and prediction accuracy decreases
Solution Approach 1:
The patent applies preliminary action by performing probability-based quantization analysis during the training phase to determine optimal quantization levels and edges before actual inference. This pre-computation of quantization parameters allows the system to maintain higher accuracy during runtime while still benefiting from quantized operations in CIM hardware.
Solution Approach 2:
The patent changes parameters by dynamically adjusting quantization levels and edges based on the probability distribution of partial sums. Instead of using fixed quantization parameters, the system adapts them to match the actual data distribution, thereby reducing quantization errors while maintaining energy efficiency benefits.
2Measurement precision
If full precision is used for partial sums, then prediction accuracy is maintained, but energy consumption increases and computational speed decreases
Solution Approach 1:
The patent applies local quality by using different precision levels for different parts of the computation. Specifically, it uses probability-based analysis to identify which partial sums require higher precision and which can tolerate lower precision, thereby optimizing the balance between accuracy and energy consumption locally rather than uniformly across all operations.
3Measurement precision
If quantization levels are increased to reduce errors, then prediction accuracy improves, but device complexity and computational overhead increase
Solution Approach 1:
The patent applies partial action by using quantization levels that are sufficient but not excessive. The probability-based approach determines the minimum necessary quantization precision for each partial sum, avoiding the complexity of uniform high-precision quantization while maintaining adequate accuracy through targeted precision allocation.
Data Source
AI summary
A quantization method for a plurality of partial sums of a convolution neural network based on a computing-in-memory hardware includes a probability-based quantizing step and a margin-based quantizing step. The probability-based quantizing step includes a network training step, a quantization-level generating step, a partial-sum quantizing step, a first network retraining step and a first accuracy generating step. The margin-based quantizing step includes a quantization edge changing step, a second network retraining step and a second accuracy generating step. The quantization edge changing step includes changing a quantization edge of at least one of a plurality of quantization levels. The probability-based quantizing step is performed to generate a first accuracy value, and the margin-based quantizing step is performed to generate a second accuracy value. The second accuracy value is greater than the first accuracy value.