Mixed-Precision CIM Accelerator Co-Design for CNN Weight Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Computing-In-Memory (CIM)-based accelerators for Convolutional Neural Networks (CNNs) are inefficient when operating on mixed-precision networks due to wasted memory and computational resources, as they only support fixed-precision calculations, leading to low utilization and reduced computational efficiency.
Innovation Solution
A hardware and software co-design method with a mixed-precision algorithm and CIM-based accelerator that includes a memory, processor, and CIM-based accelerator, performing operations such as pruning quantization joint training and mixed-precision quantization to generate mixed-precision weights, enabling full-scale computations and improving CIM utilization and computational speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional CIM-based accelerators use fixed-precision calculations, then hardware simplicity is maintained, but memory and computational resources are wasted on mixed-precision networks
Solution Approach 1:
The system dynamically adjusts precision levels during different stages of neural network processing. High-precision weights are used for critical layers while lower-precision weights are used for less critical layers, allowing the CIM accelerator to adapt to mixed-precision requirements without wasting resources on uniform high-precision calculations throughout
Solution Approach 2:
Different parts of the neural network are assigned different precision levels based on their importance. The system identifies which weight parameters require high precision and which can use lower precision, applying quality locally rather than uniformly across the entire model, thereby reducing wasted computational resources
2Measurement precision
If conventional CIM accelerators store all weight parameters at full precision, then computational accuracy is maintained, but memory utilization is reduced
Solution Approach 1:
The system applies different storage precision to different weight parameters based on their importance and sensitivity. Critical weights are stored at high precision while less critical weights are stored at lower precision, optimizing the balance between memory utilization and computational accuracy
Solution Approach 2:
The system changes the precision parameter of weight storage based on the specific requirements of different neural network layers and operations. This allows the memory to store weights efficiently without sacrificing the accuracy needed for successful inference
3Adaptability or versatility
If mixed-precision quantization is implemented without hardware optimization, then algorithm flexibility is improved, but computational speed is reduced
Solution Approach 1:
The system performs preliminary quantization and precision assignment before the actual computational work begins. By pre-determining which weights need high precision and which can use lower precision, the system avoids unnecessary computational overhead during execution, maintaining speed while preserving algorithm flexibility
Data Source
AI summary
A hardware and software co-design system with a mixed-precision algorithm and a computing-in-memory (CIM)-based accelerator includes a memory, a processor and the CIM-based accelerator. The processor performs operations including obtaining a plurality of sets of initial weight parameters of a pre-trained model from the memory; performing a pruning procedure on the sets of initial weight parameters to generate a plurality of sets of pruned weights; and performing a filter-wise mixed-precision quantization training on a plurality of non-zero weights of the sets of pruned weights to generate a plurality of filter weights with different bit widths, and pairing the filter weights to generate a plurality of paired filter weight groups, and mixing the paired filter weight groups to generate a plurality of mixed-precision weights. The CIM-based accelerator performs a CIM operation on the mixed-precision weights and a plurality of sets of input parameters to generate a plurality of CIM outputs.