Heterogeneous CNN-SNN Accelerator for Layer-Wise Energy Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network accelerators struggle with achieving high energy efficiency and accuracy due to the inherent differences in operation methods and memory access patterns between convolutional neural networks (CNN) and spiking neural networks (SNN), limiting their ability to efficiently process both types of networks simultaneously.
Innovation Solution
A complementary deep neural network accelerator with a heterogeneous convolutional neural network and spiking neural network core architecture that includes a spiking neural network processing module, a multiplier/accumulator array, a top-level RISC controller, an attention module, and a neural network operation allocator, which optimizes energy consumption by determining the mixing method of both networks and reducing repetitive operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a homogeneous architecture processes both CNN and SNN, then device complexity is reduced, but energy efficiency deteriorates due to different operation methods and memory access patterns
Solution Approach 1:
The accelerator is divided into two separate processing cores: a CNN processing core and an SNN processing core. Each core is optimized for its specific neural network type, allowing independent optimization of operation methods and memory access patterns without compromising the other. This segmentation resolves the contradiction by sacrificing some architectural simplicity to achieve superior energy efficiency for each network type.
Solution Approach 2:
Different parts of the accelerator are designed with specialized characteristics suited to their specific neural network type. The CNN core uses conventional multiply-accumulate units with regular memory access, while the SNN core uses spike-based event-driven processing with sparse memory access. This local optimization allows each component to operate at peak efficiency for its designated function.
2Measurement precision
If only CNN is used for processing, then high accuracy is achieved through back-propagation training, but energy consumption increases due to repeated calculations
Solution Approach 1:
The system dynamically selects between CNN and SNN processing modes based on the specific computational task and data characteristics. For tasks requiring high-precision training, the CNN core is activated with full back-propagation capability. For inference tasks or scenarios where approximate results suffice, the SNN core is used to reduce energy consumption. This dynamic adaptation resolves the contradiction by matching the processing method to the specific requirements of each operation.
3Use of energy by moving object
If SNN is used for low-power processing, then energy efficiency improves through spike-based operation, but accuracy deteriorates compared to CNN
Solution Approach 1:
The accelerator merges both CNN and SNN processing capabilities into a single system, allowing the two different approaches to be combined for complementary effects. The CNN core handles tasks requiring high precision and complex training, while the SNN core handles energy-constrained tasks and inference. The system can also transfer learned weights between the two cores, allowing the SNN to benefit from CNN training while maintaining its energy efficiency advantages.
4Measurement precision
If CNN-to-SNN conversion algorithm is used, then SNN achieves comparable accuracy to CNN, but energy consumption varies significantly across layers due to spike sparsity
Solution Approach 1:
The system dynamically routes different layers or batches of data to either the CNN core or SNN core based on their specific energy efficiency characteristics. Layers with high spike sparsity are directed to the SNN core for energy-efficient processing, while layers requiring dense computation are handled by the CNN core. This dynamic task allocation resolves the contradiction by optimizing energy consumption at the layer level while maintaining overall accuracy.
Data Source
AI summary
A complementary deep neural network accelerator includes: an accumulator array spiking neural network array processing module; a multiplier-accumulator convolutional neural network processing module; a highest RISC controller responsible for controlling the spiking neural network processing module and the convolutional neural network processing module, and processing an activation function and batch normalization; an attention module; and a neural network operation allocator.