Adapting CNN Architecture for Neuromorphic SNN Implementation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for converting convolutional neural networks (CNNs) to spiking neural networks (SNNs) face challenges such as high power consumption, difficulty in representing negative values, and complexity in implementing sigmoid functions, leading to accuracy loss in classification tasks.
Innovation Solution
The system adapts CNN architecture by making output values positive, removing biases, replacing sigmoid functions with HalfRect functions, and using spatial linear subsampling instead of max-pooling, allowing the converted SNN to be implemented on neuromorphic hardware with minimal performance loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If CNN architecture is directly converted to SNN, then power consumption is reduced, but classification accuracy deteriorates
Solution Approach 1:
The patent transforms the CNN architecture parameters to suit SNN requirements by: (1) replacing sigmoid activation functions with ReLU functions that are compatible with spiking neurons, (2) converting max-pooling operations to average-pooling operations, (3) removing bias terms from convolutional layers, and (4) transforming the loss function from cross-entropy to spike-based classification loss. These parameter changes enable accurate conversion while maintaining low power consumption characteristics of SNNs.
Solution Approach 2:
The patent introduces an intermediary conversion process that translates CNN weights and architecture into equivalent SNN representations. This intermediary transformation layer includes: (1) weight normalization to handle the lack of bias terms, (2) activation function mapping from sigmoid to ReLU, and (3) pooling operation transformation. This intermediary process preserves classification accuracy while enabling deployment on power-efficient SNN hardware.
2Device complexity
If sigmoid functions are implemented in SNN, then nonlinearity is maintained, but implementation complexity increases
Solution Approach 1:
The patent replaces complex sigmoid functions with simple ReLU activation functions that are computationally inexpensive and easily implementable in spiking neural networks. The ReLU function (f(x) = max(0, x)) provides sufficient nonlinearity for deep learning while being trivial to implement with spike-based computations, thus reducing implementation complexity without sacrificing the essential nonlinear transformation capability.
3Ease of manufacture
If max-pooling layers are used in converted SNN, then feature extraction is maintained, but representation of negative values becomes problematic
Solution Approach 1:
The patent inverts the pooling operation from max-pooling to average-pooling. Average pooling computes the mean of input values across the pooling window, which naturally handles both positive and negative values without the selection bias inherent in max-pooling. This inversion resolves the negative value representation problem while maintaining effective feature extraction capability through spatial averaging.
Data Source
AI summary
Described is a system for converting convolutional neural networks to spiking neural networks. A convolutional neural network (CNN) is adapted to fit a set of requirements of a spiking neural network (SNN), resulting in an adapted CNN. The adapted CNN is trained to obtain a set of learned weights, and the set of learned weights is then applied to a converted SNN having an architecture similar to the adapted CNN. The converted SNN is then implemented on neuromorphic hardware, resulting in reduced power consumption.