Hybrid ANN-SNN Network Layers for Low-Power Signal Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks face inefficiencies in converting between analog and spiking neural networks, requiring additional configurations for spike signal conversion, loss function calculations, and time delays, which hinder low-power operation.
Innovation Solution
A hybrid neural network apparatus is developed, comprising input, intermediate, and output layers of analog and spiking neural networks, with controlled data normalization and conversion between layers to minimize data conversion complexity and enable low-power operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If spiking neural network (SNN) is used for low-power operation, then energy consumption is reduced, but additional configuration for spike signal conversion and loss function calculation is required
Solution Approach 1:
The neural network is segmented into different layers: input layer group with ANN layers, intermediate layer group with SNN layers, and output layer group with ANN layers. This segmentation allows the system to leverage the low-power advantage of SNN in the intermediate layers while maintaining compatibility with ANN in input and output layers, thus reducing overall energy consumption without requiring complete conversion of all network components.
Solution Approach 2:
Conversion layers are introduced as intermediaries between ANN layers and SNN layers. These conversion layers handle the spike signal conversion and loss function calculation automatically, reducing the manual configuration complexity. The intermediary conversion layers enable seamless data exchange between different network types without requiring complex manual setup.
2Adaptability or versatility
If spike signal conversion is implemented for ANN application input data, then SNN can process level value data, but time delay occurs due to loss function calculation
Solution Approach 1:
The conversion layers perform spike signal conversion and loss function calculation in advance during the training phase. By pre-computing and storing the conversion parameters and loss function details in the conversion layers, the actual data processing during inference requires minimal additional time, thus reducing time delay while maintaining data format compatibility.
Solution Approach 2:
The conversion layers continuously perform the conversion and loss function calculation operations during both training and inference phases without interruption. This continuous operation ensures that the data flow between ANN and SNN layers remains smooth and uninterrupted, minimizing time delays while maintaining adaptability between different data formats.
3Adaptability or versatility
If hybrid neural network with ANN and SNN layers is constructed, then advantages of both networks are utilized, but data conversion complexity increases
Solution Approach 1:
The conversion layers are designed with multi-functionality, handling both spike signal conversion and loss function calculation within the same layer structure. This universal design reduces data conversion complexity by consolidating multiple functions into single layers, while enabling the hybrid network to leverage the advantages of both ANN and SNN architectures.
Solution Approach 2:
The conversion layers are self-configuring and automatically perform the necessary conversions and loss function calculations without requiring external intervention. During training, the conversion layers automatically adjust to the specific data characteristics, and during inference, they autonomously handle the data format conversions, thus reducing the overall complexity of managing data flow between different network types.
Data Source
AI summary
The present invention relates to a hybrid neural network apparatus. The hybrid neural network apparatus includes an input layer group comprising at least one analog neural network (ANN) layer and trained by information or data input from an application system, an intermediate layer group comprising at least one spiking neural network (SNN) layer and trained by a received training result of the input layer group, and an output layer group comprising at least one ANN layer, trained by a received training result of the intermediate layer group, and then outputting a final training result to the application system.