Delay Spiking Neural Network Eliminates Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks, particularly deep learning networks, face challenges in power consumption and hardware complexity, especially in embedded processors, due to the costly nature of multiplication operations and lock-step execution requirements, which limits their use in mobile and off-the-grid devices.
Innovation Solution
The development of delay spiking neural networks (DSNNs) that utilize binary spikes with varying timing, eliminating the need for multiplication circuitry by using delay circuitry to accumulate inputs over time, with each neuron generating an output signal only when a predetermined number of delayed inputs are accumulated within a specific period, and implementing these networks across multiple semiconductor dies or nodes for distributed processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional neural networks use multiplication to accumulate and combine data at neurons, then computational accuracy is improved, but power consumption and hardware area increase significantly
Solution Approach 1:
The patent extracts and removes the multiplication operation from the neural network computation. Instead of performing multiplication at each neuron to accumulate and combine data, the system uses simple addition of spike timing information. This extraction of the costly multiplication operation directly reduces power consumption and hardware complexity while maintaining computational functionality through alternative spike-based mechanisms.
Solution Approach 2:
The patent substitutes the traditional multiplication-based mechanical computation with a spike timing-based temporal encoding mechanism. Rather than multiplying numerical values, the system uses the timing of binary spikes to encode information, replacing complex arithmetic operations with simpler temporal pattern recognition that achieves similar computational accuracy with lower power consumption.
2Reliability
If traditional neural networks perform lock-step execution to ensure proper timing and routing, then computational reliability is improved, but device complexity and synchronization overhead increase
Solution Approach 1:
The patent transitions from static lock-step execution to dynamic asynchronous spike-based communication. Instead of forcing all neurons to execute in synchronized steps, the system allows spikes to propagate dynamically through the network at different times based on actual computation needs. This dynamic approach maintains reliability through event-driven synchronization while reducing the complexity of global timing control.
Solution Approach 2:
The patent replaces continuous lock-step periodic execution with event-triggered periodic spike generation. Rather than executing computations at fixed time intervals regardless of data readiness, the system generates spikes periodically only when data is ready and computation is complete, maintaining reliability through proper event sequencing while reducing synchronization overhead from unnecessary periodic executions.
3Measurement precision
If neural networks are implemented with full multiplication circuitry for accurate computation, then computational precision is improved, but hardware area and manufacturing cost increase
Solution Approach 1:
The patent extracts and removes multiplication circuitry from the hardware implementation. By eliminating the need for multipliers at each neuron, the hardware area required for computation is dramatically reduced. The system achieves comparable computational precision through addition-based spike accumulation and timing-based encoding, freeing up hardware area for other functionalities or larger network implementations.
Solution Approach 2:
The patent changes the computational parameters from numerical multiplication to temporal spike encoding. Instead of using multipliers to compute weighted sums of numerical inputs, the system uses the timing parameters of binary spikes to encode input strengths and uses addition to accumulate these temporal signals. This parameter change enables accurate computation with significantly reduced hardware area requirements.
Data Source
AI summary
A delay spiking neural network (DSNN) may include a plurality of neurons arranged in a plurality of layers, with neurons spiking based on accumulation of delayed inputs. A first to spike neuron in an output layer may provide a result of the DSNN.