Spiking Neural Network Variable Delay Path Weight Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing spiking neural network apparatus face efficiency challenges due to the need to retrieve weights from storage, consuming power and computational resources, and the use of Analog-to-Digital Converter (ADC) and Digital-to-Analog Converter (DAC) hardware blocks, which reduces efficiency and increases costs. Additionally, training neural networks on these apparatus is challenging due to the non-differentiable nature of spikes.
Innovation Solution
A spiking neural network apparatus with a plurality of neurons, each having variable delay paths, logic components, timing components, and accumulate components. The method involves presenting the apparatus with test input data, selecting a neuron, defining sleeper neurons, determining activation states, and adjusting delay values to train the network effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If weights are encoded in SRAM storage to generate values at neurons, then the neural network can perform computation, but power and computational resources are consumed due to the need to retrieve network weights from storage
Solution Approach 1:
The patent extracts the weight storage function from separate SRAM memory and integrates it directly into the neuron circuit structure. Weights are encoded in the resistance values of delay path components (such as variable resistors or synapses) that are physically embedded within the neuron, eliminating the need for separate weight retrieval operations and reducing both power consumption and computational overhead.
Solution Approach 2:
The patent merges the weight storage function with the computation function by integrating resistive memory elements (used for weight encoding) directly into the neuron circuit. This combination allows weights to be stored and utilized simultaneously without separate retrieval steps, improving both energy efficiency and computational speed by unifying storage and processing in a single integrated structure.
2Productivity
If ADC and DAC hardware blocks are used to convert spike values and perform sampling operations, then the neural network can process information, but efficiency is reduced and financial costs increase
Solution Approach 1:
The patent removes ADC and DAC hardware blocks from the spiking neural network architecture. Instead of converting spikes to digital values and back, the system processes spikes directly in the analog domain using continuous-time integration and comparison circuits, eliminating the need for these conversion blocks and reducing both complexity and processing overhead.
Solution Approach 2:
The patent replaces the mechanical/digital conversion system (ADC/DAC blocks) with an analog processing system. Spikes are processed using continuous-time analog circuits that perform integration, thresholding, and summation directly on the analog spike signals, substituting the discrete digital conversion mechanism with a continuous analog processing approach that is more efficient and less complex.
3Adaptability or versatility
If conventional training algorithms such as back-propagation are used, then training can be performed on differentiable functions, but they are not applicable to the non-differentiable spikes produced by spiking neural networks
Solution Approach 1:
The patent changes the parameter representation from discrete spike timing to continuous delay values that control the timing of spike propagation. By representing weights as continuous delay parameters in the delay paths, the system enables gradient-based optimization methods to adjust these delay values, making the non-differentiable spike-based system compatible with conventional training algorithms through parameter transformation.
Data Source
AI summary
A spiking neural network is described that comprises a plurality of neurons in a first layer connected to at least one neuron in a second layer, each neuron in the first layer being connected to the at least one neuron in the second layer via a respective variable delay path. The at least one neuron in the second layer comprises one or more logic components configured to generate an output signal in dependence upon signals received along the variable delay paths from the plurality of neurons in the first layer. A timing component is configured to determine a timing value in response to receiving the output signal from the one or more logic components, and an accumulate component is configured to accumulate a value based timing values from the timing component. A neuron fires in a case that a value accumulated at the accumulate component reaches a threshold value.