Neural Network Architecture Concurrent Learning Antidromic Spikes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spike-based neural network systems lack an efficient learning strategy, particularly for online learning, which affects throughput, latency, and introduces complexity due to the need for arbitration between inference and learning phases.
Innovation Solution
A neural network processing system with multiple layers, featuring bidirectional Synaptic Network Channels for concurrent transmission of weighted sums and inputs, unidirectional Signal Reshaping units for inference and learning, and Hybrid Couplers to connect these components, allowing for shared weights and efficient weight updates without bus arbitration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If arbitration is implemented between inference and learning phases, then learning can be supported, but throughput and latency are affected and system complexity increases
Solution Approach 1:
The patent merges inference and learning operations into a single unified neural network system that operates concurrently without arbitration. The bidirectional Synaptic Network Channel integrates both forward propagation (inference) and backward propagation (learning) pathways, allowing simultaneous execution of both functions through the same hardware infrastructure, thereby eliminating arbitration overhead and maintaining high throughput.
Solution Approach 2:
The Synaptic Network Channel is designed as a universal communication pathway that handles multiple functions: forward signal transmission for inference, backward signal transmission for learning, and weight updates. This multi-functional design eliminates the need for separate dedicated channels for inference and learning, reducing system complexity while supporting both operations concurrently.
2Adaptability or versatility
If arbitration is implemented between inference and learning phases, then learning can be supported, but system complexity increases
Solution Approach 1:
The patent merges inference and learning operations into a single unified neural network system that operates concurrently without arbitration. The bidirectional Synaptic Network Channel integrates both forward propagation (inference) and backward propagation (learning) pathways, allowing simultaneous execution of both functions through the same hardware infrastructure, thereby eliminating arbitration overhead and maintaining high throughput.
Solution Approach 2:
The Synaptic Network Channel is designed as a universal communication pathway that handles multiple functions: forward signal transmission for inference, backward signal transmission for learning, and weight updates. This multi-functional design eliminates the need for separate dedicated channels for inference and learning, reducing system complexity while supporting both operations concurrently.
3Adaptability or versatility
If separate inference and learning phases are used, then learning can be performed, but temporal jitter increases and real-time operation is affected
Solution Approach 1:
The patent implements continuous concurrent operation of inference and learning through the bidirectional SNC architecture. Forward signals for inference and backward signals for learning propagate simultaneously through the network without interruption or phase switching, eliminating temporal jitter associated with phased operations and enabling real-time adaptive learning.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables higher throughput, lower latency, and simpler architecture for concurrent inference and learning, supporting online and real-time operations with reduced energy consumption and temporal jitter.
Implementation Method 1
concurrently transmitting weighted sums, yF(t)'s and xB(t)'s, as an elastic wave superposition of inputs, xF(t)'s and yB(t)'s
Data Source
AI summary
A neural network processing system having multiple layers is provided. Each layer includes a bidirectional Synaptic Network Channel (SNC) for concurrently transmitting weighted sums, yF(t)'s and xB(t)'s, as an elastic wave superposition of inputs, xF(t)'s and yB(t)'s, respectively. Each input is multiplied and added with corresponding weights w's encoded in variable splitters and combiners in forward and backward directions, respectively. Each layer includes unidirectional Signal Reshaping (SR) units, I's and L's for inference and learning, respectively, by generating inputs for a following layer in forward and backward directions from a current layer's weighted sums yF(t)'s and xB(t)'s, respectively. Each layer includes a Hybrid Coupler (HC) to connect the bidirectional SNC and the unidirectional SR units. Each layer includes a weight update unit to calculate each weight difference using an input yBi(t) or a weighted sum yFi(t) and an input xFj(t) to update a weight wij for a current layer.