Spiking Neural Network Memory Architecture for Scalable Spike Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Scalable and efficient implementation of Spiking Neural Networks (SNNs) is challenging due to the high demand for computing resources to process spikes quickly, especially in modeling large biological neural networks like the human brain, where 86 billion neurons and 1700 synapses per neuron require advanced memory architectures for effective processing and routing of spike messages.
Innovation Solution
A memory architecture with special purpose memory devices, or 'nodes,' that implement a pipelined architecture to handle different stages of spike processing, using routers and port translators to route spike messages efficiently, and multicasting to reduce network traffic, allowing for scalable and complex SNN networks with embedded routers and optimized memory structures like ReRAM or 3D Cross Point memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If SNN scales to model large biological neural networks (e.g., human brain with 86 billion neurons), then the model accuracy and complexity increase, but the demand for computing resources and processing speed increases significantly
Solution Approach 1:
The patent segments the neural network into multiple processing stages organized as pipelines. Each stage handles specific spike processing tasks (e.g., receiving spikes, calculating synaptic currents, determining membrane potentials, generating output spikes). This segmentation allows parallel processing across stages while maintaining the complex computational requirements of large-scale SNNs.
Solution Approach 2:
The patent implements preliminary action by pre-calculating and storing synaptic connection information in lookup tables before spike processing begins. Synaptic weights, connection topologies, and other parameters are prepared in advance, enabling faster real-time spike processing without compromising the complexity of the neural network model.
2Ease of manufacture
If traditional memory architectures are used to process spikes, then implementation is simpler, but the demand for computing resources increases and scalability is limited
Solution Approach 1:
The memory architecture is segmented into multiple specialized memory sections, each optimized for specific data types (e.g., synaptic weights, neuron parameters, spike queues). This segmentation enables the system to scale by adding more memory sections while maintaining implementation simplicity through standardized interfaces and modular design.
Solution Approach 2:
The patent transitions from traditional von Neumann architecture to a memory-centric architecture where computation occurs within or near memory structures. This dimensional change in architectural organization enables scalable SNN implementation by exploiting the parallel access capabilities of modern memory technologies like ReRAM and 3D Cross Point memory.
3Ease of operation
If all spike messages are routed through standard network paths, then routing is simpler, but network traffic increases and processing efficiency decreases
Solution Approach 1:
The patent applies preliminary action by pre-computing and caching routing information for spike messages. Routing decisions are made based on pre-established rules and lookup tables that map source-destination pairs to optimal paths. This eliminates the need for complex real-time routing computations while reducing network traffic through intelligent path selection and spike aggregation.
Data Source
AI summary
The present disclosure is directed to routing of data in a spiking neural network (SNN) that performs in-memory operations. To model a computer-implemented SNN after a biological neural network, the architecture in the present disclosure involves different memory sections for storing inbound spike messages, synaptic connection data, and synaptic connection parameters. Embodiments are directed to routing spike messages through various router-based topologies. For example, spike messages may be multicasted to target routers using address tables.