Spiking Neural Network Device With Independent Synaptic Depressor
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spiking neural networks struggle to learn information with low spike density, such as blank data, due to unchanged synaptic weights, leading to failure in recognizing patterns like handwritten digits, and existing solutions require doubling hardware and energy consumption.
Innovation Solution
A spiking neural network device with a synaptic depressor that depresses weights independently of input timing and firing timing, allowing for learning of blank data without synaptic normalization, using resistive random-access memories to implement synaptic elements and potentiating/depressing weights based on schedules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If STDP learning rules are used in spiking neural networks, then synaptic weights are updated based on input timing and firing timing, but the network fails to learn information with low spike density such as blank data
Solution Approach 1:
The synaptic depressor performs depression operations in advance or independently of the STDP timing conditions, proactively reducing synaptic weights for neurons that should represent blank or low-spike-density regions before the network is exposed to the actual input data. This preliminary action ensures that the network can learn both sparse and dense information patterns effectively.
Solution Approach 2:
The synaptic depressor acts as an intermediary mechanism between the STDP learning rule and the final synaptic weight configuration. It mediates the weight updates by introducing an additional depression component that operates independently from the timing-dependent potentiation, allowing the network to learn low spike density information that STDP alone cannot capture.
2Adaptability or versatility
If synaptic normalization is used to enable learning of blank data, then the network can learn low spike density information, but hardware implementation becomes complex and energy consumption increases
Solution Approach 1:
The learning mechanism is segmented into two independent components: the STDP-based synaptic potentiator that handles timing-dependent weight updates, and the synaptic depressor that handles independent depression operations. This segmentation allows each component to be implemented with simple hardware logic, avoiding the complex global normalization operations while achieving the same learning capability for blank data.
Solution Approach 2:
The synaptic depressor autonomously performs depression operations based on predetermined schedules or simple triggering conditions, without requiring complex global computations or external control mechanisms. Each synapse independently undergoes depression based on its own activity patterns and the predetermined schedule, eliminating the need for centralized normalization hardware.
3Productivity
If conventional AI learning processes are implemented using GPUs, then large amounts of data can be processed, but energy consumption becomes extremely high
Solution Approach 1:
The patent replaces the conventional von Neumann architecture with GPUs that perform repeated memory access and computation cycles with a direct neuromorphic hardware architecture. In this architecture, synaptic weights are physically embodied in hardware elements (such as conductance values in crossbar arrays), and learning occurs through direct physical modification of these elements via voltage pulses, eliminating the need for repeated data loading and computation iterations.
Solution Approach 2:
The learning process uses periodic spike events rather than continuous analog computations. Neurons fire discrete spike voltages at specific moments, and synaptic weights are updated only at these discrete events through pulse-based modification. This periodic, event-driven operation dramatically reduces computational overhead and energy consumption compared to continuous processing in conventional systems.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables effective learning of low spike density information without increasing device size or energy consumption, achieving recognition rates comparable to synaptic normalization methods while being suitable for hardware implementation.
Implementation Method 1
synaptic elements (120) having weights (wji) and internal variables (qji), respectively
Data Source
AI summary
A spiking neural network device according to an embodiment includes a synaptic element, a neuron circuit, a synaptic potentiator, and a synaptic depressor. The synaptic element has a variable weight. The neuron circuit inputs a spike voltage having a magnitude adjusted in accordance with the weight of the synaptic element via the synaptic element, and fires when a predetermined condition is satisfied. The synaptic potentiator performs a potentiating operation for potentiating the weight of the synaptic element depending on input timing of the spike voltage and firing timing of the neuron circuit. The synaptic depressor performs a depression operation for depressing the weight of the synaptic element in accordance with a schedule independent from the input timing of the spike voltage and the firing timing of the neuron circuit.