Reward-Modulated STDP Eligibility Trace Hardware Implementation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing reward-modulated spike-timing-dependent plasticity (STDP) in neural networks is expensive due to the requirement of an exponentially decaying eligibility trace for every synapse, which occupies significant silicon area.
Innovation Solution
A neural electrical circuit with a memory system that updates synapse weights and uses a probabilistic switching mechanism in STT-RAM or digital memory to simulate the eligibility trace, reducing area requirements by using a modified eligibility trace that decays to zero after a defined time, allowing for efficient hardware implementation of reward-modulated STDP.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an exponentially decaying eligibility trace is implemented in hardware for every synapse, then the reward-modulated STDP learning rule can be accurately implemented, but the silicon area required becomes very large (hundreds of square microns per synapse)
Solution Approach 1:
The patent creates a simplified copy of the eligibility trace mechanism using a single bit per synapse in STT-RAM memory. Instead of implementing the full exponentially decaying trace in hardware, the system copies the essential functionality (eligibility state) into a minimal memory representation, achieving area efficiency while preserving the core learning rule behavior
Solution Approach 2:
The patent changes the parameter representation from continuous exponentially decaying values to discrete binary states (0 or 1) in STT-RAM. This parameter transformation allows the eligibility trace to be represented with minimal area while maintaining the functional behavior needed for reward-modulated STDP through probabilistic updates
2Duration of action of stationary object
If analog or digital circuits are placed in each synapse to create an exponential time constant, then the eligibility trace can be maintained, but the device complexity and area requirements increase significantly
Solution Approach 1:
The patent extracts the time constant functionality from individual synapse circuits and moves it to a global probabilistic update mechanism. Instead of each synapse having its own exponential decay circuit, the system updates eligibility bits probabilistically at a defined frequency, separating the time constant function from the synapse-level hardware
Solution Approach 2:
The STT-RAM memory structure serves multiple functions: it stores synaptic weights, maintains eligibility traces, and implements the time constant through its probabilistic switching behavior. This multi-functional approach eliminates the need for separate dedicated circuits for each function, reducing overall device complexity
Data Source
AI summary
Certain embodiments of the present disclosure support techniques for simplified hardware implementation of the reward-modulated spike-timing-dependent plasticity (STDP) learning rule in networks of spiking neurons.