Event Driven Neural Network Time Hopping Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural networks, particularly spiking neural networks, face significant energy consumption due to frequent memory accesses for updating neural states, even during sparse activity, where many updates perform little computation, leading to inefficiencies in memory access and energy usage.
Innovation Solution
The implementation of an event-driven and time-hopping neural network approach, where neural unit state changes are computed only on active time-steps, aggregating contributions from idle time-steps to reduce memory accesses and minimize energy consumption by skipping updates until the next active time-step.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks perform frequent memory accesses to update neural states at each time-step, then the accuracy of membrane potential tracking is maintained, but energy consumption increases significantly
Solution Approach 1:
The patent applies the skipping principle by allowing the neural network to skip memory access operations during idle time-steps when no spikes are present. Instead of updating membrane potentials at every time-step, the system calculates and applies the cumulative effect of idle periods in bulk, thereby skipping redundant memory accesses while preserving the accuracy of membrane potential tracking when updates do occur.
Solution Approach 2:
The patent implements preliminary action by pre-calculating the cumulative effect of idle time-steps before performing the next memory access. The system computes the total change in membrane potential over multiple idle periods in advance, then applies this pre-computed value in a single update operation, reducing the frequency of memory accesses while maintaining accuracy.
2Measurement precision
If neural networks update neural states at every time-step, then computational accuracy is maintained, but productivity decreases due to redundant computations during sparse activity
Solution Approach 1:
The patent applies skipping by enabling the neural network to bypass redundant computation cycles during idle time-steps. When no spikes are detected, the system skips the detailed membrane potential update computations and directly calculates the cumulative effect, thereby improving processing efficiency without sacrificing the accuracy of the final result.
Solution Approach 2:
The patent implements partial action by performing computations only when necessary - specifically, only when spikes are present. During idle time-steps, the system performs minimal computation (calculating cumulative effects) rather than full membrane potential updates, thereby improving productivity while maintaining the accuracy required for proper neural state tracking.
3Reliability
If neural networks perform memory accesses during sparse activity, then state updates are performed, but energy usage increases due to unnecessary operations
Solution Approach 1:
The patent applies the skipping principle by detecting sparse activity conditions and skipping memory access operations during idle time-steps. The system monitors for the presence of spikes and only performs memory accesses when necessary, thereby reducing energy waste while ensuring that state updates remain accurate when they do occur.
Solution Approach 2:
The patent implements periodic action by updating neural states at irregular intervals based on the presence of spikes rather than at fixed time-steps. The system performs memory accesses periodically only when activity is detected, adapting the update frequency to the actual neural activity pattern, thereby reducing energy consumption during sparse activity while maintaining reliability.
Data Source
AI summary
In one embodiment, a processor is to store a membrane potential of a neural unit of a neural network; and calculate, at a particular time-step of the neural network, a change to the membrane potential of the neural unit occurring over multiple time-steps that have elapsed since the last time-step at which the membrane potential was updated, wherein each of the multiple time-steps that have elapsed since the last time-step is associated with at least one input to the neural unit that affects the membrane potential of the neural unit.