Event-Driven Artificial Neuron for Sparse RNN Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network models face high computational costs and resource requirements due to the need for sequential updates of all artificial neurons with each new input, especially in complex tasks like deep learning, which can be resource-intensive and time-consuming.
Innovation Solution
The proposed method focuses on operating artificial neurons based on events rather than continuous updates, using thresholds and bit precision adjustments to reduce computational costs, and employs techniques like delta networks to transform dense matrix-vector multiplications into sparse operations, thereby optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If sequential updates of all artificial neurons are performed with each new input, then the neural network can process complex tasks like deep learning, but the computational costs and resource requirements increase significantly
Solution Approach 1:
The patent extracts and processes only the significant changes (deltas) in input data rather than processing all input data sequentially. By identifying and processing only the differences between consecutive inputs, the system reduces computational workload while maintaining the ability to handle complex tasks.
Solution Approach 2:
Instead of performing full sequential updates on all neurons for every input, the patent applies partial updates only to neurons whose activation changes exceed a threshold. This partial action approach reduces computational costs while preserving the network's versatility in processing complex tasks.
2Reliability
If all artificial neurons are updated with each new input, then the neural network maintains accurate processing capability, but redundant operations increase computational overhead
Solution Approach 1:
The patent implements partial updates by comparing consecutive inputs and only updating neurons when the change exceeds a threshold. This maintains processing accuracy for significant changes while avoiding redundant updates for minor variations, thereby improving computational efficiency.
Solution Approach 2:
The patent changes the parameter of update frequency from constant (all neurons every input) to variable (only when delta exceeds threshold). This dynamic parameter adjustment maintains reliability by updating when necessary while improving productivity by skipping unnecessary updates.
3Measurement precision
If dense matrix-vector multiplications are used in neural network operations, then comprehensive computations are performed, but resource usage increases
Solution Approach 1:
The patent extracts only the significant delta components from dense matrix-vector multiplications. By identifying and processing only the non-zero or significant elements in the delta matrices, the system maintains computation completeness for important data while reducing resource usage by excluding redundant zero or minor elements.
Data Source
AI summary
A method for operating an artificial neuron and an apparatus for performing the method are provided. The artificial neuron may calculate a change amount of an activation based on an input signal received via an input synapse, determine whether an event occurs in response to the calculated change amount of the activation, and transmit, to an output synapse, an output signal that corresponds to the event in response to an occurrence of the event.