Spiking Neural Network Online Learning via Hebbian Weight Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network learning methods face challenges in understanding and modifying network structures due to their distributed information storage, making it difficult to integrate new structures effectively and avoid information redundancy, especially in online learning environments.
Innovation Solution
A system implementing biologically-inspired mechanisms such as Hebbian learning for weight adaptation, neuron competition, and synaptic structure copying in multi-layered spiking neural networks, allowing for continuous online learning and adaptation of network architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural networks use distributed information storage and massively parallel processing, then robustness to noise, generalization to novel stimuli, low power consumption, and high information throughput are improved, but it becomes difficult to understand what information the network has learned, where it is stored, and what computations are being performed
Solution Approach 1:
The patent segments the neural network into distinct functional components: input layer, hidden layers with specific neuron types (e.g., orientation-selective neurons), and output layers. Each segment performs specific computations with interpretable functions, allowing researchers to trace which segments contribute to specific learned representations while maintaining the distributed processing benefits
2Adaptability or versatility
If techniques are used to modify neural network architecture online, then adaptability is improved, but the useful but confounding approaches to representing and processing information make it difficult to know how to modify network structure constructively without destroying previously learned information
Solution Approach 1:
The patent establishes preliminary structural frameworks and constraints during network design (such as fixed architectural hierarchies, predefined connection patterns, and initialization schemes) that guide subsequent online learning. These pre-established structures ensure that future modifications and learnings build upon rather than destroy existing knowledge, enabling constructive architectural evolution
Solution Approach 2:
The patent implements feedback mechanisms where network performance and internal state information are continuously monitored and used to guide architectural modifications. This feedback loop ensures that changes to network structure are made based on actual learning needs and performance requirements, preventing destructive modifications while maintaining adaptability
3Productivity
If evolutionary computation methods are used for neural network growth, then population-based optimization is achieved, but the methods lack principled approaches to increasing the likelihood that newly integrated network structure will become useful and lack means for competition among neurons to reduce information redundancy
Solution Approach 1:
The patent systematically changes key parameters during network growth and learning, including learning rates, regularization strengths, and architectural hyperparameters. These parameter adjustments are made based on training progress and performance metrics, enabling the network to efficiently integrate new structures while automatically reducing redundancy through parameter optimization
Data Source
AI summary
Described is a system for adapting neural networks. The system receives inputs to be learned by a multi-layered spiking neural network. A first mechanism is implemented to adapt weights on the connections via competition among neurons using Hebbian learning. Activity levels of the neurons are stabilized to allow the multi-layered spiking neural network to learn the inputs. A second mechanism is implemented to increase a learning rate of a neuron over time using Hebbian learning. A third mechanism is implemented, wherein newly created neurons, representing new inputs, copy at least one synaptic structure of older neurons in the multi-layered spiking neural network. The mechanisms are used for continuous, online learning of the inputs to the multi-layered spiking neural network. An autonomous system, such as an autonomous vehicle, can use the learned inputs to learn from its environment and perform tasks, such as classification and prediction.