Spiking Neural Network Evolutionary Training Circuit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing energy consumption and complexity in training neural networks, particularly for large and complex machine-learning models, pose significant challenges, including high carbon footprints and power requirements, which restrict the scalability and accessibility of machine-learning systems.
Innovation Solution
The use of spiking neural networks (SNNs) with an evolutionary training regime that employs random mutation of network parameters based on error signals generated by local operational circuits, avoiding the need for costly random number generator circuits and promoting energy efficiency and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional neural networks are trained using traditional methods, then training accuracy can be achieved, but energy consumption and computational complexity increase significantly
Solution Approach 1:
The patent replaces conventional mechanical/computational training systems with spiking neural networks that use event-driven computation. Instead of continuous computational processes, the system uses discrete spike events triggered by input signals, fundamentally changing how training computations are performed to reduce energy consumption while maintaining accuracy
Solution Approach 2:
The training process uses periodic error signals generated at specific intervals during network operation. These error signals are used to mutate network parameters in periodic updates, allowing the system to achieve training convergence through rhythmic, energy-efficient updates rather than continuous computation
2Adaptability or versatility
If neural network parameters are mutated using random number generator circuits, then training diversity is improved, but device complexity and energy consumption increase
Solution Approach 1:
The system uses the network's own operational circuits to generate error signals that drive parameter mutation. Instead of requiring separate random number generator circuits, the network's operational circuits serve dual purposes: performing computations and generating the error signals needed for evolutionary training, thereby reducing overall device complexity
Solution Approach 2:
The operational circuits are designed to perform multiple functions: they execute network computations and simultaneously generate error signals for parameter mutation. This multi-functionality eliminates the need for dedicated random number generator circuits, reducing device complexity while maintaining training adaptability
3Reliability
If traditional training methods are used for large neural networks, then model performance can be optimized, but training time and computational resources increase
Solution Approach 1:
The event-driven architecture allows the system to skip unnecessary computations by only processing events that actually occur during network operation. This rushing through of redundant computational steps significantly reduces training time while maintaining model performance optimization
Data Source
AI summary
Example methods, devices and/or circuits to be implemented in a processing device to perform operations based, at least in part, on machine-learning. According to an embodiment, one or more parameters of a neural network node may be altered based, at least in part, on one or more error signals that are based, at least in part, on one or more errors generated by a local operational circuit.