Spiking Neural Network Early Determination Training Accelerator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spiking neural networks face high energy consumption and prolonged training times due to the repetitive processes of inference, backpropagation, and weight update during training, especially as the number of training images increases, with a significant proportion of images contributing little to the training process.
Innovation Solution
The method involves timestep splitting, where the training process is accelerated by measuring membrane potentials, calculating a threshold value based on these potentials, and early terminating training for images with no contribution, thereby reducing computational load and time without compromising accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If training is performed on all training images using complete timesteps, then training accuracy is maintained, but training energy consumption and training time increase significantly
Solution Approach 1:
The training process is segmented into multiple timesteps, and the patent applies early termination at intermediate timesteps for images that do not contribute to training. This segmentation allows the system to process training images in discrete time intervals and identify non-contributing images before completing the full training cycle, thereby reducing energy consumption while maintaining accuracy for contributing images.
Solution Approach 2:
The patent applies partial action by performing only a portion of the training process (early termination at intermediate timesteps) for images that do not contribute to training. Instead of applying the complete training process to all images, the system selectively applies partial training to non-contributing images, reducing overall energy consumption while maintaining training accuracy for contributing images.
2Measurement precision
If training is performed on all training images using complete timesteps, then training accuracy is maintained, but training time increases significantly
Solution Approach 1:
The training process is divided into multiple timesteps, enabling the system to monitor membrane potentials at intermediate points. This segmentation allows early termination for non-contributing images before the complete training cycle is finished, significantly reducing training time while preserving accuracy for contributing images that complete the full training process.
Solution Approach 2:
The patent implements partial action by applying only a portion of the training process to non-contributing images through early termination. This approach reduces the time spent on images that would not contribute to training accuracy, while maintaining complete training for contributing images, thereby optimizing overall training time.
3Use of energy by moving object
If timestep splitting and early termination are applied, then training energy and time are reduced, but system complexity increases due to membrane potential measurement and threshold calculation
Solution Approach 1:
The system uses self-service by leveraging the inherent membrane potential measurements that occur naturally during the spiking neural network training process. Instead of requiring external monitoring systems, the patent utilizes the network's own internal state (membrane potentials) to determine early termination conditions, reducing the need for additional complex monitoring infrastructure.
Solution Approach 2:
The patent implements feedback mechanisms by continuously monitoring membrane potentials during training and using this information to dynamically adjust the training process through early termination. The threshold calculation based on membrane potential differences provides a feedback loop that guides the early termination decision, managing system complexity through intelligent use of existing network states.
4Loss of time
If timestep splitting is applied with early termination, then training time is reduced, but the complexity of determining training contribution increases
Solution Approach 1:
The patent replaces complex analytical methods for determining training contribution with a biological-inspired mechanism based on membrane potential differences. Instead of using traditional computational metrics to assess training contribution, the system leverages the natural membrane potential dynamics of spiking neurons, substituting mechanical/computational analysis with a biologically-inspired measurement approach that is inherently integrated into the network's operation.
Data Source
AI summary
Disclosed are a method for accelerating early determination training. The method for accelerating early determination training includes a timestep splitting operation of splitting a timestep, a membrane potential measuring operation of measuring first and second membrane potentials for each splitted timestep during a current training process, a threshold value calculation operation of calculating a threshold value to be used in a subsequent training process based on the first and second membrane potentials, and when a difference between the first and second membrane potentials in the splitted timestep is greater than the threshold value, an early training termination operation of determining that the image does not have the training contribution and terminating training at the splitted timestep.