Resetting Dormant Neurons in Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks often suffer from dormant neurons during training, leading to reduced capacity and stagnation in performance improvement, which can result in under-utilization and increased computational costs, especially when using reinforcement learning with high replay ratios.
Innovation Solution
A system that periodically checks for dormant neurons and resets them by modifying their incoming and outgoing weights, allowing them to contribute actively to the network output, thereby maintaining network capacity and improving training efficiency without sacrificing previously learned knowledge.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If conventional training techniques are used with high replay ratios in reinforcement learning, then data efficiency is improved, but training quality deteriorates due to increased fraction of dormant neurons
Solution Approach 1:
The patent changes the state parameters of dormant neurons by resetting their incoming and outgoing weights to non-zero values, transforming them from dormant to active state. This parameter change allows the network to maintain high replay ratios without the degradation of training quality that would otherwise occur due to neuron dormancy.
Solution Approach 2:
The patent implements periodic detection and resetting of dormant neurons during the training process. By periodically identifying neurons with near-zero activation expectations and resetting their weights, the system maintains training quality throughout extended training periods with high replay ratios, preventing the accumulation of dormant neurons.
2Reliability
If larger neural networks are used to maintain capacity during training, then performance is improved, but computational costs increase
Solution Approach 1:
The patent enables the neural network to self-diagnose and self-repair by automatically detecting dormant neurons through monitoring activation expectations and resetting them without external intervention. This self-service mechanism maintains network capacity and performance without requiring larger network architectures or additional computational resources.
3Measurement precision
If training continues for extended periods to achieve desired performance, then model accuracy is improved, but network capacity is reduced due to accumulation of dormant neurons
Solution Approach 1:
The patent implements a feedback mechanism that continuously monitors the activation expectations of neurons during training. When neurons are detected to be dormant (near-zero activation expectations), the system provides feedback by resetting their weights, creating a closed-loop control system that maintains network capacity throughout extended training periods.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network and, during the training, resetting neurons that have been classified as being dormant.