Sparse Neural Network Weight Indexing for Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning models require specialized hardware for training and prediction due to their large number of weights, making it impractical to run them on most endpoint devices without dedicated accelerator hardware.
Innovation Solution
The method involves generating a sparse neural network by pruning weak synapse connections during the training phase or evolving the network using evolutionary computation, along with a new compact data representation that independently indexes each weight to eliminate the need to store pruned synapse weights.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional neural networks with millions to billions of weights are used, then prediction accuracy and model capability are improved, but memory requirements and computational resources increase significantly
Solution Approach 1:
The patent extracts and removes unnecessary or weak synapse connections from the neural network, retaining only the most important connections. This pruning approach eliminates redundant weights while preserving the essential information needed for accurate predictions, thereby reducing memory requirements without significantly compromising model capability
Solution Approach 2:
The patent applies different treatment to different parts of the neural network by identifying and retaining only the locally important synapse connections. Instead of uniformly treating all weights, the method selectively preserves connections with higher importance scores while eliminating others, creating a non-uniform, optimized network structure that uses less memory
2Adaptability or versatility
If traditional neural networks with billions of weights are used, then model capability is improved, but processing speed decreases due to computational complexity
Solution Approach 1:
By extracting and removing unnecessary computational operations associated with pruned synapse connections, the patent reduces the total number of computations required during inference. This eliminates redundant mathematical operations while preserving the essential computational pathways, thereby improving processing speed without significantly degrading model capability
Solution Approach 2:
The patent segments the computational graph by separating important synapse connections from unimportant ones. This segmentation allows the system to process only the essential connections during inference, dividing the computational workload into necessary and optional parts, thereby improving processing efficiency
3Quantity of substance
If neural networks are pruned to reduce weights, then memory usage and computational requirements are reduced, but network accuracy may deteriorate
Solution Approach 1:
The patent performs preliminary identification and scoring of synapse connection importance before pruning. By calculating importance scores for all connections in advance and selecting which connections to retain based on these scores, the system ensures that the most critical connections are preserved, thereby maintaining network accuracy while achieving memory reduction
Solution Approach 2:
The patent uses feedback from training data and performance metrics to guide the pruning process. Importance scores are calculated based on feedback from the training process, allowing the system to make informed decisions about which connections to retain, thereby preserving accuracy while reducing memory usage
Data Source
AI summary
A device, system, and method is provided for storing a sparse neural network. A plurality of weights of the sparse neural network may be obtained. Each weight may represent a unique connection between a pair of a plurality of artificial neurons in different layers of a plurality of neuron layers. A minority of pairs of neurons in adjacent neuron layers are connected in the sparse neural network. Each of the plurality of weights of the sparse neural network may be stored with an association to a unique index. The unique index may uniquely identify a pair of artificial neurons that have a connection represented by the weight. Only non-zero weights may be stored that represent connections between pairs of neurons (and zero weights may not be stored that represent no connections between pairs of neurons).