Neural Network Pruner Reducing Memory Footprint
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training and inference with neural networks require significant memory, time, and computing resources, limiting their efficiency and accuracy, especially in real-time applications like autonomous vehicles.
Innovation Solution
A neural network pruner that reduces the size of neural networks by identifying and removing zero-weighted neurons and propagating bias terms, allowing for efficient inference without retraining, using techniques like scatter operations to maintain model accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks are used for training and inference, then model accuracy and intelligence are improved, but memory usage and computation time increase significantly
Solution Approach 1:
The patent extracts and removes zero-weighted neurons from the neural network. By identifying neurons with all-zero weights and removing them entirely, the system reduces memory footprint while preserving inference accuracy, as these neurons contribute nothing to the computation
Solution Approach 2:
The patent discards zero-weighted neurons that provide no value to the inference process. By propagating bias terms to compensate for removed neurons, the system recovers functional equivalence while reducing resource consumption
2Measurement precision
If neural networks are used for training and inference, then model capabilities are improved, but computation time increases significantly
Solution Approach 1:
By extracting and removing zero-weighted neurons, the patent reduces the number of computational operations required during inference. This directly decreases computation time while maintaining the accuracy of remaining non-zero neurons
Solution Approach 2:
The patent changes the network parameters by propagating bias terms from pruned layers to subsequent layers. This parameter adjustment compensates for removed neurons and maintains inference accuracy without requiring retraining, thus reducing computation time
3Quantity of substance
If neural networks are pruned by removing zero-weighted neurons, then memory footprint is reduced, but model complexity increases
Solution Approach 1:
The patent segments the neural network into layers and processes pruning layer by layer. By identifying zero-weighted neurons in each layer and propagating their bias terms to subsequent layers, the system manages complexity through systematic segmentation rather than holistic reprocessing
Solution Approach 2:
The patent performs preliminary identification and marking of zero-weighted neurons before actual removal. By pre-computing which neurons to remove and propagating their bias terms in advance, the system avoids complex real-time computations during the pruning execution
4Measurement precision
If bias terms are propagated to compensate for pruned neurons, then inference accuracy is maintained, but computational overhead increases
Solution Approach 1:
The patent merges the bias term propagation with the existing neural network forward propagation process. By incorporating the propagated bias terms into the standard computation flow, the system maintains inference accuracy without adding separate computational passes or significant energy overhead
Data Source
AI summary
Apparatuses, systems, and techniques to reduce a size of neural networks. In at least one embodiment, a size of a neural network is reduced by at least removing one or more neurons of the neural network and adjusting one or more layers of the neural network to compensate for the removed one or more neurons.