Neuron Pruning for Deep Neural Network Resource Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require significant computing power and memory, which can be a challenge for vehicles with limited resources, necessitating a method to reduce resource requirements while maintaining accuracy for perception tasks.
Innovation Solution
A processor-implemented method that identifies and classifies layer types in a DNN, evaluates neuron activation scores, and iteratively removes lower-ranked neurons to generate a lean DNN that balances accuracy and resource usage, allowing the DNN to perform perception tasks with reduced memory and computational power.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a trained deep neural network is used for perception tasks in a vehicle, then accuracy and performance are improved, but computing power and memory requirements increase
Solution Approach 1:
The patent extracts and removes redundant or less important neurons from the deep neural network while preserving the essential functionality. This is achieved through iterative analysis of neuron importance metrics and selective removal of neurons that contribute minimally to the overall accuracy, thereby reducing computing power and memory requirements while maintaining acceptable accuracy levels
Solution Approach 2:
The patent changes the parameter of network size by dynamically adjusting the number of neurons based on importance metrics. Through iterative pruning processes, the network parameters are modified to remove redundant connections and neurons, transforming the network from a dense structure to a sparser structure that requires fewer computational resources while maintaining performance
2Use of energy by moving object
If the number of neurons in a DNN is reduced to lower resource requirements, then computing power and memory usage decrease, but accuracy may deteriorate
Solution Approach 1:
The patent implements feedback mechanisms where the accuracy of the pruned network is continuously evaluated against a threshold. After each iteration of neuron removal, the network's performance is assessed, and if accuracy falls below the threshold, the pruning process stops or adjusts its aggressiveness. This feedback loop ensures that resource reduction does not compromise accuracy beyond acceptable limits
Solution Approach 2:
The patent applies partial action by removing only a portion of neurons rather than eliminating them all. The pruning process is controlled and iterative, removing neurons gradually in stages while monitoring accuracy. This approach allows the system to achieve resource reduction without excessive pruning that would harm accuracy, finding an optimal balance point
Data Source
AI summary
A method for reducing the number of neurons in a trained deep neural network (DNN) includes classifying layer types in a plurality of hidden layers; evaluating the accuracy of the DNN using a validation set of data; and generating a layer specific ranking of neurons, wherein the generating includes: analyzing, using the validation set of data for one or more of the plurality of hidden layers, the activation function for each neuron in the analyzed layers to determine an activation score for each neuron; and ranking, on a layer type basis, each neuron in the analyzed layers based on the neuron's activation score to generate a layer specific ranking of neurons. The method further includes removing a number of lower ranked neurons from the DNN that does not result in the DNN after the removal of selected lower ranked neurons to fall outside of an accuracy threshold limit.