Self-tuning DNN Compression for Resource-Limited Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale Deep Neural Networks are not deployable on resource-limited devices like mobile phones due to high computational intensity and memory requirements, and learning from scratch is not feasible with limited datasets, preventing end-users from developing customized models.
Innovation Solution
A self-tuning model compression methodology that uses a pre-trained DNN model and a dataset to compress the model into a smaller size with acceptable computational complexity and accuracy loss, applicable for end-user applications, involving steps like analyzing sparsity, pruning, and quantizing the network to remove redundancies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large-scale pre-trained DNN model is used to achieve high accuracy in computer vision and speech recognition tasks, then the model accuracy is improved, but the computational complexity and memory requirements increase, making it undeployable on resource-limited devices
Solution Approach 1:
The patent extracts and removes redundant neurons and connections from the pre-trained DNN model through pruning techniques. Specifically, neurons with minimal contribution to the final output are identified and removed, along with their associated weights and biases. This extraction process reduces the model size and computational complexity while preserving the essential functionality and accuracy for the target application.
Solution Approach 2:
The patent applies different compression strategies to different parts of the network based on their importance. Critical layers and neurons that contribute most to accuracy are preserved with higher fidelity, while less important portions are compressed more aggressively. This local differentiation allows the model to maintain high accuracy in critical regions while reducing overall complexity.
2Measurement precision
If a large-scale pre-trained DNN model is used to achieve high accuracy, then the model accuracy is improved, but the memory storage requirements increase, making it undeployable on devices with limited memory
Solution Approach 1:
The patent removes redundant parameters (weights and biases) from the model through systematic pruning. By identifying and eliminating neurons that contribute minimally to the output, the total number of parameters is reduced, directly decreasing the memory storage requirements while maintaining acceptable accuracy levels.
Solution Approach 2:
The patent implements dynamic pruning where the model structure is adaptively adjusted based on performance metrics. Less important neurons are pruned more aggressively, while critical neurons are preserved. This dynamic approach optimizes the balance between model size and accuracy, reducing memory storage requirements effectively.
3Adaptability or versatility
If learning from scratch is attempted with a limited dataset to create customized models, then model customization is achieved, but the learning process is not feasible due to insufficient data
Solution Approach 1:
The patent utilizes a pre-trained model that has already learned general features from large datasets as a starting point. This preliminary training phase is performed offline using abundant data, and the pre-trained model is then fine-tuned or pruned using the user's limited dataset. This approach transfers knowledge from the pre-trained model to the customized model, making learning feasible with limited data while achieving customization.
Solution Approach 2:
The self-tuning mechanism allows the model to automatically adapt to the specific application domain using the limited available dataset. Through automated pruning and refinement processes, the model learns to prioritize relevant features for the specific task without requiring extensive manual intervention or large amounts of domain-specific training data.
Data Source
AI summary
A self-tuning model compression methodology for reconfiguring a Deep Neural Network includes: receiving a DNN model and a data set, wherein the DNN includes an input layer, at least one hidden layer and an output layer, and said at least one hidden layer and the output layer of the DNN model includes a plurality of neurons; compressing the DNN model into a reconfigured model according to the data set, wherein the reconfigured model includes an input layer, at least one hidden layer and an output layer, and said at least one hidden layer and the output layer of the reconfigured model includes a plurality of neurons, and a size of the reconfigured model is smaller than a size of the DNN model; and executing the reconfigured model on a user terminal for an end-user application.