Context-Specific Neural Network Model Generation for Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks are computationally intensive and challenging to execute efficiently in resource-constrained environments, such as edge devices, due to their large size and high computational requirements, which existing techniques often fail to address effectively.
Innovation Solution
A system that generates and executes a context-specific neural network model based on target runtime parameters, including desired operating conditions, resource utilization, and accuracy, by selectively masking and pruning the network, and deploying it on appropriate computing nodes closer to data sources, using techniques like knowledge distillation and quantization to reduce model size and computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If deep neural networks are executed in resource-constrained environments, then computational intensity and resource consumption increase, but device capabilities and battery life are limited
Solution Approach 1:
The system dynamically adapts the neural network model based on runtime conditions by selectively executing different model versions (e.g., full model vs. pruned model) depending on available computational resources and power constraints, allowing the system to adjust its computational intensity dynamically rather than statically
Solution Approach 2:
The system extracts and executes only the essential portions of the neural network model by applying pruning techniques to remove redundant neurons and connections, thereby reducing the computational workload and power consumption while maintaining core functionality
2Productivity
If existing techniques selectively execute portions of neural network models, then resource utilization improves, but manipulation complexity increases
Solution Approach 1:
The neural network model is segmented into multiple versions with different levels of pruning and optimization, allowing the system to select and execute appropriate segments based on runtime conditions rather than manipulating a single monolithic model
Solution Approach 2:
Multiple model versions are pre-generated with different pruning levels and optimizations before runtime execution, eliminating the need for complex real-time model manipulation by simply selecting from pre-prepared options
3Quantity of substance
If model size is reduced for resource-constrained environments, then computational load decreases, but accuracy may deteriorate
Solution Approach 1:
The system dynamically switches between models of different sizes based on the trade-off between accuracy requirements and computational constraints, allowing flexible adaptation to maintain acceptable accuracy while optimizing for resource efficiency
Solution Approach 2:
Different portions of the neural network are pruned with different aggressiveness levels, preserving critical pathways that maintain accuracy while removing redundant components to reduce overall model size and computational load
Data Source
AI summary
The disclosed embodiments relate to a system that generates and executes a deep neural network (DNN) based on target runtime parameters. During operation, the system receives a trained original model and a set of target runtime parameters for the DNN, wherein the target runtime parameters are associated with one or more of the following for the DNN: desired operating conditions, desired resource utilization, and desired accuracy of results. Next, the system generates a context-specific model based on the original model and the set of target runtime parameters. The system also generates an operational plan for executing both the original model and the context-specific model to meet requirements of the target runtime parameters. Finally, the system controls execution of the original model and the context-specific model based on the operational plan.