Pruned Model Selection for Edge Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deploying machine learning models to edge nodes is challenging due to resource constraints and the need for data privacy, as each device has local data that should be kept private, and computing resources at many nodes are inadequate for exhaustive model training.
Innovation Solution
A system that generates and deploys pruned machine learning models by initializing candidate models at source nodes, training them using distilled datasets, pruning them based on magnitude criteria, and validating them across multiple nodes to ensure accuracy and privacy, with a central node coordinating the process to select and deploy the winning candidate model to target nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are trained exhaustively at edge nodes, then model accuracy is improved, but computing resource requirements increase beyond what many nodes can provide
Solution Approach 1:
The model training process is segmented into two phases: (1) centralized pre-training at a server with sufficient resources to create a high-accuracy base model, and (2) distributed fine-tuning at edge nodes to adapt the model to local data. This segmentation allows the accuracy-intensive work to be done centrally while the resource-constrained nodes perform lighter adaptation tasks.
Solution Approach 2:
The base model is preliminarily trained and optimized at a centralized server before being deployed to edge nodes. This preliminary action includes comprehensive training on aggregated data, hyperparameter tuning, and model pruning, so that when the model reaches edge nodes, it already possesses high accuracy and requires only minimal local adaptation.
2Adaptability or versatility
If local data is shared across nodes for model training, then model generalization is improved, but data privacy is compromised
Solution Approach 1:
A centralized server acts as an intermediary that aggregates data from multiple nodes, trains the base model using this aggregated data, and then distributes the trained model back to nodes. This intermediary approach allows the model to learn from diverse data sources improving generalization, while individual nodes never directly share their raw data with other nodes, preserving data privacy.
Solution Approach 2:
Instead of sharing original data across nodes, the system creates and shares copies in the form of trained model parameters and weights. The centralized server creates a copy of the learned knowledge from aggregated data and distributes it to all nodes, enabling generalization without exposing sensitive raw data.
3Measurement precision
If full models are deployed to edge nodes, then model accuracy is maintained, but device resource consumption increases
Solution Approach 1:
The system extracts and removes unnecessary components from the full model before deploying to edge nodes. This includes pruning redundant neurons and connections, removing unused layers, and optimizing the model architecture based on what is actually needed for the specific edge application, thereby reducing resource consumption while maintaining essential accuracy.
Solution Approach 2:
The model parameters are changed and optimized for edge deployment by adjusting hyperparameters, quantizing weights to lower precision, and retraining with constraints on model size. These parameter changes enable the model to function with reduced computational requirements while preserving acceptable accuracy levels.
Data Source
AI summary
Searching for a model is disclosed. Source nodes are configured to generate pruned candidate models starting from a distribution of models. A central node receives the pruned candidate models and their associated loss values. The central mode causes the pruned candidate models to be tested in a distributed manner at generalization nodes. Loss values returned to the central mode are associated with the pruned candidate models. The pruned candidate model with a lowest loss score, based on the distributed generalization testing, is selected as a winning candidate model and deployed to target nodes.