ML Model Swapping With Similarity-Based Subnet Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ML model optimization techniques are time-consuming, resource-intensive, and fail to adapt efficiently to varying hardware platforms and environmental conditions, leading to suboptimal performance for users with different hardware and performance metric preferences.
Innovation Solution
A machine learning model scaling (MLMS) system that selects and deploys ML models in an energy and communication-efficient manner, using a similarity-based subnet selection process to minimize memory write operations and adapt to real-time system changes, while being agnostic to specific ML model search approaches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional NAS techniques are used to automatically discover ideal ML models, then model performance is improved, but training time and computational resources increase significantly
Solution Approach 1:
The patent pre-trains a supernet model that encompasses multiple possible subnet architectures before deployment. This preliminary action allows the system to quickly swap between pre-configured subnets based on runtime conditions without performing time-consuming NAS training at deployment time, thus resolving the contradiction between model performance and training time.
Solution Approach 2:
The system dynamically selects and swaps between different subnet architectures from the pre-trained supernet based on real-time hardware conditions, power modes, and performance requirements. This dynamic adaptation eliminates the need for repeated NAS training while maintaining optimal model performance across varying operational conditions.
2Productivity
If ML models are optimized for specific hardware platforms, then inference efficiency is improved, but adaptability to different hardware platforms decreases
Solution Approach 1:
The patent creates a universal supernet model that can adapt to multiple hardware platforms through subnet swapping. The supernet is designed to encompass various subnet architectures that can be selectively activated based on the target hardware platform, thus achieving both high inference efficiency on specific platforms and broad adaptability across different platforms.
Solution Approach 2:
The system changes architectural parameters by swapping between different subnet configurations within the supernet to match specific hardware platform characteristics. This allows the model to optimize inference efficiency for each platform without requiring separate training processes, maintaining both efficiency and adaptability.
3Measurement precision
If larger ML architectures are used, then model accuracy is improved, but resource consumption and training time increase
Solution Approach 1:
The patent segments the large supernet model into multiple smaller subnet architectures that can be selectively deployed. Each subnet represents a different trade-off between accuracy and resource consumption, allowing the system to choose the appropriate subnet size based on available resources while maintaining the option to access higher-accuracy configurations when resources permit.
4Reliability
If ML models are manually designed and tuned, then model performance is improved, but design time and expertise requirements increase
Solution Approach 1:
The patent performs the time-consuming model design and tuning process in advance by pre-training the supernet and configuring multiple subnet architectures. This preliminary action eliminates the need for manual design and iterative tuning at deployment time, as the system can directly utilize the pre-optimized subnets that are ready for immediate deployment.
Data Source
AI summary
The present disclosure is related to machine learning model swap (MLMS) framework for that selects and interchanges machine learning (ML) models in an energy and communication efficient way while adapting the ML models to real time changes in system constraints. The MLMS framework includes an ML model search strategy that can flexibly adapt ML models for a wide variety of compute system and/or environmental changes. Energy and communication efficiency is achieved by using a similarity-based ML model selection process, which selects a replacement ML model that has the most overlap in pre-trained parameters from a currently deployed ML model to minimize memory write operation overhead. Other embodiments may be described and/or claimed.