Hardware-Aware AutoML Optimization Using Operator Performance Proxies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AutoML systems face challenges in efficiently exploring the high-dimensional space of ML models and neural network architectures, requiring extensive resource-intensive training and testing, leading to high energy consumption, long training times, and lack of knowledge sharing between runs, which hinders the optimization of ML models for specific hardware.
Innovation Solution
An automated machine learning (AutoML) system that calculates and compares real-time performance metrics across multiple processor hardware chips, optimizing ML models by simultaneously testing and selecting the best model and chip combination based on power, performance, and accuracy, using a multi-tenant, large-scale architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If extensive training and testing of multiple ML models is performed to explore the high-dimensional model space, then model optimization accuracy is improved, but energy consumption increases significantly
Solution Approach 1:
The system performs preliminary characterization of neural network operators' execution performance on specific hardware before actual model training. This pre-analysis includes measuring execution time, memory usage, and power consumption of various operators on the target hardware platform, storing these measurements in a lookup table that guides subsequent model exploration, thereby avoiding repeated full training cycles and reducing overall energy consumption
Solution Approach 2:
The system creates simplified copies or proxies of the full training process through operator-level performance measurements. Instead of training complete models to evaluate performance, the system uses measured operator execution characteristics as proxies to predict and compare full model performance across different architectures, dramatically reducing the computational energy required while maintaining optimization accuracy
2Manufacturing precision
If multiple ML models are trained and tested to determine optimal model architecture, then model performance is improved, but training time increases significantly
Solution Approach 1:
The system performs preliminary characterization of neural network operators' execution performance on specific hardware before actual model training. This pre-analysis includes measuring execution time, memory usage, and power consumption of various operators on the target hardware platform, storing these measurements in a lookup table that guides subsequent model exploration, thereby avoiding repeated full training cycles and reducing overall energy consumption
Solution Approach 2:
The system changes the evaluation parameters from full model training metrics to operator-level execution characteristics. By measuring and comparing individual operator performance (execution time, memory footprint, power consumption) on the target hardware, the system can predict overall model performance without completing full training, enabling rapid comparison of multiple architectures and significantly reducing the time required to identify optimal models
3Manufacturing precision
If hardware-specific performance characterization is performed for each model, then hardware optimization accuracy is improved, but computational resources required increase
Solution Approach 1:
The system creates a universal characterization framework where a single set of operator performance measurements on specific hardware can be applied to evaluate multiple different neural network models. The lookup table storing operator execution characteristics serves as a universal reference that guides model selection and optimization across various architectures without requiring separate hardware-specific training for each model, thereby reducing computational resource requirements while maintaining hardware optimization accuracy
Solution Approach 2:
The system creates simplified copies or proxies of the full training process through operator-level performance measurements. Instead of training complete models to evaluate performance, the system uses measured operator execution characteristics as proxies to predict and compare full model performance across different architectures, dramatically reducing the computational energy required while maintaining optimization accuracy
Data Source
AI summary
Automated machine learning (Auto ML) for creating and optimizing ML models using a model store for storing: trained ML models and hardware models; test metrics data corresponding to the stored models; ML advised-models. Using a model meta-services (MMS) for: accessing the stored models and the test metrics data; creating the ML meta-models based on the runtime test metrics data; and answering MPC queries. Using a models producer and consumer (MPC) for: selecting a ML advised-model; testing the selected ML advised-model using selected ML test inputs and outputs to provide runtime test metrics data; optimizing the selected ML advised-model using the runtime test metrics data; sending the optimized ML advised-model to the model store unit for storing as one of the stored ML advised-models; and sending the runtime test metrics data to the model store unit for storing as part of the runtime test metrics data; and sending the MPC queries.