Hardware-Aware AutoML Using Virtual Processor Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AutoML systems face challenges in efficiently exploring the high-dimensional space of ML models, require resource-intensive training and testing, lack knowledge sharing, and struggle with transfer learning, leading to inefficient and costly model optimization for edge devices.
Innovation Solution
A system and method for optimizing trained ML hardware models by simultaneously calculating and comparing real-time performance metrics across multiple processor chips, using a multi-tenant AutoML architecture to automate the selection and optimization of model architectures and hardware processors, enabling efficient and accurate model creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional AutoML systems are used to explore ML model space, then model selection can be performed, but the process requires resource-intensive training and testing that consumes high energy and time
Solution Approach 1:
The patent creates virtual copies of hardware processors with simulated performance characteristics. Instead of training and testing models on actual physical hardware, the system uses virtualized processor models that replicate key performance metrics. This allows comprehensive model evaluation without the energy cost of actual hardware training, resolving the contradiction between accurate performance measurement and energy consumption.
Solution Approach 2:
The system performs preliminary characterization of hardware processor performance characteristics before actual model training. By pre-establishing virtual models that capture processor behavior, the system avoids repeated resource-intensive training runs. The preliminary virtual characterization enables subsequent rapid model evaluation, reducing both energy consumption and training time while maintaining evaluation accuracy.
2Manufacturing precision
If comprehensive ML model space exploration is performed through training and testing, then optimal model selection is achieved, but the process takes long training time
Solution Approach 1:
The patent uses virtual copies of hardware processors to simulate model execution performance. These virtual models allow rapid evaluation of multiple ML architectures without the time penalty of actual training runs. The virtualized environment maintains sufficient fidelity to produce reliable optimization results while reducing evaluation time from days to minutes, resolving the contradiction between optimization quality and time loss.
Solution Approach 2:
The system extracts and separates the performance evaluation function from the actual training process. By pulling out the evaluation component and implementing it through virtual hardware models, the system enables parallel assessment of multiple models without sequential training delays. This extraction allows comprehensive model space exploration to occur independently and concurrently, maintaining optimization quality while eliminating time bottlenecks.
3Adaptability or versatility
If multiple hardware processors are tested with different ML models, then hardware-model compatibility is determined, but the computational burden increases significantly
Solution Approach 1:
The patent creates virtual representations of multiple hardware processors, each with simulated performance characteristics. These virtual models allow simultaneous evaluation of ML model compatibility across different hardware architectures without the computational burden of actual multi-processor testing. The virtualization approach maintains hardware-model matching accuracy while reducing the computational power requirement by several orders of magnitude.
Solution Approach 2:
The system develops a universal virtualization framework that can simulate multiple different hardware processor types within a single computational environment. This multi-functional virtual platform eliminates the need for separate physical testbeds for each hardware architecture. The universal virtual environment provides accurate hardware-model compatibility assessment across diverse processors while consolidating computational resources, resolving the contradiction between adaptability and power consumption.
4Ease of manufacture
If manual curation of ML model space is performed using heuristics, then some model selection is achieved, but most of the model space remains unexplored
Solution Approach 1:
The patent uses virtual hardware processor models to enable automated exploration of extensive ML model spaces. The virtualization framework allows systematic evaluation of numerous model architectures that would be impractical to assess manually. This approach maintains the simplicity of automated process execution while dramatically increasing model space exploration coverage from sparse sampling to comprehensive analysis.
Solution Approach 2:
The system introduces virtual hardware models as an intermediary between manual heuristic selection and actual hardware deployment. This intermediate layer enables automated exploration and comparison of extensive model spaces using simulated performance data. The virtual intermediary provides structured, scalable evaluation that bridges the gap between simple manual processes and comprehensive automated analysis, increasing productivity while maintaining ease of use through standardized virtual interfaces.
Data Source
AI summary
Automated machine learning (Auto ML) for creating and optimizing ML models using a model store for storing: trained ML models and hardware models; test metrics data corresponding to the stored models; ML advised-models. Using a model meta-services (MMS) for: accessing the stored models and the test metrics data; creating the ML meta-models based on the runtime test metrics data; and answering MPC queries. Using a models producer and consumer (MPC) for: selecting a ML advised-model; testing the selected ML advised-model using selected ML test inputs and outputs to provide runtime test metrics data; optimizing the selected ML advised-model using the runtime test metrics data; sending the optimized ML advised-model to the model store unit for storing as one of the stored ML advised-models; and sending the runtime test metrics data to the model store unit for storing as part of the runtime test metrics data; and sending the MPC queries.