ML-Based Accelerator Workload Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine learning models on large datasets is a computationally expensive and time-consuming task, as different workloads are better suited for specific types of hardware, leading to varying execution times across machine learning accelerators.
Innovation Solution
Implementing a workload orchestrator that uses artificial intelligence to dynamically select the appropriate machine learning accelerator based on workload attributes, such as parallelization and resource intensity, by evaluating and training neural networks with performance metrics from potential accelerators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are trained on large datasets using traditional methods, then the models can be developed, but the training process is computationally expensive and time-consuming
Solution Approach 1:
The system performs preliminary actions by pre-training the neural network model with workload-accelerator mapping data before actual workload execution. This preliminary training enables the system to quickly determine optimal accelerator assignments without performing exhaustive searches during runtime, thereby reducing training time while maintaining model development capability
Solution Approach 2:
The system dynamically adapts the neural network model based on varying workload characteristics and accelerator performance metrics. By continuously updating the model with new data and retraining it to reflect changing conditions, the system optimizes training efficiency while maintaining the ability to handle diverse machine learning workloads across different hardware accelerators
2Productivity
If different workloads are assigned to different hardware accelerators based on their characteristics, then execution efficiency is improved, but the complexity of workload management increases
Solution Approach 1:
The neural network model serves as an intermediary between workloads and hardware accelerators, translating workload characteristics into optimal accelerator assignments. This intermediary layer simplifies workload management by automatically determining the best accelerator match based on learned patterns, thereby improving execution efficiency while reducing the complexity of direct workload-allocator interactions
Solution Approach 2:
The system changes parameters by transforming workload attributes and accelerator characteristics into a format suitable for neural network processing. By encoding workload properties and accelerator metrics as input parameters for the model, the system enables efficient automated decision-making that improves execution efficiency while managing complexity through parameter transformation rather than complex rule-based logic
3Loss of time
If a neural network model is trained to predict optimal accelerator selection, then workload execution time is reduced, but the initial training cost and computational resources increase
Solution Approach 1:
The system applies partial training by using a subset of available workload-accelerator data for initial model training, rather than exhaustively training on all possible combinations. This partial action approach reduces the initial training computational cost while still achieving sufficient model performance to significantly reduce workload execution time in production
Solution Approach 2:
The system implements periodic retraining of the neural network model at scheduled intervals or when new data becomes available, rather than continuously training. This periodic action maintains model accuracy and adapts to changing conditions while controlling computational costs by limiting training to specific time periods, thereby reducing overall training energy consumption while maintaining fast execution performance
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture for allocating a workload to an accelerator using machine learning are disclosed. An example apparatus includes a workload attribute determiner to identify a first attribute of a first workload and a second attribute of a second workload. An accelerator selection processor causes at least a portion of the first workload to be executed by at least two accelerators, accesses respective performance metrics corresponding to execution of the first workload by the at least two accelerators, and selects a first accelerator of the at least two accelerators based on the performance metrics. A neural network trainer trains a machine learning model based on an association between the first accelerator and the first attribute of the first workload. A neural network processor processes, using the machine learning model, the second attribute to select one of the at least two accelerators to execute the second workload.