Distributed ML Compiler Scheduling for Heterogeneous Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing the compilation of machine learning models for execution on edge devices with heterogeneous hardware architectures is challenging due to the complexity of hardware environments and the NP-hard nature of compiler optimization, leading to inefficient resource utilization and performance degradation.
Innovation Solution
A distributed compiler optimization process that allocates compute nodes, applies sequencing and scheduling solutions based on hardware information and precedence constraints, and identifies the optimal solution for mapping kernel operators to hardware resources, resulting in improved performance metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized compiler optimization process is used for machine learning models on heterogeneous edge devices, then the optimization can be coordinated centrally, but the computational complexity and time consumption increase significantly
Solution Approach 1:
The patent divides the compiler optimization process into two segments: a centralized controller that coordinates optimization and a distributed compute node system that executes optimization tasks. This segmentation allows complex optimization to be distributed across multiple compute nodes, reducing the time burden on any single system while maintaining centralized coordination for optimization effectiveness.
2Productivity
If the compiler optimization process is distributed across multiple compute nodes, then the computational load is reduced, but the system complexity increases
Solution Approach 1:
The compute nodes in the patent are designed as universal components that can perform multiple functions: executing machine learning models, performing optimization tasks, and communicating with the centralized controller. This multi-functionality reduces the need for specialized hardware components, thereby managing system complexity while enabling distributed optimization throughput.
3Productivity
If kernel operators are mapped to hardware resources based on hardware information and precedence constraints, then the hardware utilization is optimized, but the search space for optimal mappings becomes extremely large
Solution Approach 1:
The patent implements dynamic optimization where the controller receives hardware information and precedence constraints, then dynamically determines optimal kernel operator mappings. The system adapts its optimization strategy based on real-time hardware status and model requirements, allowing efficient hardware utilization while managing search space complexity through dynamic rather than static mapping approaches.
Data Source
AI summary
A method for optimizing the compilation of a machine learning model to be executed on target edge devices is provided. Compute nodes of a plurality of compute nodes are allocated to a compiler optimization process for a compiler of said machine learning model. The machine learning model has a compute graph representation having nodes that are kernel operators necessary to execute the machine learning model and edges that connect said kernel operators to define precedence constraints. A round of optimization is scheduled for the process amongst the allocated compute nodes. At each allocated compute node a sequencing and scheduling solution is applied per round to obtain a performance metric for the machine learning model. From each compute node the performance metric is received and a solution that has the best performance metric is identified and implemented for execution of the machine learning model on the target edge devices.