ML Model Graph Partitioning for Heterogeneous Processing Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models face challenges in efficient execution in heterogeneous processing environments due to varying performance characteristics and overhead costs such as context switching and memory thrashing across different processing units.
Innovation Solution
The method involves partitioning a graph representing a machine learning model into subgraphs, simulating execution paths based on permutations of using different processing units for each subgraph, and selecting the execution path with the lowest cost. This approach considers overhead costs and optimizes resource utilization across heterogeneous processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If machine learning models are executed on heterogeneous processing units, then computing resources can be utilized more flexibly, but context switching overhead and memory thrashing increase
Solution Approach 1:
The machine learning model is divided into multiple subgraphs, where each subgraph can be executed on a specific processing unit. This segmentation allows the system to minimize context switching by keeping related operations together on the same processing unit, while still utilizing heterogeneous processing units for different subgraphs based on their computational characteristics.
2Productivity
If machine learning models are executed on heterogeneous processing units, then resource utilization can be optimized, but memory thrashing increases
Solution Approach 1:
The system performs simulation of execution paths before actual model execution to identify the optimal processing unit for each subgraph. This preliminary analysis allows the system to pre-determine the execution plan, avoiding runtime decisions that would cause memory thrashing and enabling efficient resource allocation across heterogeneous processing units.
3Measurement precision
If execution paths are simulated based on all permutations of processing units, then optimal execution can be found, but computational complexity increases
Solution Approach 1:
By dividing the model into subgraphs and simulating execution paths for each subgraph separately rather than for the entire model, the system reduces the search space complexity. Each subgraph simulation considers only relevant processing units and operations, making the overall optimization tractable while still achieving near-optimal execution paths.
Data Source
AI summary
Certain aspects provide techniques and apparatuses for efficient operation of a machine learning model in a heterogeneous computing environment. An example method includes partitioning a graph representing a machine learning model into a plurality of subgraphs. Each subgraph generally represents a portion of the machine learning model. For each subgraph, a plurality of execution paths are simulated based on permutations of using different processing unit types to execute portions of the subgraph and starting with each input source processing unit type selected from the different processing unit types, and an execution path having a lowest cost is selected from the plurality of execution paths. The machine learning model is implemented based on the selected execution path for each subgraph.