Inference Model Partitioning for Resource-Constrained Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently executing inference models across multiple data processing systems due to high computing resource consumption and excessive communication bandwidth usage, particularly when individual systems lack sufficient resources to complete timely execution.
Innovation Solution
The inference model is partitioned into portions and distributed across multiple data processing systems using a bipartite graph representation, optimizing the model to reduce data dependencies and communications, thereby conserving resources and bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the inference model is executed on a single data processing system, then the execution is simplified, but the computing resource consumption becomes excessive and individual systems lack sufficient resources to complete timely execution
Solution Approach 1:
The inference model is divided into multiple partitions, each assigned to different data processing systems. This segmentation allows the computational workload to be distributed across multiple systems, reducing the resource consumption burden on any single system while maintaining overall execution capability and timeliness.
2Use of energy by moving object
If the inference model is partitioned and distributed across multiple data processing systems, then computing resources are conserved, but the communication bandwidth usage becomes excessive
Solution Approach 1:
Data processing systems that are geographically or network-wise close to each other are grouped into the same partition. This merging strategy reduces the communication distance and bandwidth requirements between systems within a partition, minimizing the communication overhead while still distributing the computational workload effectively.
3Loss of energy
If the inference model is partitioned to reduce communication, then bandwidth is conserved, but the system complexity increases due to optimization requirements
Solution Approach 1:
The partitioning optimization is performed locally at each data processing system based on its specific characteristics, available resources, and proximity to other systems. This local optimization approach reduces the overall system complexity by avoiding the need for centralized complex optimization while still achieving effective partitioning that conserves communication bandwidth.
Data Source
AI summary
Methods and systems for inference generation are disclosed. To manage inference generation, a system may include an inference model manager and any number of data processing systems. The inference model manager may represent an inference model as a bipartite graph. To obtain portions of the inference model, the bipartite graph may be partitioned into portions and the portions may undergo an optimization process. The optimization process may include adding and/or competing for nodes of the bipartite graph in order to increase the stability of the portion with respect to the available computing resources of a corresponding data processing system. The optimization process may continue until all portions achieve stability in a way that reduces necessary communications between the data processing systems. Each portion of the inference model may be distributed to one data processing system so that the data processing systems may collectively generate inferences usable by a downstream consumer.