Machine Learning Model Computation Graph Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models require significant computing resources and communication bandwidth, making it challenging to deploy large-scale models efficiently due to limitations in central processing units (CPUs) and communication bandwidth, and existing methods struggle with model parallelism and tensor data management.
Innovation Solution
A method that involves obtaining an intermediate representation of a machine learning model, generating a computation graph, and partitioning it into sequential parts for parallel execution, eliminating the need for inter-layer and intra-layer communication and tensor data division, allowing for efficient processing at a function level.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are made more complicated to handle larger data sets, then model accuracy and capability are improved, but computing resources and communication bandwidth requirements increase
Solution Approach 1:
The patent segments the computation graph into multiple independent parts that can be executed in parallel. Each part contains a subset of functions from the machine learning model, allowing the model to be distributed across multiple computing devices. This segmentation enables large-scale models to be processed without requiring excessive computing resources on a single machine.
2Measurement precision
If machine learning models are made more complicated to handle larger data sets, then model accuracy and capability are improved, but communication bandwidth requirements increase
Solution Approach 1:
The computation graph is divided into separate parts that can be executed independently in parallel. This segmentation reduces the need for inter-layer and intra-layer communication between computing devices, as each part can be processed autonomously. The patent explicitly states that this approach eliminates the need to take inter-layer and intra-layer communication into consideration.
3Productivity
If existing methods are used for model deployment, then current computing capabilities are utilized, but deployment efficiency is insufficient due to CPU limitations
Solution Approach 1:
The patent introduces an intermediate representation (IR) as a mediator between the source language model and the target execution environment. The IR is independent of both source and target languages and represents the model in a structured text format. This intermediary enables efficient compilation and deployment across different platforms without being constrained by specific CPU architectures or programming languages.
Solution Approach 2:
The patent changes the representation parameters of the machine learning model by converting it from source code into an intermediate representation with a computation graph. This parameter change allows the model to be processed, optimized, and deployed more efficiently across different computing environments, overcoming CPU limitations.
4Speed
If computation graph is partitioned into sequential parts for parallel execution, then processing speed is improved, but implementation complexity increases
Solution Approach 1:
The computation graph is automatically segmented into multiple parts based on function dependencies. Each part contains a subset of functions that can be executed in parallel without requiring complex coordination. The segmentation is performed in a way that simplifies the execution model, allowing parallel processing while maintaining clear boundaries between parts.
Solution Approach 2:
The system performs automatic partitioning of the computation graph without requiring manual intervention or complex configuration. The intermediate representation enables the system to self-organize the computation into executable parts, reducing the implementation complexity while achieving parallel execution.
Data Source
AI summary
A method comprises obtaining an intermediate representation of the machine learning model written in a source language. The intermediate representation is independent of the source language and a target language and comprises a structured text. The method also comprises generating a computation graph based on the intermediate representation. Nodes in the computation graph represent functions related to the machine learning model and a directed edge in the computation graph represents a dependency between functions. The method further comprises partitioning the computation graph into sequential parts in sequence such that the parts are executed sequentially, and functions corresponding to nodes in each part are executed in parallel.