ML Model Deployment via Unified Intermediate Representation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of deploying large-scale machine learning models is hindered by the limitations of single machine computation capacity and communication bandwidth, requiring developers to have knowledge of low-level infrastructure and multiple frameworks, leading to increased difficulty and inefficiency.
Innovation Solution
A method that converts machine learning models into a unified intermediate representation, determining computing resources based on the original code, allowing for deployment across different devices without requiring knowledge of specific devices or frameworks, thereby simplifying the deployment process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single machine is used to execute the machine learning model, then the deployment process is simple, but the computation capacity is insufficient for large-scale models
Solution Approach 1:
The patent segments the machine learning model into multiple operators and distributes them across multiple computing nodes. The model is divided into sub-models that can be executed in parallel on different machines, thereby scaling computation capacity while maintaining manageable deployment complexity through automated segmentation.
Solution Approach 2:
The patent transitions from single-machine execution to multi-node distributed execution by adding the dimension of network topology. Computing nodes are arranged in a distributed architecture with parameter storage nodes and computing nodes working together across the network, enabling scalability beyond single-machine limitations.
2Adaptability or versatility
If developers use multiple frameworks and low-level infrastructure knowledge, then the model can be deployed on different devices, but the deployment difficulty increases
Solution Approach 1:
The patent creates a universal intermediate representation that can be executed on multiple types of computing nodes including CPUs, GPUs, and FPGAs. This unified representation format enables the same model to be deployed across different device types without requiring separate framework-specific implementations, thereby achieving device compatibility while simplifying the deployment process.
Solution Approach 2:
The patent introduces an intermediate representation as a mediator between the model definition and device-specific execution. This intermediate format acts as a universal interface that decouples the model logic from hardware specifics, allowing automated deployment across diverse devices without requiring developers to learn multiple frameworks or low-level infrastructure details.
3Ease of operation
If the model is converted to a unified intermediate representation, then the deployment process is simplified, but the communication bandwidth requirement between nodes increases
Solution Approach 1:
The patent extracts parameter storage functions into separate parameter storage nodes, isolating the heavy parameter data from the computing nodes. This separation allows parameters to be stored and managed independently, reducing the communication overhead between computing nodes during model execution while maintaining deployment simplicity through the unified intermediate representation.
Data Source
AI summary
Embodiments of the present disclosure relate to a method, device and computer program product for deploying a machine learning model. The method comprises: receiving an intermediate representation indicating processing of a machine learning model, learning parameters of the machine learning model, and a computing resource requirement for executing the machine learning model, the intermediate representation, the learning parameters, and the computing resource requirement being determined based on an original code of the machine learning model, the intermediate representation being irrelevant to a programming language of the original code; determining, at least based on the computing resource requirement, a computing node and a parameter storage node for executing the machine learning model; storing the learning parameters in the parameter storage node; and sending the intermediate representation to the computing node for executing the machine learning model with the stored learning parameters.