ML Model Deployment Using Intermediate Representation on Edge Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning frameworks face challenges in deploying models on edge nodes due to limitations in computing resources, storage resources, and inefficiencies in scheduling and parameter management, leading to reduced efficiency and user experience.
Innovation Solution
A method and framework that utilizes an open neural network exchange format and multi-level intermediate representation to deploy machine learning models across various computing devices, allowing for customizable scheduling and parallelism, optimizing parameter usage and deployment efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are deployed on edge nodes, then computing resources are provided closer to users, but storage resources are limited and parameters occupy excessive space
Solution Approach 1:
The patent implements a universal parameter storage space that can serve multiple machine learning models simultaneously. Instead of dedicating separate storage for each model, the system creates a shared storage infrastructure that accommodates parameters from different models, thereby reducing overall storage consumption while maintaining the ability to deploy multiple models on edge nodes with limited resources
Solution Approach 2:
The patent merges parameter storage resources into a unified pool that can be dynamically allocated across different machine learning models. By combining previously isolated storage spaces into a shared resource, the system optimizes storage utilization and reduces the total parameter occupancy on edge nodes, directly addressing the storage resource limitation problem
2Loss of energy
If machine learning models are deployed on edge nodes, then network transmission losses are reduced, but scheduling complexity increases
Solution Approach 1:
The patent introduces a parameter server as an intermediary component that manages the complex scheduling tasks. The parameter server acts as a mediator between the central control system and edge nodes, handling parameter distribution, updates, and coordination. This intermediary absorbs the scheduling complexity, allowing edge nodes to focus on executing machine learning inference while the parameter server manages the complex coordination required for multi-model deployment
3Ease of operation
If machine learning models are deployed on edge nodes, then user experience is improved, but repetitive parameters waste storage resources
Solution Approach 1:
The patent implements a universal parameter storage space that enables multiple machine learning models to share common parameters. Instead of each model maintaining its own complete parameter set, the system allows models to reference and share parameters from the universal storage space, eliminating repetitive parameter storage while maintaining model functionality and user experience
Data Source
AI summary
Embodiments of the present disclosure relate to a method, an electronic device, and a computer program product for deploying a machine learning model. The method includes: acquiring a machine learning model in accordance with an open neural network exchange format; converting the machine learning model to an intermediate representation using a multi-level intermediate representation method; and deploying a computation associated with the machine learning model to at least one computing device using the intermediate representation.