Multi-tenant ML Architecture Stateless VMs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deploying machine learning solutions to application platform-as-a-service (aPaaS) is challenging due to an impedance mismatch between aPaaS architectures, which are geared for transaction-processing systems, and the requirements of compute-bound machine learning systems that need long-running processing and large client-dependent states.
Innovation Solution
Implementing a system architecture that uses stateless virtual machines in aPaaS to support machine learning by providing training and inference services, with dynamic loading/unloading of ML models, sticky sessions for inference, and a Training Provisioner to manage training worker instances, allowing for multi-tenancy and efficient resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If aPaaS uses stateless virtual machines with minimal computational resources for transaction-processing, then resource utilization and scalability are improved, but machine learning workloads requiring long-running compute-bound processing and large client-dependent state cannot be effectively supported
Solution Approach 1:
The system segments ML training workloads into discrete tasks that can be distributed across multiple stateless VM instances. The training master coordinates these segmented tasks, allowing the system to maintain stateless VM benefits while supporting compute-bound ML processing through task-level granularity.
Solution Approach 2:
The training master acts as an intermediary between the aPaaS platform and ML training workers. It manages the coordination, state tracking, and workflow orchestration, enabling stateless VMs to participate in long-running ML training processes without requiring individual VMs to maintain state.
2Reliability
If aPaaS packages server software into self-contained stateless virtual machines, then system reliability and ease of deployment are improved, but the ability to maintain long-running compute-bound processing and client-dependent state is lost
Solution Approach 1:
The system performs preliminary actions by having the training master pre-coordinate training tasks and prepare workload distributions before VM instances are needed. This allows stateless VMs to be rapidly provisioned and terminated while maintaining the appearance of long-running processing through continuous task reassignment.
Solution Approach 2:
The stateless VM instances are designed to be universal and multi-functional, capable of executing different ML training tasks assigned by the training master. This allows the same pool of stateless VMs to handle various compute-bound workloads sequentially, maintaining reliability while supporting diverse long-running ML processes.
3Productivity
If aPaaS uses multiple small stateless virtual machines, then scalability and cost-efficiency are improved, but machine learning systems requiring large computational resources and persistent state face deployment challenges
Solution Approach 1:
The training master and worker system implements self-service by automatically managing task distribution, progress tracking, and instance coordination without requiring complex manual configuration. This reduces deployment complexity for ML systems while maintaining the scalability benefits of multiple small stateless VMs.
Solution Approach 2:
The system uses feedback mechanisms where training workers report progress and status to the training master, which then adjusts task distribution and resource allocation accordingly. This automated feedback loop simplifies the deployment and management of ML workloads across scalable stateless VM infrastructure.
Data Source
AI summary
Implementations of the present disclosure include methods, systems, and computer-readable storage mediums for training ML models in aPaaS architectures including actions of receiving, at a training master provided using a first VM, a training request to train a ML model, the training request being received from a first tenant in a multi-tenant, aPaaS architecture, initiating, by the training master, training of the ML model by a training worker provided using a second VM, during training of the ML model, periodically updating training storage metadata based on metadata describing progress of the training of the ML model, and in response to completion of the training of the ML model, storing a trained ML model in a model storage of the aPaaS architecture.