Edge ML Model Switching Without Container Restart Downtime
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for updating machine learning models in edge applications result in significant downtime and data loss due to the need to stop and restart containers, which is inefficient for continuous inferencing tasks.
Innovation Solution
A method where machine learning models are mounted from storage to application containers at runtime, allowing for seamless switching between model versions without restarting the application, using an edge manager to signal the application to switch to a new model version.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are updated using traditional methods, then model versions can be changed, but significant downtime is required causing application unavailability
Solution Approach 1:
The system segments the model update process into independent components: multiple model versions are maintained separately in local storage, each with its own inference endpoint. The application can switch between segmented model versions without affecting overall system operation, enabling zero-downtime updates.
Solution Approach 2:
New model versions are pre-loaded and validated in local storage before being activated. The system performs preliminary actions by preparing the next model version in advance while the current model continues serving requests, allowing seamless transitions without application downtime.
2Speed
If model versions are stored and switched at network edge, then update speed improves, but data loss may occur during transitions
Solution Approach 1:
The system implements feedback mechanisms through configuration files and health check endpoints that monitor model version status and data integrity. During transitions, the application receives feedback about model readiness and data synchronization status, allowing it to pause or rollback if data loss is detected, thus preventing information loss while maintaining fast update speeds.
3Adaptability or versatility
If multiple model versions are maintained in local storage, then update flexibility increases, but storage requirements increase
Solution Approach 1:
The system applies local quality by storing multiple model versions with different retention policies and priorities. Recently used or active model versions are maintained in local storage for fast access, while older versions can be archived or removed. This selective local storage approach provides version flexibility while controlling storage volume requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Resources, such as machine learning models, can be updated for an application without any significant downtime for that application. For an application hosted at a network edge, the application can be deployed in a container and one or more model versions stored in local storage at the edge, which can be mounted into the container as necessary. When a different model version is to be used, a configuration change or new context can be used to trigger the application to automatically change to the different model version. This updating can be performed seamlessly, without any loss of data.