Edge AI Model Deployment Workflow With Real-Time Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a lack of a comprehensive end-to-end (E2E) solution for integrating and deploying artificial intelligence (AI) models in industrial scenarios, particularly on edge devices, and existing monitoring solutions are inadequate for real-time performance verification and retraining.
Innovation Solution
A method and system for deploying AI models on edge devices, involving the generation of a deployment file containing the AI model, workflow description, wrapper script, and dependency library, enabling the edge device to create and run the workflow, with integrated monitoring nodes for real-time verification and retraining based on real-time data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a comprehensive E2E solution for integrating AI models in industrial scenarios is implemented, then the adaptability and integration capability are improved, but the device complexity and deployment difficulty increase
Solution Approach 1:
The deployment package is segmented into distinct functional modules: workflow description file (defining node relationships and data flow), dependency library (managing runtime environment requirements), wrapper script (automating deployment operations), and monitoring node (enabling real-time performance verification). This segmentation allows each component to be independently developed, tested, and maintained, reducing overall deployment complexity while preserving comprehensive integration capability
Solution Approach 2:
A configuration apparatus serves as an intermediary between the AI model development environment and the edge device deployment environment. This intermediary automatically generates the workflow description file, dependency library, and wrapper script based on the trained AI model, eliminating manual configuration efforts and bridging the gap between different technical domains (IT and OT)
2Measurement precision
If real-time monitoring nodes are integrated into the workflow, then the measurement precision and performance verification capability are improved, but the device complexity increases
Solution Approach 1:
The monitoring node is merged into the existing workflow as an integrated component rather than a separate external system. The workflow description file naturally incorporates monitoring operations alongside data processing and AI inference operations, allowing performance verification to occur seamlessly within the same execution context without requiring additional standalone monitoring infrastructure
Solution Approach 2:
The monitoring node enables the workflow to self-verify its own performance by collecting and analyzing runtime data automatically. The system monitors its own execution metrics, model inference accuracy, and resource utilization without external intervention, achieving precise performance verification while maintaining workflow simplicity
3Measurement precision
If the AI model is retrained using real-time data from the edge device, then the model accuracy is improved, but the loss of time and computational resources increase
Solution Approach 1:
The system implements periodic retraining cycles where the AI model is retrained at predetermined intervals or when performance degradation thresholds are met, rather than continuously. This periodic action allows the system to maintain high model accuracy while minimizing the time and computational resources spent on retraining operations
Solution Approach 2:
The system selectively discards outdated or low-quality training data while recovering and retaining valuable real-time data from the edge device. By curating training datasets that focus on high-value, representative samples rather than processing all available data, the retraining process achieves improved model accuracy with reduced computational overhead and faster execution time
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Disclosed are a method and system for deploying an artificial intelligence (AI) model in an edge device, a device, and a medium. The method includes: acquiring a trained AI model; generating a first deployment file, where the first deployment file includes the AI model, a workflow description file, a wrapper script, and a dependency library; the workflow description file is adapted to describe a workflow in the edge device, and the workflow includes an inference node deployed with the AI model; the dependency library is adapted to create a running environment for each node in the workflow; and the wrapper script is adapted to run each node in the workflow; and sending the first deployment file to the edge device, to enable the edge device to generate the workflow based on the workflow description file, create the running environment for each node in the workflow based on the dependency library, and run each node in the workflow based on the wrapper script. The AI model can be deployed in the edge device conveniently and trained anew through real-time data of the edge device, so that the accuracy of the model is improved.