Methods and systems for automated deployment and updating of edge AI models

Abstract

This invention proposes a method and system for automated deployment and updating of edge AI models, relating to the field of artificial intelligence technology. The method includes: unified management of edge device information to be deployed by users; an automatic push mechanism based on time periods to transmit changes in parameters of remote server models to target edge devices; unattended configuration of the edge device model operating environment through intelligent detection, automated script generation, and remote execution; seamless deployment and optimization of model inference services on edge devices by intelligently generating configuration files through platform tools deployed on edge devices; and automatic generation of API interfaces and API documentation for edge device models.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to methods and systems for the automated deployment and updating of edge AI models. Background Technology

[0002] With the rapid development of artificial intelligence technology, edge computing, as a distributed computing architecture, pushes computing and data storage from the central node to the edge of the network to improve response speed and reduce network bandwidth requirements. Edge AI model deployment is an important part of edge computing, which enables intelligent analysis and decision-making to be carried out near the data source, thereby reducing data transmission latency and bandwidth consumption.

[0003] On the one hand, traditional model deployment and update methods are difficult to meet the complexity of edge computing environments. Edge devices are often resource-constrained, including relatively limited computing power, storage capacity, and network bandwidth. Moreover, the devices are extremely dispersed, with diverse models and different operating systems. Manually deploying and updating models one by one not only consumes a lot of manpower and time, but is also prone to configuration errors, causing the models to malfunction.

[0004] On the other hand, business scenarios place extremely high demands on the timeliness of edge AI models. In intelligent security monitoring scenarios, once a new threat detection algorithm is updated, it needs to be rapidly deployed to the edge devices corresponding to each camera to respond promptly to new security risks. On industrial production lines, if the product defect detection model is delayed in updating, it may cause large batches of products to be produced, resulting in huge economic losses. Traditional deployment and update operations require manually downloading the complete model and then uploading it to the edge for deployment. This process is not only cumbersome but also time-consuming, especially for large models with large model parameters. Because it is necessary to transmit the complete model file, network bandwidth consumption is high, and the update process is time-consuming, resulting in model updates that are often not timely enough to meet the needs of rapid iteration. Therefore, how to perform updates and accurately grasp the push time to complete model updates without affecting the normal business operation of edge devices is an urgent problem to be solved.

[0005] Furthermore, with frequent model iterations, the problem of chaotic version management becomes increasingly apparent. If historical versions cannot be properly preserved and change logs clearly recorded, it becomes difficult to quickly roll back to a usable version when encountering unstable new models or models that are incompatible with business needs, severely impacting system stability and reliability. In addition, different edge devices may run different operating systems, increasing deployment complexity. These issues significantly affect the efficiency and flexibility of model deployment, especially in scenarios requiring frequent updates and large-scale deployments. Developers are forced to spend a significant amount of time on basic operational tasks rather than focusing on model optimization and innovation. Summary of the Invention

[0006] To address the aforementioned issues, this application provides an automated deployment and update method and system for edge AI models. This system enables unified management of edge device information, intelligent push and updates based on device characteristics and business needs, and automatic configuration of the model runtime environment, optimization of inference services, and automatic generation of API interfaces and documentation, thereby meeting the diverse needs of edge AI model deployment.

[0007] The objective of this application is achieved through the following technical solution:

[0008] In a first aspect, this application provides a method for automated deployment and updating of edge AI models, the method comprising:

[0009] Obtain edge device information through the user interface; manage the edge device information that users need to deploy in a unified manner through grouping and tagging settings;

[0010] An automatic push mechanism based on time periods transmits the changing parameters of the remote server model to the target edge device.

[0011] Through intelligent detection, automated script generation, and remote execution, unattended configuration of the edge device model's operating environment can be achieved;

[0012] By deploying platform tools on edge devices, configuration files are intelligently generated, enabling seamless deployment and optimization of model inference services on edge devices;

[0013] Automatically generate API interfaces and API documentation for edge device models.

[0014] On the other hand, this application provides a system for implementing the method for automated deployment and updating of edge AI models, the system comprising:

[0015] The edge service management module is used to obtain edge device information through the user interface; and to manage the edge device information that users need to deploy in a unified manner through grouping and tagging settings.

[0016] The model push module is used for an automatic push mechanism based on time periods to transmit the changing parameters of the remote server model to the target edge device.

[0017] The environment configuration module is used to enable unattended configuration of the operating environment of edge device models through intelligent detection, automated script generation and remote execution;

[0018] The inference service deployment module is used to intelligently generate configuration files by deploying platform tools on edge devices, thereby enabling seamless deployment and optimization of model inference services on edge devices.

[0019] The API auto-generation module is used to automatically generate API interfaces and API documentation for edge device models.

[0020] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored on and executable on the memory, wherein the processor executes the program to implement the steps of any of the methods described in this application.

[0021] The beneficial effects of this invention include: By centrally managing edge device information, including IP addresses, operating system types, and hardware configurations, the complexity of device management and maintenance is greatly simplified. Unified management not only improves work efficiency but also ensures the accuracy and consistency of information. The time-based automatic push mechanism can intelligently identify and transmit changes in remote server model parameters to the target edge device. This feature not only reduces manual intervention but also ensures the timeliness and accuracy of model updates. Furthermore, through intelligent detection, automated script generation, and remote execution, unattended configuration of the edge device model operating environment is achieved, further improving deployment efficiency. The optimal time period for model push is intelligently determined based on the actual load and business needs of the edge device, thereby avoiding service interruptions or performance degradation caused by updates during peak business periods. In addition, through intelligent configuration file generation, seamless deployment and optimization of model inference services on edge devices are achieved, ensuring effective resource utilization and high service availability. A new version number is generated for each model update, and historical version information is retained, making rollback operations simple and reliable. Simultaneously, based on factors such as edge device capacity, computing resources, model capacity, and energy consumption, the maximum number of models to retain on edge devices is intelligently determined. This ensures model diversity, eliminates the need to update models from remote services during rollback, and avoids excessive resource consumption. In the event of edge device environment configuration failure, the system automatically rolls back to the stable state before configuration and records environment configuration logs, ensuring system stability and reliability. This also facilitates troubleshooting and subsequent optimization. Based on the model's input and output characteristics, RESTful API code and documentation are automatically generated, simplifying API development and deployment while improving API availability and usability. During model updates, relevant APIs are automatically updated, ensuring API-model consistency.

[0022] In summary, this application significantly improves the deployment and updating efficiency of edge AI models, reduces operation and maintenance costs, and enhances service stability and availability through automation, intelligence, and optimization. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of an automated deployment and update method for edge AI models provided in an embodiment of this application;

[0024] Figure 2 This is a schematic diagram illustrating the process of updating the remote service model to the edge device according to an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of the process for obtaining the changed parameters provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of the push time period determination process provided in the embodiments of this application. Detailed Implementation

[0027] The present application will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0028] See attached document Figure 1 This application provides a method for automated deployment and updating of edge AI models, the method comprising:

[0029] Obtain edge device information through the user interface; manage the edge device information that users need to deploy in a unified manner through grouping and tagging settings;

[0030] An automatic push mechanism based on time periods transmits the changing parameters of the remote server model to the target edge device.

[0031] Through intelligent detection, automated script generation, and remote execution, unattended configuration of the edge device model's operating environment can be achieved;

[0032] By deploying platform tools on edge devices, configuration files are intelligently generated, enabling seamless deployment and optimization of model inference services on edge devices;

[0033] Automatically generate API interfaces and API documentation for edge device models.

[0034] The working principle and effects of the above technical solution are as follows: Users input or select relevant information about edge devices through a dedicated user interface, such as device type, model, IP address, and location. This information is received and stored, providing basic data for subsequent edge AI model deployment and updates. Unified management is achieved through grouping and tagging: Users can group and tag edge devices according to actual needs, such as by geographical location, function type, or department. Tagging allows users to more quickly filter and locate target edge devices, improving management efficiency.

[0035] Real-time monitoring of changes in AI model parameters on remote servers, such as model version, parameter updates, and performance optimizations;

[0036] When a change is detected in the model, the system records these changed parameters and prepares to transmit them to the target edge device. Users can set specific time periods, such as off-peak hours at night, for automatic push of model change parameters. Based on the user-defined time period, the system automatically transmits model change parameters from the remote server to the target edge device; alternatively, the system can automatically set a time period for transmitting model change parameters from the remote server to the target edge device. Before model deployment, the system intelligently detects the operating environment of the target edge device, including device performance, storage space, and network connectivity. Based on the detection results, it assesses whether the hardware and software requirements for model deployment are met. If the deployment requirements are met, a deployment script is automatically generated based on the specific configuration of the target edge device and user needs. Through a remote execution mechanism, the system sends the deployment script to the target edge device and automatically executes the script to complete the configuration of the model's operating environment. A dedicated platform tool is deployed on the edge device to receive, parse, and execute model deployment instructions from the remote server. This platform tool also monitors the operational status of the model inference service, ensuring its stability and efficiency. Based on the model configuration information on the remote server and the actual situation of the target edge device, it intelligently generates a configuration file. The configuration file contains all the parameters and settings required for the model inference service, ensuring the model can run correctly on the edge device. The configuration file and model files are sent to the target edge device and seamlessly deployed using the platform tool. During deployment, the model inference service is optimized, such as adjusting resource allocation and optimizing algorithm parameters, to improve service performance and efficiency. Based on the model's functionality and user requirements, an API interface for the edge device model is automatically generated. The API interface provides access to the model inference service and parameter settings, facilitating user integration and invocation. Detailed API documentation is automatically generated, including interface descriptions, parameter explanations, and return value descriptions. The API documentation provides users with clear usage guidelines and reference information to help them better understand and use the edge device model's API interface.

[0037] In summary, the automated deployment and update method for edge AI models provided in this application achieves unified management of edge device information, automatic push of model change parameters, unattended configuration of model operating environment, seamless deployment and optimization of model inference service, and automatic generation of API interface and API documentation through a series of automated and intelligent means. This greatly improves the deployment and update efficiency of edge AI models, reduces operation and maintenance costs, and provides strong support for the widespread application of edge AI.

[0038] In some embodiments, the step of obtaining edge device information through a user interface and uniformly managing the edge device information that the user needs to deploy through grouping and tagging settings includes:

[0039] Edge device information is obtained through a user interface; the edge device information includes the edge device's IP address, operating system type, and hardware configuration.

[0040] Automatically detects the operating system type of edge devices;

[0041] Adjust the deployment strategy of edge device models according to operating system characteristics;

[0042] Automatically install the agent on edge devices and maintain communication with the server;

[0043] Regularly monitor the status of edge devices and report it to the management terminal in real time; display monitoring alarm information through a visual UI interface;

[0044] Edge devices can be categorized and managed through grouping and labeling.

[0045] The working principle of the above technical solution is as follows:

[0046] The overall system architecture of this method is as follows:

[0047] Server-side: As the central management node, it is responsible for receiving user instructions, storing device information, and processing monitoring data.

[0048] Agent: A client program installed on the edge device, responsible for collecting device information, executing server-side instructions, and reporting monitoring data.

[0049] User Interface (UI): A visual interface that provides functions such as user registration, device management, and monitoring.

[0050] The specific principle is as follows:

[0051] Users register accounts and log in to the system through the UI interface; users enter information such as the edge device's IP address, operating system type, and hardware configuration to register the device; the system receives the user's input information, performs preliminary verification, and stores it; a multi-operating system compatibility detection mechanism is provided, which can automatically detect the edge device's operating system type (such as Linux, Windows, etc.). Based on the detected operating system type, the system adjusts the deployment strategy and selects a suitable Agent installation method and communication protocol. The edge device receives the installation package and automatically installs the Agent; after the Agent is installed, it establishes a connection with the server and maintains communication based on the authentication information provided by the system (such as SSH keys or WinRM credentials). During data transmission, for Linux edge hosts, SSH key authentication is used for remote connection; for Windows systems, the WinRM (Windows Remote Management) protocol is used.

[0052] The agent can periodically check the online status and resource usage (CPU, memory, disk, etc.) of edge devices and report them to the management terminal in real time. Users can view monitoring-level alarm information through a visual UI interface. The server performs real-time analysis on the received monitoring data and triggers alarm information according to preset alarm rules.

[0053] Alarm information is displayed to users through a UI interface, where users can view detailed alarm information and a list of affected devices.

[0054] Users can group and tag devices through the UI interface, which facilitates the management and batch operation of large-scale devices. The grouping and tagging function allows users to classify and manage devices according to their geographical location, purpose and other information, thereby improving management efficiency.

[0055] The benefits of the above technical solution are as follows: Users can register, manage, and monitor all edge devices on a unified platform without having to log into multiple systems or tools separately. The system can automatically detect the device's operating system type and adjust deployment strategies accordingly, automatically installing agents, significantly reducing the time and workload of manual configuration. Through device grouping and tagging functions, users can easily perform batch operations on a large number of devices, such as upgrading software and modifying configurations, improving management efficiency. The system supports multiple operating system types, including Linux and Windows, and can automatically detect and adapt to the characteristics of different operating systems to ensure smooth device access and management. Based on the device's operating system type, it can automatically select the appropriate communication protocol (such as SSH key authentication or WinRM protocol) for data transmission, improving system flexibility and compatibility. The system employs secure authentication methods such as SSH key authentication and WinRM protocol to ensure the security and integrity of data transmission. The agent can periodically check the device's online status and resource usage, and report anomalies to the management terminal in real time, helping users to promptly identify and resolve problems, improving system reliability and stability. The system can collect and analyze device resource usage in real time, such as CPU, memory, and disk, helping users understand the device's operating status and resource bottlenecks. Based on monitoring data, the system can intelligently schedule and optimize resource allocation, improving device operating efficiency and resource utilization. The modular design facilitates future functional expansion and upgrades. Users can intuitively view device information, monitoring data, and alarm information through a visual UI interface, reducing operational difficulty and maintenance costs.

[0056] In summary, the unified edge service management system provides users with a comprehensive, efficient, and secure edge device management solution by improving management efficiency, enhancing system compatibility, improving security and reliability, optimizing resource utilization, and facilitating expansion and maintenance.

[0057] See attached document Figure 2 In some embodiments, the time-based automatic push mechanism transmits the changing parameters of the remote server model to the target edge device; including:

[0058] The model selection and configuration interface allows you to obtain the models and target edge devices that need to be deployed.

[0059] Compare the parameter differences between the remote service model and the edge device model to obtain the variable parameters;

[0060] Set a push time period, and within the push time period, transmit the changed parameters to the edge device;

[0061] A new version number is generated for each model update, and historical version information is retained on the remote server. The model version to be retained is selected on the edge device.

[0062] In response to a rollback request, the edge device model is rolled back to the target version;

[0063] Monitor the model update status in real time and record the change log.

[0064] The working principle and effect of the above technical solution are as follows: In the time-based automatic push mechanism, the changing parameters of the remote server model are efficiently transmitted to the target edge device. This process depends on the following key steps and components:

[0065] It allows users to select the model to be deployed and the target edge device through the user interface.

[0066] Users can set the start and end times for model push on this interface, or select a specific periodic push time period (such as 2:00 AM to 4:00 AM every day); or the system can automatically set the push time period.

[0067] First, the latest model parameters on the remote server are obtained and compared with the model parameters currently running on the edge device; through algorithm analysis, the differences between the two, i.e., the changed parameters, are identified.

[0068] A task queue is used to manage push tasks, ensuring that changed parameters are pushed to edge devices in an orderly manner according to priority and load within a specified time period. To cope with large-scale concurrent pushes, the system employs concurrency control strategies, such as limiting the number of devices pushing simultaneously to maintain system stability; only the changed parameter portions are transmitted, rather than the entire model, thereby significantly reducing data transmission volume. For large models, the system uses parameter block transmission, appending a checksum to each block to ensure the accuracy of incremental updates.

[0069] Each push generates a new version number and updates the edge devices. The server retains all historical version information so users can roll back versions when needed. Edge devices can selectively retain which version of the model to use for quick rollback without retransmission. A parameter change log is maintained, recording detailed changes for each update, including update time, updated parameters, and updated version. A real-time monitoring interface is provided, allowing users to view key information such as push progress and success rate. Detailed push logs are automatically recorded, including the start and end times of the push task, the list of devices pushed to, and the push result (success / failure). This log information is crucial for troubleshooting and performance optimization.

[0070] Through the above process, the changed parameters of the remote server model can be transmitted to the target edge device efficiently and accurately, while providing rich version management and logging functions, providing strong support for users' model deployment and operation and maintenance.

[0071] See attached document Figure 3 In some embodiments, comparing the parameter differences between the remote service model and the edge device model to obtain the changed parameters includes:

[0072] For the lightweight model of edge devices, a parameter mapping table is constructed to obtain the mapping relationship between the edge device model and the corresponding full model of remote services;

[0073] If multiple versions of the lightweight model exist for the edge device, the latest full remote service model is compared with the full model versions corresponding to the multiple versions of the lightweight model for the edge device to obtain the minimum parameter difference.

[0074] Based on the parameter mapping table and the minimum parameter difference, the variable parameters of the lightweight edge device model are obtained.

[0075] The working principle of the above technical solution is as follows: First, the structure of the full model in the remote service and the lightweight model on the edge device are analyzed; this includes, but is not limited to, identifying the layers of the model, the parameters of each layer (such as weights and biases), activation functions, etc.; based on the structural similarity and functional correspondence between the two, mapping rules between parameters are defined; for example, the lightweight model may be derived from the full model through pruning or quantization, so mapping needs to be created according to these transformation rules. Using the above rules, the corresponding item in the full model is found for each lightweight model parameter, and this mapping relationship is recorded in a parameter mapping table. This table can be a data structure containing the lightweight model parameter ID, the corresponding full model parameter ID, and any necessary transformation information (such as scaling factors).

[0076] For edge devices with multiple versions of the lightweight model, determine the minimum parameter difference. Identify all available lightweight model versions on the edge device; map each lightweight model version back to the corresponding full model version according to the parameter mapping table. If possible, directly use the stored historical full model version to save computational resources. Compare the latest remote service full model with each reconstructed full model version, calculating the parameter differences between them. This process can be a layer-by-layer, parameter-by-parameter comparison of numerical differences, or a more advanced algorithm to measure changes in overall model performance; select the smallest difference from all calculated differences, i.e., the version closest to the latest remote service full model. This version of the lightweight model is considered the most suitable baseline for the current update; extract the specific changed parameters of the lightweight model according to the parameter mapping table and the minimum parameter difference; based on the result of the minimum parameter difference, determine which parameters have changed; use the parameter mapping table to map these changes back to the specific parameter locations of the lightweight model. This is because what ultimately needs updating is the lightweight model on the edge device, not the full model. Based on the located changed parameters, an incremental update packet is generated, containing only those parameters that have actually changed. For large models, these parameters can be further processed in chunks to ensure transmission efficiency and accuracy. Before sending the incremental update packet, a verification step is performed to ensure the correctness of the update. Once confirmed to be correct, the incremental update packet can be pushed to the edge device for application.

[0077] The effects of the above technical solution are as follows: By constructing a parameter mapping table and transmitting only the changed parameters, the amount of data that needs to be transmitted is significantly reduced. This not only speeds up the update process but also reduces bandwidth consumption. Since only the changed parts need to be processed, edge devices can apply updates more quickly, reducing downtime and resource consumption. Edge devices typically have limited resources; incremental updates instead of re-downloading the entire model effectively save storage space. Updating only the changed parts on the edge device reduces unnecessary computational work, improving device response speed and energy efficiency. In some cases, the latest full remote service model may not be updated based on the previous model but on an earlier version (such as the model from two years ago). Directly comparing the latest full remote service model with the most recently deployed lightweight model on the edge device may lead to significant parameter differences, increasing the amount of data and complexity of incremental updates. Therefore, by comparing multiple versions of the lightweight model and selecting the version closest to the latest full remote service model for updating, the stability and compatibility of the model update are ensured, the risk of update failure is reduced, and the number of changed parameters that need to be transmitted is correspondingly reduced. The update process is completed faster, reducing waiting time and resource consumption on the edge device.

[0078] See attached document Figure 4In some embodiments, setting a push time period, during which the changed parameters are transmitted to the edge device, includes:

[0079] Based on the changed parameters and historical transmission duration, determine the predicted transmission duration for this change in parameters;

[0080]

[0081] Among them, T cy D is the predicted transmission duration for the changed parameters. c This refers to the magnitude of the parameter change; D i T represents the magnitude of the i-th change parameter in historical statistical data. i T represents the actual transmission time of the i-th changed parameter in the historical data; iy is the prediction transmission duration for the i-th parameter change in historical data; n is the number of model updates; wi is the first weight coefficient; q is the index, which is a positive integer from 1 to n.

[0082] Based on the predicted transmission duration of the changed parameters, a first time window is set; the first time window T w Satisfy: k1×T cy ≤T w ≤k2×T cy Where k1 and k2 are coefficients, 1≤k1≤3, 3≤k2≤10, k1 <k2;

[0083] Divide a day into multiple first-time windows; predict the load and business demand of edge devices in each first-time window based on historical data;

[0084] The push time period is determined based on the load of edge devices and business needs within the first time window.

[0085] In some embodiments, determining the push time period based on edge device load and service requirements includes:

[0086] If business requirements allow for updates, and the load on the edge devices in the first time window is less than the preset load, then the first time window is marked.

[0087] If P consecutive first time windows are marked, then the P consecutively marked first time windows are taken as a first time window group;

[0088] If there are P marked first time windows, and the time interval between any two adjacent marked windows is less than a preset interval threshold, then the P first time windows are combined into a first time window group.

[0089] The initial time of the first time window group closest to the remote service model update is used as the initial transmission start time.

[0090] If multiple edge devices start transmission at the same time, the transmission tasks will be sorted by priority.

[0091] Within the same priority level, each transmission task is sorted in ascending order of its transmission volume;

[0092] Based on the second sorting result and the current network load, the number of tasks that can be processed simultaneously in each batch is dynamically determined.

[0093] If at any given time there are multiple tasks in the second order (such as the first 5) that can be safely processed under the current network conditions, then these tasks will immediately begin transmission;

[0094] For subsequent tasks (such as the 6th and thereafter), check whether the current network load and the remaining first time window group meet the preset conditions before each new task starts;

[0095] If a previous task has been completed, and the addition of a new task (such as the 6th task) will not cause the network load to exceed the safety threshold, and the remaining first time window of the task still meets the requirements of the predicted transmission time, then the transmission of the next task will start immediately; if the new task does not meet the requirements, then the task will be skipped and the next task to be processed will be evaluated.

[0096] If subsequent tasks meet the preset duration requirements but not the network load requirements, then wait for more resources to be released.

[0097] Once the network load allows new tasks to be added without exceeding the safety threshold, the system will re-examine the remaining pending tasks and immediately start the eligible tasks.

[0098] For tasks that fail to be transmitted within the first time window group due to resource constraints, they are marked as "pending" and their original priority, transmission volume, and other information are recorded. If a task fails to be transmitted in more than two time window groups, its priority and / or second sorting order are increased.

[0099] The working principle of the above technical solution is as follows:

[0100] First, the predicted transmission time of the current parameter change is calculated based on the magnitude of the parameter change and historical statistical data (including the magnitude of previous parameter changes and their actual transmission time, as well as the predicted transmission time of previous parameter changes, although these may not be directly used in this decision-making process, they can be used to evaluate the accuracy of the prediction model). The formula comprehensively considers the linear relationship between parameter magnitude and transmission time, as well as the deviation of historical transmission, in order to improve the accuracy of the prediction. By introducing weighting coefficients to adjust the impact of each historical transmission on the current prediction, the weighting coefficients adopt an exponential decay form, giving higher weight to historical data closer to the present. When calculating the predicted transmission duration, the transmission performance of recent model updates is given special consideration. This aligns with the dynamic changes in the actual network environment and device performance, as recent conditions are more relevant to the current prediction. This effectively reduces prediction bias caused by interference from distant historical data, making the predicted duration closer to the actual value. The deviation between the actual duration and the predicted duration in historical transmissions is incorporated, and together with the estimation based on historical transmission efficiency in the first half, the predicted value is corrected from different dimensions to compensate for the shortcomings of a single estimation method and further improve the prediction accuracy. k1 and k2 are coefficients used to set the range of the first time window, providing a reasonable time range based on the predicted duration to ensure that the update process has sufficient flexibility to accommodate possible network fluctuations, temporary device lag, and other unexpected situations. This ensures that the transmission has sufficient buffer time but is not excessively long.

[0101] The day is divided into multiple first-time windows to assess the load and service demands of edge devices at different times. Historical data is used to predict the expected load and service demands of edge devices within each first-time window. Expected load includes key indicators such as CPU utilization and memory usage. Each first-time window is checked individually. If the service demand during that period allows for updates (i.e., the update operation will not interfere with core business operations) and the device load is below a preset load, then the window is marked as available. If P consecutive first-time windows are marked as available, these consecutive windows are combined into a first-time window group, indicating that updates are more appropriate within these time periods. Similarly, if P marked windows exist and the time interval between them is less than a preset interval threshold, these windows can also be combined into a first-time window group to utilize fragmented available time.

[0102] From all the marked and combined window groups, select the initial time of the first time window group closest to the remote service model update as the start time of the transmission. This ensures that the model update is as timely as possible while minimizing the impact on the normal operation of edge devices.

[0103] The effects of the above technical solution are as follows: By using a prediction formula based on historical data and the magnitude of current changing parameters, the system can more accurately estimate the transmission time, reducing unnecessary waiting time and ensuring efficient resource utilization. By considering the difference between historical transmission duration and predicted duration, the impact of short-term fluctuations on the prediction results can be effectively reduced, providing more stable prediction performance. By combining historical transmission data and exponential decay weights, the transmission duration of current changing parameters can be predicted more accurately, reducing prediction deviations caused by changes in network conditions. The time window set based on the predicted transmission duration has sufficient flexibility to adapt to changes in network fluctuations and edge device processing capabilities, ensuring that the transmission process can proceed under optimal conditions, thereby improving overall efficiency. Utilizing historical data to predict the load and service demands of edge devices allows the system to transmit during periods of low load, avoiding resource contention and performance degradation that may occur during peak periods. By marking and combining first-time windows, and forming first-time window groups, selecting the group closest to the remote service model update as the starting point, the system balances update timeliness and device stability. On the one hand, it avoids delays in updates due to excessive waiting for the ideal window, allowing new model parameter changes to be applied to edge devices as quickly as possible. On the other hand, the window group strategy ensures that devices are likely in a suitable update state when the push begins, guaranteeing the continuity and stability of critical services while optimizing resource utilization. By intelligently selecting push time periods, the system reduces the risk of transmission failures and improves system reliability. It can dynamically adjust the push strategy based on historical data and real-time conditions to adapt to different network environments, edge device configurations, and business needs. By combining priority sorting and transmission volume sorting, it ensures the effective utilization of network resources. High-priority and low-transmission-volume tasks are processed first, reducing the long-term network bandwidth occupation of large tasks and improving overall resource utilization efficiency. The system dynamically determines the number of tasks that can be processed simultaneously in each batch and adjusts them according to real-time network load. This allows the system to flexibly respond to different network conditions and avoid service interruptions or delays caused by network congestion. Before each new task starts, check whether the current network load and the remaining first time window group meet the preset conditions to ensure that the task starts at the most appropriate time, without wasting time or causing resource overload.Tasks that fail to be transmitted on time are marked and their relevant information is recorded. If they are still not transmitted after more than two time windows, their priority is increased and / or their second sorting order is raised, ensuring that critical tasks are not postponed indefinitely and enhancing system reliability and user experience. When subsequent tasks meet the preset duration requirements but not the network load requirements, the system waits for more resources to be released before evaluating the start conditions, effectively reducing unnecessary waiting time and potential network congestion problems. The system continuously monitors network conditions and task completion status. Once the network load allows new tasks to be added without exceeding the safety threshold, the system immediately starts the eligible tasks. This instant response mechanism improves overall transmission efficiency and shortens the total task completion time. "Pending" tasks are sorted together with new tasks to ensure that incomplete tasks are not ignored, while also allowing new tasks to start as soon as conditions permit, achieving a balance between fairness and efficiency.

[0104] In some embodiments, the step of selecting the model version to retain at the edge device includes:

[0105] The maximum number of edge device models to be retained is determined based on the edge device's capacity, computing resources, model capacity, and model energy consumption.

[0106]

[0107] Among them, g max M represents the maximum number of devices to retain in the edge device model. compute The average computational resources required to run a single edge model; this can be obtained from historical data or model evaluation; M capacity M represents the average capacity of a single model. energy E represents the average energy consumption of a single model running on an edge device. compute Computing resources for edge devices (expressed in some quantifiable metric, such as floating-point operations per second, FLOPS); E energy R represents the total energy consumption that an edge device can withstand within a specific time period (e.g., a day); ratio The reserved resource ratio is in the range of (0,1), and α is a coefficient in the range of (0,1). This represents the total capacity of the currently retained m models. The total computing resource requirements for the currently retained m models are as follows: This represents the total energy consumption of the currently retained m models; floor() rounds down; E capacity Total capacity of edge devices;

[0108] The model score is determined based on the historical prediction results of the edge model and the rollback selection results;

[0109] The model version retained by the edge device is determined based on the model score and the model update frequency.

[0110] The model score is as follows:

[0111] S u =x1×Pa u +x2×Pmax u +x3×Pmin u +x4×Rc u -x5×Rbc u

[0112] Where x1, x3, x4, and x5 are the second weighting coefficients, Pacc u Pmax represents the average accuracy of the u-th model over the past h predictions. u pmin represents the maximum accuracy of the u-th model in the past h predictions. u Rc represents the minimum accuracy of the u-th model in the past h predictions. u Rbc is the normalized value representing the number of times the model is rolled back from other models to the u-th model. u This is a normalized value representing the number of times the model (u-th model) is rolled back to other models.

[0113] Based on the model scores from highest to lowest, the top g models are selected as the model versions to be retained for edge devices;

[0114]

[0115] g min The minimum number of models to retain is the preset minimum; fa is the model update frequency, fy is the preset update frequency, and g is the preset minimum number of models to retain. max This represents the maximum number of devices to retain in the edge device model. min() takes the minimum value, and ceiling rounds up.

[0116] The working principle of the above technical solution is as follows: the computing resources, total capacity, and total energy consumption that the edge device can withstand are taken as input; the average computing resources, average capacity, and average energy consumption required for the operation of a single edge model are calculated; the available amount of each resource (capacity, computing resources, energy consumption) is calculated, and the minimum available amount of the three resources is taken as the bottleneck resource, considering the reserved resource ratio; the maximum number of models to be retained is calculated based on the bottleneck resource and the adjustment factor α; the reserved resource ratio is set to the resource ratio reserved by the edge device to prevent complete full load.

[0117] The model's performance stability is evaluated based on its average accuracy, maximum accuracy, and minimum accuracy over the past h predictions. The number of times the model was rolled back and the number of times it was removed from the model are statistically analyzed and normalized.

[0118] Based on the model scores from highest to lowest, the top g models are selected as the model versions retained for edge devices.

[0119] If the update frequency is lower than the preset frequency, then retain g. min There are 1 model; if the update frequency is higher than the preset frequency, the number of models to be retained is increased according to the frequency difference, but not exceeding 1g. max .

[0120] The effect of the above technical solution is that it accurately calculates the maximum number of models that the edge device can safely retain through the formula, ensuring the effective use of device resources and avoiding resource waste or excessive occupation.

[0121] Through a model scoring mechanism, considering the model's prediction accuracy, stability, and rollback behavior, high-performance model versions with reasonable resource consumption are selected for retention, achieving a balance between performance and resource consumption. Reserved resource proportions ensure sufficient resources to handle sudden demands even in extreme situations, improving system stability and reliability. A new version number is generated for each model update, and historical version information is retained, enabling edge devices to quickly roll back to previous stable versions as needed, enhancing the flexibility and security of model updates. Prioritizing the retention of stable model versions with fewer rollbacks on edge devices reduces the risks associated with unstable model updates. Selecting the optimal model version for retention on edge devices ensures efficient service response and improves user experience. Retaining multiple model versions on edge devices ensures a rapid switch to other stable versions even if a new model version encounters problems, guaranteeing business continuity and stability.

[0122] In some embodiments, the unattended configuration of the edge device model operating environment through intelligent detection, automated script generation, and remote execution includes:

[0123] Match the Agent program for the edge device based on the edge operating system type;

[0124] The Agent program is used to intelligently detect the operating environment of edge devices, identify installed software and libraries, and identify missing dependencies.

[0125] Upon receiving server-side instructions from the remote controller, the Agent program intelligently selects the appropriate version based on model dependency requirements, resolves version conflicts or compatibility issues, and generates and executes an environment installation script adapted to the edge device.

[0126] If the edge device environment configuration fails, it automatically rolls back to the stable state before configuration, records the environment configuration log, and uploads the environment configuration log to the server.

[0127] The working principle and effect of the above technical solution are as follows: First, the operating system type of the edge device is identified (such as Linux distribution, Windows IoT, etc.). Based on the operating system type, the most suitable Agent program is automatically selected and deployed. This Agent program has the following functions:

[0128] Automatically detects the software and libraries installed on edge devices and identifies missing dependencies.

[0129] Based on the model's dependencies and the environment of the edge devices, an adapted installation script is automatically generated.

[0130] When encountering version conflicts or compatibility issues, it can intelligently select the appropriate version or provide a solution.

[0131] The Agent program performs a comprehensive scan of the edge device's operating environment to ensure all necessary dependencies are correctly identified. This includes: after the Agent program starts, it automatically scans the current environment of the edge device, recording installed software, libraries, and their version information. Based on the requirements of the model to be deployed, it analyzes the required dependencies and compares them with the existing environment to identify any missing or incompatible dependencies. It also checks whether the software and library versions in the existing environment conflict with the requirements of the new model, proactively preventing potential problems.

[0132] The system sends environment installation instructions to the edge device via remote commands, automatically generating and executing environment installation scripts. This includes: after receiving an environment installation request from a user or system, the remote controller sends instructions to the Agent program on the corresponding edge device. The Agent program, based on the received instructions, model dependencies, and environment monitoring results, automatically generates an appropriate installation script. The Agent program executes the generated script, automatically installing the necessary runtime environment (such as Python, PyTorch, etc.) on the terminal system, using package management tools suitable for the operating system (such as apt, yum, pip, etc.). During installation, the Agent program records detailed installation progress and status in real time and uploads these logs to the server. Users can view the installation progress on a web page, ensuring the stability and recoverability of the environment configuration, recording each configuration operation in detail, and supporting automatic rollback in case of failure.

[0133] The agent program meticulously records the operation content, time, and results at each configuration step, forming a complete environment configuration log. It monitors state changes in real time during the configuration process, promptly identifying and addressing potential issues. If errors or failures occur during configuration, the agent program automatically rolls back to a previous stable state, ensuring that existing services are not affected by configuration failures. All configuration logs are uploaded to the server for subsequent auditing and troubleshooting.

[0134] In some embodiments, the method of deploying platform tools on edge devices to intelligently generate configuration files and achieve seamless deployment and optimization of model inference services on edge devices includes:

[0135] Based on the edge device information, deploy the appropriate version of the platform tools on the edge device;

[0136] Based on model characteristics and edge device resources, an optimal configuration file is generated; the configuration file includes resource allocation and concurrency settings.

[0137] Remotely start the inference service, monitor the service status in real time, and set up a heartbeat monitoring mechanism;

[0138] Automatically diagnose and recover from inference service startup failures or operational anomalies;

[0139] Collect and analyze operational data from edge device inference services, and provide optimization suggestions based on the analysis results.

[0140] The working principle and effects of the above technical solution are as follows: First, the operating system type, hardware configuration (such as CPU architecture, memory size, storage space, etc.) and other relevant information of the edge device are collected. Based on this information, the developed automatic deployment script automatically selects the most suitable version of platform tools such as Ollama and PyTorch for the device. This ensures that the selected version is compatible with the device hardware and operating system to maximize performance and stability. Through a remote execution mechanism, the selected version of the platform tools is automatically deployed to the edge device, reducing manual intervention and accelerating deployment speed. The characteristics of the model to be deployed are analyzed, including but not limited to model size, input / output format, and computational complexity. The available resources of the edge device are evaluated, such as the number of CPU cores, memory capacity, and network bandwidth. Using an automatic configuration file generator, the optimal configuration file is generated by comprehensively considering model characteristics and device resources. This configuration file includes, but is not limited to, the reasonable allocation of CPU, memory, and other resources to ensure the efficient operation of the model inference service. Based on the device capabilities and model requirements, the optimal concurrent processing capacity is set to improve throughput and response speed. The inference service deployed on the edge device is remotely started from the central management system via remote commands or API interfaces. A real-time monitoring mechanism is introduced to continuously monitor the status of the inference service, including key indicators such as CPU utilization, memory usage, and network traffic. A heartbeat monitoring mechanism is set up to periodically check the service's health and ensure its continuous online operation. If an anomaly is detected, an alert is immediately triggered or appropriate measures are taken.

[0141] Once the monitoring mechanism detects a service startup failure or operational anomaly, it immediately triggers the error handling process. The system automatically performs fault diagnosis, analyzing logs, error codes, and other relevant data to determine the cause of the problem. Based on the diagnostic results, the system attempts to automatically repair the problem, such as restarting the service, adjusting resource configurations, or rolling back to a stable version. For certain types of errors, the system can retry multiple times according to a preset strategy until the service returns to normal or the maximum number of retries is reached.

[0142] The system periodically collects operational data from the inference service, including but not limited to performance metrics, resource utilization, and latency. Through a built-in data analysis module, the collected data is analyzed in depth to identify potential performance bottlenecks and areas for improvement. Based on the analysis results, the system provides users with specific performance optimization suggestions, such as adjusting configuration parameters, upgrading hardware, and optimizing model structure. Users can then make adjustments based on these suggestions to further improve the performance and efficiency of the inference service, forming a closed loop of continuous improvement.

[0143] The above methods not only enable seamless deployment of model inference services on edge devices, but also provide comprehensive monitoring, fault handling, and performance optimization functions, reducing manual intervention and improving service reliability and efficiency.

[0144] In some embodiments, the API interface and API documentation for automatically generating edge device models include:

[0145] Based on the input and output characteristics of the model, automatically generate API code that conforms to the RESTful specification;

[0146] The API program auto-deployer encapsulates the automatically generated API code into a container and automatically pushes it to the cloud platform; the API service is started through containerization.

[0147] The API service is invoked remotely via an edge agent program;

[0148] Automatically generate API documentation for edge device models;

[0149] When the model is updated, the relevant APIs are automatically updated.

[0150] The working principle of the above technical solution is as follows: Based on different types of models (such as image classification, text generation, etc.), corresponding API templates are provided, including: establishing an API template library containing multiple model types (such as image classification, text generation, speech recognition, etc.), each template predefines common API endpoints, request methods, parameter structures and response formats; based on the model type provided by the user, the most suitable API template is automatically matched to ensure that the generated API code conforms to the RESTful specification and meets the needs of the specific model.

[0151] Based on the model's input and output characteristics, RESTful API code is automatically generated. This includes extracting input and output features from the model, such as data format, parameter types, and expected results. Based on the selected API template and the extracted feature information, complete API code is automatically generated, including functional modules such as route definition, request processing logic, data validation, and error handling. The generated API code strictly adheres to RESTful design principles, such as using HTTP verbs (GET, POST, PUT, DELETE, etc.) and resource path naming rules.

[0152] The system automatically encapsulates the generated API code into containers and pushes them to the PaaS cloud platform for containerized startup. Remote calls are then made via edge agents. This includes packaging the generated API code and its dependencies into Docker images or other container formats to ensure consistent operation in any container-enabled environment. An API auto-deployer automatically pushes the container images to the PaaS cloud platform (such as a Kubernetes cluster); the deployer manages the container lifecycle, including startup, shutdown, and scaling. The API service is launched in containerized form on the cloud platform, ensuring service isolation and stability. Remote calls to the API service via edge agents enable seamless integration of cloud and edge inference services. User inference service requests are forwarded to the edge AI model for processing, and the results are automatically sent back to the cloud after inference.

[0153] The system automatically generates API documentation for edge device models, ensuring the documentation is detailed and easy to understand. This includes using Swagger or similar tools to automatically generate API documentation. These tools can parse comments and metadata in the API code and automatically generate detailed documentation content. The generated API documentation includes interface descriptions, parameter descriptions, request examples, response formats, etc., ensuring that developers can quickly understand and use the API. Whenever the API code changes, the documentation generation system automatically regenerates the latest documentation, ensuring that the documentation is always in sync with the actual API.

[0154] A new version number is generated for each API update, and historical version information is retained. Each version has an independent API path (e.g., / v1, v2, etc.), ensuring that different versions can coexist without interference. When designing a new API version, existing clients are ensured not to be affected. For non-destructive changes, the availability of older versions is maintained as much as possible; for destructive changes, a reasonable transition plan is provided. When the model is updated, the API code generator automatically detects the changes and updates the relevant APIs. The new API version is automatically deployed and released according to a predetermined process, ensuring a smooth and uninterrupted update process.

[0155] This application provides a system for implementing the automated deployment and updating method for edge AI models according to embodiments of this application, the system comprising:

[0156] The edge service management module is used to obtain edge device information through the user interface; and to manage the edge device information that users need to deploy in a unified manner through grouping and tagging settings.

[0157] The model push module is used for an automatic push mechanism based on time periods to transmit the changing parameters of the remote server model to the target edge device.

[0158] The environment configuration module is used to enable unattended configuration of the operating environment of edge device models through intelligent detection, automated script generation and remote execution;

[0159] The inference service deployment module is used to intelligently generate configuration files by deploying platform tools on edge devices, thereby enabling seamless deployment and optimization of model inference services on edge devices.

[0160] The API auto-generation module is used to automatically generate API interfaces and API documentation for edge device models.

[0161] This application also provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described in this application.

[0162] This application describes the invention from the perspectives of purpose, effectiveness, progress, and novelty, and it meets the functional enhancement and use requirements emphasized by the Patent Law. The above description and accompanying drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

Claims

1. A method for automated deployment and updating of edge AI models, characterized in that, The method includes: Obtain edge device information through the user interface; manage the edge device information that users need to deploy in a unified manner through grouping and tagging settings; Compare the parameter differences between the remote service model and the edge device model to obtain the changing parameters; based on an automatic push mechanism over a time period, transmit the changing parameters of the remote server model to the target edge device; Through intelligent detection, automated script generation, and remote execution, unattended configuration of the edge device model's operating environment can be achieved; By deploying platform tools on edge devices, configuration files are intelligently generated, enabling seamless deployment and optimization of model inference services on edge devices; Automatically generate API interfaces and API documentation for edge device models; The comparison of parameter differences between the remote service model and the edge device model to obtain the changing parameters includes: For the lightweight model of edge devices, a parameter mapping table is constructed to obtain the mapping relationship between the edge device model and the corresponding full model of remote services; If multiple versions of the lightweight model exist for the edge device, the latest full remote service model is compared with the full model versions corresponding to the multiple versions of the lightweight model for the edge device to obtain the minimum parameter difference. Based on the parameter mapping table and the minimum parameter difference, the variable parameters of the lightweight edge device model are obtained; The method achieves unattended configuration of the edge device model's operating environment through intelligent detection, automated script generation, and remote execution; including: Match the Agent program for the edge device based on the edge operating system type; The Agent program is used to intelligently detect the operating environment of edge devices, identify installed software and libraries, and identify missing dependencies. Upon receiving server-side instructions from the remote controller, the Agent program intelligently selects the appropriate version based on model dependency requirements, resolves version conflicts or compatibility issues, and generates and executes an environment installation script adapted to the edge device. If the edge device environment configuration fails, it automatically rolls back to the stable state before configuration, records the environment configuration log, and uploads the environment configuration log to the server.

2. The method according to claim 1, characterized in that, The time-based automatic push mechanism transmits the changing parameters of the remote server model to the target edge device; including: Set a push time period, and within the push time period, transmit the changed parameters to the edge device; A new version number is generated for each model update, and historical version information is retained on the remote server. The model version to be retained is selected on the edge device. In response to a rollback request, the edge device model is rolled back to the target version; Monitor the model update status in real time and record the change log.

3. The method according to claim 2, characterized in that, The step of setting a push time period, during which the changed parameters are transmitted to the edge device, includes: Based on the changed parameters and historical transmission duration, determine the predicted transmission duration for this change in parameters; Based on the predicted transmission duration of the changed parameters, a first time window is set; the first time window satisfy: Where k1 and k2 are coefficients, 1 k1 3,3 k2 10, k1 <k2; The predicted transmission time for this parameter change; Divide a day into multiple first-time windows; predict the load and business demand of edge devices in each first-time window based on historical data; The push time period is determined based on the load of edge devices and business needs within the first time window.

4. The method according to claim 3, characterized in that, The first time window is based on the edge device load and service requirements; Determine the push notification time period, including: If business requirements allow for updates, and the load on the edge devices in the first time window is less than the preset load, then the first time window is marked. If P consecutive first time windows are marked, then the P consecutively marked first time windows are taken as a first time window group; If there are P marked first time windows, and the time interval between any two adjacent marked windows is less than a preset interval threshold, then the P first time windows are combined into a first time window group. The initial time of the first time window group closest to the remote service model update is used as the initial transmission start time.

5. The method according to claim 2, characterized in that, The selection of the model version to be retained on the edge device includes: The maximum number of edge device models to be retained is determined based on the edge device's capacity, computing resources, model capacity, and model energy consumption. The model score is determined based on the historical prediction results of the edge model and the rollback selection results; The model version retained by the edge device is determined based on the model score and the model update frequency. Based on the model scores from highest to lowest, the top g models are selected as the model versions to be retained for edge devices; The minimum number of models to retain is preset; For model update frequency, To preset the update frequency, This represents the maximum number of edge device models to retain, and min() is used to find the minimum value. This is for rounding up.

6. The method according to claim 1, characterized in that, The method involves deploying platform tools on edge devices to intelligently generate configuration files, enabling seamless deployment and optimization of model inference services on edge devices; including: Based on the edge device information, deploy the appropriate version of the platform tools on the edge device; Based on model characteristics and edge device resources, an optimal configuration file is generated; the configuration file includes resource allocation and concurrency settings. Remotely start the inference service, monitor the service status in real time, and set up a heartbeat monitoring mechanism; Automatically diagnose and recover from inference service startup failures or operational anomalies; Collect and analyze operational data from edge device inference services, and provide optimization suggestions based on the analysis results.

7. The method according to claim 1, characterized in that, The API interface and API documentation for automatically generating edge device models include: Based on the input and output characteristics of the model, automatically generate API code that conforms to the RESTful specification; The API program auto-deployer encapsulates the automatically generated API code into a container and automatically pushes it to the cloud platform; the API service is started through containerization. The API service is remotely invoked through the edge agent program; API documentation for the edge device model is automatically generated.

8. A system for implementing the automated deployment and update method for edge-oriented AI models as claimed in any one of claims 1-7, characterized in that, The system includes: The edge service management module is used to obtain edge device information through the user interface; and to manage the edge device information that users need to deploy in a unified manner through grouping and tagging settings. The model push module is used for an automatic push mechanism based on time periods to transmit the changing parameters of the remote server model to the target edge device. The environment configuration module is used to enable unattended configuration of the operating environment of edge device models through intelligent detection, automated script generation and remote execution; The inference service deployment module is used to intelligently generate configuration files by deploying platform tools on edge devices, thereby enabling seamless deployment and optimization of model inference services on edge devices. The API auto-generation module is used to automatically generate API interfaces and API documentation for edge device models.

Images