Model end-side batch deployment and intelligent operation and maintenance method and system in autonomous controllable environment

Abstract

The embodiment of the invention provides a model end-side batch deployment and intelligent operation and maintenance method and system in an autonomous controllable environment, and belongs to the technical field of model deployment. The model end-side batch deployment and intelligent operation and maintenance method in the autonomous controllable environment comprises the following steps: acquiring a trained model file and domestic hardware; constructing a plurality of deployment packages according to the trained model file and the domestic hardware; and distributing the deployment packet to a target device corresponding to the target end side. According to the method, model files and domestic hardware are combined to construct deployment packages respectively, and the deployment packages are distributed to corresponding target equipment in a targeted manner, so that the precision, reliability and efficiency of batch deployment of models can be effectively improved; the dynamic monitoring of the target equipment can be realized in a mode of monitoring and uploading the key indexes of the target equipment through the lightweight proxy, so that the dynamic adjustment of deployment is realized, and the deployment precision and the operation and maintenance precision of the target equipment are further improved.

Description

Technical Field

[0001] This invention relates to the field of model deployment technology, and more specifically to a method and system for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment. Background Technology

[0002] With the large-scale application of artificial intelligence technology in key industries such as power and industrial manufacturing, deploying trained lightweight AI models (such as substation equipment fault identification models and wildfire early warning models) to a massive number of edge devices (such as inspection terminals, cameras, and edge servers) has become a crucial step in achieving business intelligence. However, under the background of emphasizing technological self-reliance and control, environments based on domestic hardware (such as Huawei Ascend and Hygon DCU) and domestic operating systems (such as UnionTech UOS) face severe challenges.

[0003] Currently, most mainstream deployment methods rely on manual operation or general software deployment tools, which have the following inherent drawbacks: 1. Low deployment efficiency: The number of edge devices is huge, their geographical locations are scattered, and their hardware is heterogeneous (domestic AI accelerators with different architectures). Manually configuring the environment, installing models and dependency libraries for each device is time-consuming and labor-intensive, making it impossible to achieve rapid large-scale promotion.

[0004] Second, poor compatibility with domestic environments: General deployment tools are not deeply optimized for domestic chip drivers and computing power libraries (such as Ascend's CANN), which makes it impossible for models to fully utilize hardware acceleration features, and even causes incompatibility and inability to run.

[0005] 3. Insufficient Intelligent Operation and Maintenance Management: After model deployment, there is a lack of effective monitoring methods. It is impossible to perceive the model's operating status on the edge in real time (such as inference performance and hardware resource usage), and it is impossible to detect model performance degradation, data drift, or hardware failures in a timely manner. Operation and maintenance rely entirely on manual inspections, which is slow and costly.

[0006] In the prior art, such as Chinese invention patent application number 202410761684.0, "A Method and System for Deployment Management of Edge Computing Devices," a deployment scheme based on edge devices is disclosed. While this scheme achieves device management and deployment, it has the following problems: Weak environmental adaptability: Its container image does not take into account the differences of domestic heterogeneous AI chips, and cannot dynamically adapt to the specific drivers and acceleration libraries required by different hardware, resulting in a low deployment success rate in mixed environments such as Ascend and Hygon.

[0007] The deployment strategy is simplistic: the batch deployment process is "one-size-fits-all" and does not use intelligent grouping and differentiated deployment strategies based on the network conditions and hardware performance of the devices, which can easily lead to network congestion or deployment failure of low-performance devices.

[0008] Lack of operational capabilities for AI models: This solution only monitors the lifecycle of the container itself, but cannot monitor business metrics (such as accuracy and latency) of AI model inference within the container, nor can it identify model-specific faults (such as abnormal input data and performance drift). The operation and maintenance dimensions are limited and the level of intelligence is low.

[0009] Meanwhile, in the field of edge computing, model compression (such as pruning) is one of the key technologies for deploying AI models on resource-constrained devices. Specifically, existing technologies, such as the Chinese invention patent application number 202511127140.X, "A Model Deployment Method Based on Pruning Compression in Edge Devices," disclose a dynamic adaptive pruning technology. This technology analyzes the hardware architecture, determines the compression ratio of each layer of the model, and reserves redundant weight blocks during pruning, enabling the model to be fine-tuned online according to the operating environment after deployment, effectively improving the adaptability of a single model on a single device.

[0010] However, this existing technology and similar solutions mainly focus on the micro-optimization of a single model. When facing large-scale application scenarios such as power line inspection and disaster early warning, which require the unified and efficient deployment and operation of multiple lightweight models among a massive and heterogeneous group of domestically produced edge devices, there are obvious limitations. The disadvantages are as follows: Lack of scalable deployment capability: This technology focuses on the deployment and optimization process of a single model, without addressing the mechanism for efficiently and reliably distributing the same model to tens of thousands of devices in batches. When faced with deployments on tens of thousands of devices, this approach requires manual or semi-automatic operation on a per-device basis, resulting in extremely low efficiency and failing to meet the needs of rapid business deployment.

[0011] In the process of realizing this invention, the inventors of this application discovered that the above-mentioned solutions in the prior art have the defects of low accuracy, poor reliability and low efficiency in batch deployment of models. Summary of the Invention

[0012] The purpose of this invention is to provide a method and system for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment. This method and system for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment has the functions of high model deployment accuracy, high reliability and high efficiency.

[0013] To achieve the above objectives, one embodiment of the present invention provides a method for batch deployment and intelligent operation and maintenance of models on the edge in an autonomous and controllable environment, including: Obtain pre-trained model files and domestically produced hardware; Multiple deployment packages are built based on the trained model file and the domestically produced hardware; The deployment package is distributed to the target device corresponding to the target endpoint. A lightweight agent runs on each of the target devices on the target end side; The lightweight agent installs and starts the deployment package; The lightweight agent monitors and uploads key metrics of the target device.

[0014] Optionally, the deployment package built based on the trained model file and the domestically produced hardware includes: Construct the basic environment image and image package for the domestically produced hardware; Obtain the hardware model of each target device in the target end-side; Obtain the corresponding basic environment image and image package based on the hardware model of the target device; The base environment image and image package of the target device are packaged, and a deployment package is obtained.

[0015] Optionally, the basic environment image and image package for building domestically produced hardware include: Obtain the identifier of the domestically produced hardware and the list of software ecosystem dependencies; A feature library of domestically produced hardware is constructed based on the identifiers of the domestically produced hardware and the software ecosystem dependency list. Construct a minimal runtime environment based on the aforementioned software ecosystem dependency list; The model file, the minimized runtime environment, and the startup script are packaged to obtain an image package, and a configuration template is used to manage the model and service configuration of the image package.

[0016] Optionally, distributing the deployment package to the target device corresponding to the target endpoint includes: Obtain the static and dynamic attributes of each target device in the target end-side; The target devices are grouped according to their static and dynamic attributes to obtain deployment groups; The different deployment packages are distributed to the corresponding deployment groups.

[0017] Optionally, distributing different deployment packages to the corresponding deployment groups includes: The deployment package is distributed using a controllable concurrency method; The token bucket algorithm is used to control the number of concurrently deployed tasks globally.

[0018] Optionally, distributing different deployment packages to the corresponding deployment groups further includes: Determine whether the deployment package is a large deployment package; If the deployment package is determined to be a large deployment package, the large deployment package is split into multiple smaller deployment package blocks; Obtain the ACK feedback speed from the target end; The size of the sliding window is determined based on the ACK feedback speed at the target end. The sliding window is used to transmit multiple deployment packet chunks.

[0019] Optionally, the static attributes include: hardware architecture, type and number of AI accelerators, memory / storage capacity, and operating system version.

[0020] Optionally, the dynamic attributes include: current network bandwidth, real-time CPU / memory load, and geographical region.

[0021] On the other hand, the present invention also provides a system for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment, comprising: The target-side module includes multiple target devices; A cloud management platform, connected to the target edge module, is used to execute the batch deployment and intelligent operation and maintenance method of the model edge in an autonomous and controllable environment as described above.

[0022] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to cause the machine to execute the model edge-side batch deployment and intelligent operation and maintenance method in an autonomous and controllable environment as described above.

[0023] Through the above technical solution, the method and system for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment provided by this invention acquires trained model files and domestically produced hardware. Based on these model files and the types of domestically produced hardware, deployment packages suitable for different domestically produced hardware can be constructed. Multiple deployment packages can then be distributed to different target devices on the corresponding target edge. Simultaneously, a lightweight agent runs on each target device to install and start the distributed deployment packages, and monitors and uploads key indicators of the target device. By combining model files and domestically produced hardware to construct deployment packages and distributing them specifically to corresponding target devices, the accuracy, reliability, and efficiency of batch model deployment can be effectively improved. The monitoring and uploading of key indicators of the target devices through lightweight agents enables dynamic monitoring of the target devices, facilitating dynamic adjustments to the deployment and further improving the deployment and operation and maintenance accuracy of the target devices.

[0024] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0025] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a method for batch deployment and intelligent operation and maintenance of models on the edge in an autonomous and controllable environment according to an embodiment of the present invention. Figure 2 This is a flowchart of constructing a deployment package in a method for batch deployment and intelligent operation and maintenance of models in an autonomous and controllable environment according to an embodiment of the present invention. Figure 3 This is a flowchart of the image package construction process in the method for batch deployment and intelligent operation and maintenance of models in an autonomous and controllable environment according to an embodiment of the present invention. Figure 4 This is a flowchart of deployment package distribution in a method for batch deployment and intelligent operation and maintenance of models in an autonomous and controllable environment according to an embodiment of the present invention. Figure 5 This is a flowchart of the deployment package distribution strategy in a method for batch deployment and intelligent operation and maintenance of models in an autonomous and controllable environment according to an embodiment of the present invention. Figure 6 This is a flowchart of the distribution of large deployment packages in a method for batch deployment and intelligent operation and maintenance of models in an autonomous and controllable environment according to an embodiment of the present invention. Figure 7 This is a structural block diagram of a model edge-side batch deployment and intelligent operation and maintenance system under an autonomous and controllable environment according to an embodiment of the present invention. Detailed Implementation

[0026] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0027] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0028] Figure 1 This is a flowchart of a method for batch deployment and intelligent operation and maintenance of models on the edge in an autonomous and controllable environment according to an embodiment of the present invention. Figure 1 In this autonomous and controllable environment, the method for batch deployment and intelligent operation and maintenance of models on the edge can include: In step S1, the trained model file and domestically produced hardware are obtained. The trained model file may include a lightweight AI model file, such as a .onnx file or an Ascend-compatible .om file, which is received by a unified model packaging center in the cloud.

[0029] In step S2, multiple deployment packages are built based on the trained model files and domestically produced hardware. Since different domestically produced hardwares have varying compatibility, deployment packages can be built separately for different domestic hardware. Specifically, the unified model packaging center pre-configures basic environment images for different domestic hardware architectures (ARM for Ascend, x86 for Hygon DCU) and operating systems (UOS), which include necessary runtime libraries (such as Ascend CANN Toolkit and Hygon DTK). Simultaneously, a domestic hardware feature library is also built, used to package the model files and other components into deployment packages.

[0030] In step S3, the deployment package is distributed to the corresponding target device on the target side. Since there are multiple target devices on the target side, and different target devices have different types of domestically produced hardware, the multiple target devices on the target side can be classified and grouped, and the deployment package can be distributed to the corresponding target device or group of target devices.

[0031] In step S4, a lightweight proxy is run on each target device on the target side. Specifically, a lightweight proxy module can be configured / set up on each target device on the target side. This lightweight proxy module can be understood as an application used to send and receive requests, and is relatively lightweight.

[0032] The agent employs a microkernel architecture of "core + plugins," enabling hot-swapping and on-demand loading of functions. The heartbeat communication between the agent and the cloud is not fixed but dynamically adjusted based on the resource status of the end device, avoiding resource contention during periods of high device load. In a self-controlled environment, communication security is paramount. All communication between the agent and the cloud undergoes high-strength encryption and authentication.

[0033] In step S5, the lightweight agent installs and starts the deployment package. Specifically, after receiving the deployment package, the agent automatically completes decompression, environment verification, and the installation and startup of the model service. In step S6, the lightweight agent monitors and uploads key metrics for the target device. Specifically, after the trained model file is deployed, the lightweight agent continuously collects key metrics from the target device and periodically uploads them to the cloud-based intelligent operations and maintenance platform. This provides real-time feedback and monitoring of model deployment on each target device, and dynamically optimizes subsequent deployment strategies. Specifically, key metrics may include: model inference latency, CPU / memory / AI accelerator utilization, model input data distribution, and service health status.

[0034] In steps S1 to S6, the trained model files and the architecture of the domestically produced hardware are first acquired, and deployment packages for different domestically produced hardware are constructed based on these data. Then, considering the type of domestically produced hardware on the target device, different deployment packages are distributed to the corresponding target devices. With the help of a lightweight agent on the target device, the deployment packages can be decompressed, installed, and the corresponding model can be started. After deployment and installation, the lightweight agent can continuously monitor the key indicators of the target device and periodically upload data, dynamically acquiring the status and parameters of the target device after model deployment. Simultaneously, it can dynamically classify and optimize deployment strategies for subsequent model deployments on each target device.

[0035] Traditional model deployment methods often employ a unified model deployment file for batch deployment or prune and compress individual models for deployment. This means that existing technologies suffer from low accuracy, poor reliability, and low efficiency in batch model deployment. In this embodiment of the invention, a deployment package is constructed by combining model files with domestically produced hardware and then distributed specifically to the corresponding target devices. This effectively improves the accuracy, reliability, and efficiency of batch model deployment. Furthermore, by using a lightweight agent to monitor and upload key indicators of the target devices, dynamic monitoring of the target devices is achieved, facilitating dynamic adjustments to the deployment and further improving the deployment and maintenance accuracy of the target devices.

[0036] In this embodiment of the invention, the construction of the deployment package can be achieved by packaging the model files separately with different domestic hardware architectures, etc. Specific steps can be as follows: Figure 2 As shown. Specifically, in Figure 2 The build steps for this deployment package may include: In step S20, a basic environment image and image package for domestically produced hardware are constructed. The basic environment image may include the architecture, operating system, and necessary runtime libraries of the domestically produced hardware, while the image package may include model files, a domestically produced hardware feature library, and startup scripts; that is, one consists of the hardware and operating system, and the other contains the software runtime environment related to different hardware and operating systems.

[0037] In step S21, the hardware model of each target device in the target end side is obtained. The hardware model of the target device is also the model of the domestically produced hardware of the target device, and falls within the scope of the aforementioned domestically produced hardware.

[0038] In step S22, the corresponding basic environment image and image package are obtained according to the hardware model of the target device. Specifically, the corresponding basic environment image and image package can be automatically selected based on the hardware model of the target device.

[0039] In step S23, the base environment image and image package of the target device are packaged, and a deployment package is obtained. This deployment package may include containers based on the domestically developed iSula container technology or a compressed package format.

[0040] In steps S20 to S23, a basic environment image and image package for the domestically produced hardware are first constructed, and the hardware model of each target device is obtained. Based on the hardware model, the corresponding basic environment image and image package are selected. After selection, the basic environment image and image package of the target device are packaged together to obtain a deployment package capable of deploying the corresponding domestically produced hardware. Using this method, the goal of building once and deploying in multiple locations is achieved. A machine-readable resource manifest description file is created for each generated deployment package, containing model information, hardware requirements, software dependencies, and deployment configuration.

[0041] In this embodiment of the invention, the construction of the image package for domestically produced hardware needs to be determined based on the software ecosystem dependency list of the domestically produced hardware, and startup scripts need to be packaged, etc. Specific steps can be as follows: Figure 3 As shown. Specifically, in Figure 3 In this context, the steps for building the image package may include: In step S200, the identifier of the domestic hardware and the software ecosystem dependency list are obtained. For mainstream domestic hardware, this may include Huawei Ascend 310 / 910, Hygon DCU series, Cambricon MLU, etc. The software ecosystem dependency list is the list of software required by the operating model file of this domestic hardware.

[0042] In step S201, a domestic hardware feature library is constructed based on the identifiers of domestically produced hardware and the software ecosystem dependency list. Based on the obtained software ecosystem dependency list, dependency parsing and conflict detection can be performed to determine the installation order and dependencies of each dependency, and to check for version conflicts or conflicts with other software packages between different dependencies.

[0043] In step S202, a minimal runtime environment is constructed based on the software ecosystem dependency list. The minimal runtime environment refers to one that contains only the software and libraries necessary for running the model.

[0044] In step S203, the model file, the minimized runtime environment, and the startup script are packaged to obtain an image package. A configuration template is used to manage the model and service configuration within the image package. Specifically, based on domestic container technology (such as Huawei iSula), the model and the minimized runtime environment are packaged into a Docker-compatible image package. This image package contains a startup script that automatically activates the AI ​​accelerator and starts the inference service. During the packaging process, a configuration template can be used to manage the model and service configuration, and it supports dynamically injecting device-specific parameters during deployment. Specifically, to enable the image package to be deployed and run on different target devices, the model and service configuration need to be separated from the image package. The model and service configurations are abstracted into template files. At startup, parameters are injected into the configuration template based on the hardware model of the current target device to generate the actual configuration file used.

[0045] In steps S200 to S203, identifiers for various domestically produced hardware and a software ecosystem dependency list are first obtained, and a domestic hardware feature library is constructed. Then, a minimal runtime environment is built based on this software ecosystem dependency list. The model files and the minimal runtime environment are packaged, along with the startup script, to obtain an image package. Simultaneously, a configuration template is used to manage the model and service configurations of this image package.

[0046] In this embodiment of the invention, after the deployment package is constructed, different target devices can be grouped, and the deployment package can be distributed to the corresponding target devices. Specific steps can be as follows: Figure 4 As shown. Specifically, in Figure 4 In this context, the deployment package distribution step may include: In step S30, the static and dynamic attributes of each target device on the target side are obtained. This can be achieved by pre-collecting the static and dynamic attributes of the devices through a lightweight proxy on the target side and constructing a profile. Specifically, the static attributes may include: hardware architecture (ARM / x86), AI accelerator type and quantity (Ascend 310 / Hygon DCU), memory / storage capacity, and operating system version; the dynamic attributes may include: current network bandwidth (e.g., 4G / 5G / Wi-Fi signal strength), real-time CPU / memory load, and geographical region.

[0047] In step S31, target devices are grouped according to their static and dynamic attributes to obtain deployment groups. Specifically, the intelligent batch deployment engine automatically and dynamically divides eligible devices into different deployment groups based on a strategy, providing a foundation for subsequent differentiated deployment. Grouping can be based on factors such as hardware type.

[0048] In step S32, different deployment packages are distributed to the corresponding deployment groups. The distribution strategy for the deployment packages can be as follows: Figure 5 As shown, specifically, in Figure 5 The distribution steps of this deployment package may include: In step S320, the deployment package is distributed using a controllable concurrent method. Specifically, the engine employs a controllable concurrent transmission strategy, simultaneously distributing deployment packages to multiple device groups and supporting breakpoint resumption to ensure reliable completion even in industrial environments with unstable networks or power supply conditions. In particular, during deployment package distribution, if the number of parallel distributions reaches its limit, other distributions must wait, effectively preventing network congestion.

[0049] In step S321, a token bucket algorithm is used to control the total number of concurrent deployment tasks. Specifically, the token bucket algorithm controls the total number of concurrent deployment tasks. Only deployment tasks that obtain a "token" can begin execution, thus keeping the system load within a safe threshold. Each deployment task needs to obtain a token from the bucket to begin execution; if there are no tokens in the bucket, other tasks must wait for new tokens to be added, thereby improving the stability and reliability of the deployment tasks.

[0050] In steps S30 to S32, the static and dynamic attributes of the target devices are first collected through a lightweight proxy. Based on the collection results, multiple target devices on the target side are grouped to obtain multiple deployment groups, and then deployment packages are distributed in a targeted manner. Simultaneously, the use of parallel distribution and / or token bucket methods can effectively control the number of distributions and avoid network congestion.

[0051] In this embodiment of the invention, the distribution of large deployment packages requires splitting them before sending. The specific steps can be as follows: Figure 6 As shown. Specifically, in Figure 6 In this context, the distribution steps for this large deployment package may include: In step S33, it is determined whether the deployment package is a large deployment package. The determination of a large deployment package can be achieved by pre-setting a threshold for the deployment package size, thus classifying deployment packages into normal deployment packages and large deployment packages. Specifically, if the current deployment package is larger than the corresponding threshold, it is determined to be a large deployment package.

[0052] In step S34, if it is determined that the deployment package is a large deployment package, the large deployment package is split into multiple smaller deployment package chunks. Specifically, a large deployment package can be split into multiple smaller chunks to facilitate subsequent transmission.

[0053] In step S35, the ACK feedback rate at the target end is obtained. Here, the ACK is the acknowledgment signal from the receiver to the sender, and the ACK feedback rate is also the transmission rate.

[0054] In step S36, the size of the sliding window is obtained based on the ACK feedback speed at the target end. Specifically, if the network is good / the transmission rate is fast, the sliding window is increased to speed up transmission; conversely, if the network is poor / the transmission rate is slow, the sliding window is decreased to ensure stability.

[0055] In step S37, a sliding window is used to transmit multiple deployment packet chunks.

[0056] In steps S33 to S37, the current deployment packet is first diagnosed to confirm whether it is a large deployment packet. If the current deployment packet is a large deployment packet, it can be divided into multiple smaller deployment packet blocks. Simultaneously, the ACK feedback speed at the target end is monitored, and the size of the sliding window is dynamically adjusted based on the ACK feedback speed to improve transmission reliability.

[0057] On the other hand, such as Figure 7 As shown, the present invention also provides a system for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment, which may include a target edge module and a cloud management platform. Specifically, the target edge module may include multiple target devices. The cloud management platform is connected to the target edge module and is used to execute any of the above-described methods for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment.

[0058] In this embodiment of the invention, the cloud management platform may include a unified model packaging center, an intelligent batch deployment engine, and a cloud-based intelligent operation and maintenance platform. Specifically, the unified model packaging center is used to build deployment packages, the intelligent batch deployment engine is used to distribute deployment packages, and the cloud-based intelligent operation and maintenance platform is used to perform operation and maintenance monitoring on the target devices at the target end.

[0059] In this embodiment of the invention, the cloud-based intelligent operation and maintenance platform serves as the "brain" of the system, enabling real-time monitoring and visualization. It globally displays the model operating status, health, and business metrics of all end-devices on a dashboard. The cloud-based intelligent operation and maintenance platform can also perform anomaly detection and root cause analysis. It incorporates a lightweight time-series anomaly detection algorithm to analyze reported inference latency and resource utilization, automatically issuing warnings of potential faults. When the accuracy of a device's model decreases, the platform can correlate and analyze its hardware metrics (such as ECC errors in the Hygon DCU's memory) and input data distribution to determine whether it's "model performance drift" or "hardware failure," and generate a diagnostic report. Furthermore, the cloud-based intelligent operation and maintenance platform also supports one-click gray-scale upgrades. When releasing a new model, the platform supports upgrading a small number of devices (gray-scale group) first, automatically comparing the business metrics of the gray-scale group and the stable group, and only pushing the upgrade to the entire system after confirming the effect, greatly reducing upgrade risks.

[0060] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to enable the machine to execute any of the above-mentioned methods for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment.

[0061] Through the above technical solution, the method and system for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment provided by this invention acquires trained model files and domestically produced hardware. Based on these model files and the types of domestically produced hardware, deployment packages suitable for different domestically produced hardware can be constructed. Multiple deployment packages can then be distributed to different target devices on the corresponding target edge. Simultaneously, a lightweight agent runs on each target device to install and start the distributed deployment packages, and monitors and uploads key indicators of the target device. By combining model files and domestically produced hardware to construct deployment packages and distributing them specifically to corresponding target devices, the accuracy, reliability, and efficiency of batch model deployment can be effectively improved. The monitoring and uploading of key indicators of the target devices through lightweight agents enables dynamic monitoring of the target devices, facilitating dynamic adjustments to the deployment and further improving the deployment and operation and maintenance accuracy of the target devices.

[0062] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0064] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0065] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0066] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0067] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0068] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0069] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0070] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment, characterized in that, include: Obtain pre-trained model files and domestically produced hardware; Multiple deployment packages are built based on the trained model file and the domestically produced hardware; The deployment package is distributed to the target device corresponding to the target endpoint. A lightweight agent runs on each of the target devices on the target end side; The lightweight agent installs and starts the deployment package; The lightweight agent monitors and uploads key metrics of the target device.

2. The method for batch deployment and intelligent operation and maintenance of models on the edge side under an autonomous and controllable environment according to claim 1, characterized in that, The deployment package built based on the trained model file and the domestically produced hardware includes: Construct the basic environment image and image package for the domestically produced hardware; Obtain the hardware model of each target device in the target end-side; Obtain the corresponding basic environment image and image package based on the hardware model of the target device; The base environment image and image package of the target device are packaged, and a deployment package is obtained.

3. The method for batch deployment and intelligent operation and maintenance of models on the edge side under an autonomous and controllable environment according to claim 2, characterized in that, The basic environment image and image package for building domestically produced hardware include: Obtain the identifier of the domestically produced hardware and the list of software ecosystem dependencies; A feature library of domestically produced hardware is constructed based on the identifiers of the domestically produced hardware and the software ecosystem dependency list. Construct a minimal runtime environment based on the aforementioned software ecosystem dependency list; The model file, the minimized runtime environment, and the startup script are packaged to obtain an image package, and a configuration template is used to manage the model and service configuration of the image package.

4. The method for batch deployment and intelligent operation and maintenance of models on the edge side under an autonomous and controllable environment according to claim 1, characterized in that, Distributing the deployment package to the target device corresponding to the target endpoint includes: Obtain the static and dynamic attributes of each target device in the target end-side; The target devices are grouped according to their static and dynamic attributes to obtain deployment groups; The different deployment packages are distributed to the corresponding deployment groups.

5. The method for batch deployment and intelligent operation and maintenance of models on the edge side under an autonomous and controllable environment according to claim 4, characterized in that, Distributing different deployment packages to the corresponding deployment groups includes: The deployment package is distributed using a controllable concurrency method; The token bucket algorithm is used to control the number of concurrently deployed tasks globally.

6. The method for batch deployment and intelligent operation and maintenance of models on the edge side under an autonomous and controllable environment according to claim 4, characterized in that, Distributing different deployment packages to the corresponding deployment groups also includes: Determine whether the deployment package is a large deployment package; If the deployment package is determined to be a large deployment package, the large deployment package is split into multiple smaller deployment package blocks; Obtain the ACK feedback speed from the target end; The size of the sliding window is determined based on the ACK feedback speed at the target end. The sliding window is used to transmit multiple deployment packet chunks.

7. The method for batch deployment and intelligent operation and maintenance of models on the edge side in an autonomous and controllable environment according to claim 4, characterized in that, The static attributes include: hardware architecture, type and number of AI accelerators, memory / storage capacity, and operating system version.

8. The method for batch deployment and intelligent operation and maintenance of models on the edge side in an autonomous and controllable environment according to claim 4, characterized in that, The dynamic attributes include: current network bandwidth, real-time CPU / memory load, and geographical region.

9. A system for batch deployment and intelligent operation and maintenance of models on the edge under an autonomous and controllable environment, characterized in that, include: The target-side module includes multiple target devices; A cloud management platform, connected to the target edge module, is used to execute the model edge batch deployment and intelligent operation and maintenance method in an autonomous and controllable environment as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that are read by a machine to cause the machine to execute the model edge-side batch deployment and intelligent operation and maintenance method as described in any one of claims 1-8 under an autonomous and controllable environment.

Images