A cloud-edge AI model management method, storage medium, and electronic device
Through the cloud edge AI model management method, Kubernetes is used to create and manage edge devices and services, solving the complexity of AI model deployment and management in the hydropower industry, and achieving rapid deployment and efficient management.
Patent Information
- Application Number
- CN202311457294.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-02
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2043-11-02
AI Technical Summary
In the hydropower industry, there is a complex programming and configuration process for the deployment and management of AI models, which requires professional skills and a lot of time. The update and version control of the model are chaotic, so the management work after deployment is also troublesome.
Provides a cloud edge AI model management method. By responding to the configuration request of edge devices, configuring edge devices based on the configuration request instructions, and creating a service deployment set in Kubernetes, monitoring state changes and making callbacks, supporting creation and activation , delete edge devices and services, adjust the number of copies and other operations.
It realizes the rapid deployment and management of cloud-edge AI models, provides more control capabilities for the deployed AI model services, simplifies configuration and operation processes, and improves deployment efficiency and controllability.
Smart Images

Figure CN117742813B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information processing technology, and in particular to a cloud-edge AI model management method, storage medium, and electronic device. Background Art
[0002] The hydropower industry is a key infrastructure industry, and its importance lies in the fact that water resource management and power supply are crucial to social and economic development. With the continuous advancement of science and technology, the application of artificial intelligence (AI) technology in the hydropower industry has gradually emerged. These AI models are widely used for data analysis, prediction, equipment monitoring and decision support to improve the efficiency, reliability and operation management of hydropower systems.
[0003] In the hydropower industry, the training and deployment of AI models are key steps to ensure that they can achieve their maximum potential in actual operations. For example, an AI model for fault detection of hydropower station equipment needs to be trained and then deployed to field equipment to monitor equipment status in real time and predict potential failures. The effective management and deployment of these models is crucial to the sustainable development of the hydropower industry.
[0004] However, in the hydropower industry, there are still some challenges in deploying AI models on edge devices. Traditional model deployment methods may involve complex programming and configuration deployment processes, which not only require professional skills and a lot of time, but also require deployment on field devices. In addition, AI model updates and version control may also become confusing, and management tasks such as starting, stopping, and deleting models after deployment are also troublesome. Summary of the invention
[0005] The present application provides a cloud-edge AI model management method, storage medium, and electronic device to solve problems existing in related technologies.
[0006] In the first aspect, the present invention provides a cloud-edge AI model management method, including responding to a configuration request of an edge device, configuring the edge device based on the content indicated by the configuration request; based on receiving a model service deployment request, configuring the configuration information required for service deployment, and creating a service deployment set in Kubernetes; monitoring the status changes of the service deployment set, and calling back the status changes.
[0007] Optionally, in response to a configuration request from an edge device, configuring the edge device based on the content indicated by the configuration request includes: responding to a creation request from the edge device, completing the creation of the edge device by creating fields; in response to a request to add the edge device to a cluster, generating a script for adding, and joining the cluster after the script is called and executed to complete the activation of the created edge device; and deploying a daemon process for the edge device.
[0008] Optionally, in response to a configuration request from an edge device, configuring the edge device based on the content indicated by the configuration request further includes: in response to a deletion request from the edge device, exiting the daemon process and deleting the edge device as a node.
[0009] Optionally, creating an AI model service deployment set in Kubernetes includes: creating a Deployment in Kubernetes, wherein the download container is used to download the model, and the algorithm container provides algorithm services for the model.
[0010] Optionally, monitoring the status change of the service deployment set and calling back the status change includes: monitoring the status change of the Deployment and the container, and calling back the status change through a request message.
[0011] Optionally, the method further includes: in response to receiving a model processing request, calling the algorithm service based on information indicated by the request.
[0012] Optionally, the method further includes: in response to a request to modify a service status deployed in a service deployment set, adjusting a maximum number of replicas and a minimum number of replicas of the service.
[0013] Optionally, the method further includes: upon receiving a deletion request for the service deployment set, deleting the service deployment set.
[0014] In a second aspect, the present invention provides a computer-readable storage medium, characterized in that the storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method described in any implementation method of the first aspect above.
[0015] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method provided in the first aspect when executing the program.
[0016] The present invention discloses a cloud-edge AI model management method, storage medium and electronic device, wherein the method includes responding to a configuration request of an edge device, configuring the edge device based on the content indicated by the configuration request; configuring the configuration information required for service deployment based on the received model service deployment request, and creating a service deployment set in Kubernetes; monitoring the state changes of the service deployment set, and calling back the state changes. Through simple configuration and operation, rapid deployment of cloud-edge AI models can be achieved, AI models can be used faster, and more control capabilities can be provided for deployed AI model services. The defects existing in the related art are overcome. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0018] Figure 1 A flowchart of a cloud-edge AI model management method for this application;
[0019] Figure 2 This is an application diagram of a cloud-edge AI model management method in this application;
[0020] Figure 3 This is another application diagram of a cloud-edge AI model management method in this application;
[0021] Figure 4 A method corresponding to the present application Figure 1 Schematic diagram of an electronic device. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0023] The system architecture applicable to the method of this embodiment can refer to the figure, including AI platform, reasoning service management system, kubernetes, edgeNode, etc. Users can implement configuration operations and data presentation through the AI platform, the reasoning service management system can respond to requests, and kubernetes can create, update or delete deployment sets. The method of this embodiment can be applicable to the reasoning service management system.
[0024] The following is combined with Figure 1 The cloud-edge AI model management method shown is exemplified. The method includes the following steps:
[0025] Step 101: In response to a configuration request from an edge device, configure the edge device based on the content indicated by the configuration request.
[0026] In this embodiment, the edge deployment of the AI model requires the creation of an edge host device in advance, and the management of the edge device is achieved through configuration. The edge device management includes the creation, editing, query, and deletion of edge devices.
[0027] As an optional implementation method of this embodiment, the response to the configuration request of the edge device and the configuration of the edge device based on the content indicated by the configuration request include: responding to the creation request of the edge device, completing the creation of the edge device through the creation field; responding to the request to add the edge device to the cluster, generating a script for adding, and joining the cluster after the script is called and executed to complete the activation of the created edge device; and deploying the daemon process of the edge device.
[0028] In this optional implementation, when creating an edge device, the edge device information can be created by creating fields, including but not limited to the device name, device IP, device location, device tag, etc. After creating the edge device, the user can trigger a script addition request, generate a script based on the request, and enable the edge device to execute the script to join the cluster, and deploy the edge daemon after joining the cluster. The host of the edge device executes the activation script to obtain the edge device information, including the system architecture and hard disk ID, download and decompress the activation script, and execute the command to join the cluster.
[0029] As an optional implementation method of this embodiment, responding to a configuration request from the edge device and configuring the edge device based on the content indicated by the configuration request also includes: responding to a deletion request from the edge device, exiting the daemon process and deleting the edge device as a node.
[0030] In this optional implementation, edge devices can be created or deleted based on configuration requests, and the state changes of edge devices can be monitored. The node status information such as readiness information can be sent to Kafka to further present the node's heartbeat and consumption on the AI platform.
[0031] refer to Figure 1 The schematic diagram of the edge node management framework is shown. The AI platform can send a node addition script to the cloud. Based on the request, the cloud generates a script and is called by the AI platform. After the script is called, the script is executed on the edge device. After the script is executed, the cluster is joined, and then the daemon is deployed to complete the deployment of the edge device. After joining the cluster, the node is monitored and the monitored status information is presented on the AI platform. If the node is offline, the node status will also be monitored. If an edge node deletion request is received from the AI platform, the node is deleted through kubernetes, and based on the request, a request is made to the daemon to exit the process, causing the edge node to exit, and finally the status information after successful deletion is displayed on the AI platform.
[0032] Step 102: Based on the received model service deployment request, configure the configuration information required for service deployment, and create a service deployment set in Kubernetes.
[0033] In this embodiment, the inference service management system can accept service creation requests and configure resource parameters for the selected AI model, including receiving user-configured parameters, including but not limited to memory, number of CPU cores, GPU video memory, maximum number of instances, and other resource parameters.
[0034] Step 103: monitor the status change of the service deployment set and call back the status change.
[0035] In this embodiment, the Kubernetes container status change data will be called back, and the information can be presented in the AI platform.
[0036] As an optional implementation method of this embodiment, creating an AI model service deployment set in Kubernetes includes: creating a Deployment in Kubernetes, wherein the download container is used to download the model, and the algorithm container provides algorithm services for the model.
[0037] In this optional implementation, a Deployment is created in Kubernetes through Serverless, and the edge deployment deploys the model service to the edge machine based on native Kubernetes. The download container is responsible for downloading the model, and the algorithm container provides algorithm services based on the model.
[0038] As an optional implementation manner of this embodiment, the method further includes: in response to receiving a model processing request, calling the algorithm service based on information indicated by the request.
[0039] In this optional implementation, the AI model algorithm service is successfully deployed, and the AI platform provides the algorithm service url, token, sample data and return structure, through which the deployed algorithm service can be directly called. For example, when Kubernetes creates a Pod container, the download container will download the AI model file from the MinIO specified location according to the model name. The number of models is single or multiple, and is used by the algorithm container.
[0040] When calling the deployed algorithm service, the Kong gateway will record the call volume and call time through http request callback to the AI platform.
[0041] As an optional implementation of this embodiment, monitoring the state change of the service deployment set and calling back the state change includes: monitoring the state change of the Deployment and the container, and calling back the state change through a request message.
[0042] In this optional implementation, the Deployment and Pod status changes are monitored. When the Kubernetes container status changes, the service status will be called back to the AI platform through an http request to update the service status; the service status includes but is not limited to: not created, creating, scheduling, scheduling successfully, running successfully, running failed, deleting, and deleted successfully.
[0043] As an optional implementation of this embodiment, the method further includes: in response to a request to modify a service status deployed in a service deployment center, adjusting a maximum number of replicas and a minimum number of replicas of the service.
[0044] In this optional implementation, by adjusting the maximum and minimum number of replicas of the service nodes, modifying the resident / non-resident status of the service and shutting down the service, the AI platform is called back after monitoring the change in the Kubernetes service status.
[0045] Only cloud deployment can modify the resident / non-resident status. Resident services will be deployed and run all the time, and resources will be occupied all the time. Non-resident services allow services to sleep and run algorithm services when there is a request to call them.
[0046] As an optional implementation manner of this embodiment, the method further includes: upon receiving a deletion request for the service deployment set, deleting the service deployment set.
[0047] In this optional implementation, the Kubernetes deployment set is deleted after receiving the deletion request, and the service data is updated after the deletion is successful.
[0048] refer to Figure 2 , the AI platform can send a request to create a service to the inference service management system. The inference service management system checks the request and creates the service after the check is passed, and creates a deployment set through kubernets. After the creation is completed, the database is updated and the creation completion information is displayed on the AI platform. Users can send a request to modify the service status through the AI platform. The inference service management system checks the request and updates the service status after the check is passed, updates the deployment set, further updates the database, and finally displays the successful modification on the AI platform. Users can send a request to delete the service through the AI platform. After the request is checked, the service is deleted, the database is updated, and displayed on the AI platform. Creating a deployment set, updating a deployment set, or deleting a deployment set can all be done through the kubernets controller to create, modify, or delete edge node pods.
[0049] This embodiment is based on cloud-native technology functions and only requires simple configuration and operation to achieve rapid deployment of cloud-edge AI models, faster use of AI models, and more control over the deployed AI model services, including resident / non-resident status modification, shutdown, and deletion, etc. It can also observe the service call volume and call time, and provide a stable, efficient, and controllable environment for AI models.
[0050] The present application also provides a cloud-edge AI model management device, including a configuration unit, configured to respond to a configuration request of the edge device and configure the edge device based on the content indicated by the configuration request; a service deployment set creation unit, configured to configure the configuration information required for service deployment based on the received model service deployment request, and create a service deployment set in Kubernetes; a monitoring unit is configured to monitor the state changes of the service deployment set and call back the state changes.
[0051] Optionally, in response to a configuration request from an edge device, configuring the edge device based on the content indicated by the configuration request includes: responding to a creation request from the edge device, completing the creation of the edge device by creating fields; in response to a request to add the edge device to a cluster, generating a script for adding, and joining the cluster after the script is called and executed to complete the activation of the created edge device; and deploying a daemon process for the edge device.
[0052] Optionally, in response to a configuration request from an edge device, configuring the edge device based on the content indicated by the configuration request further includes: in response to a deletion request from the edge device, exiting the daemon process and deleting the edge device as a node.
[0053] Optionally, creating an AI model service deployment set in Kubernetes includes: creating a Deployment in Kubernetes, wherein the download container is used to download the model, and the algorithm container provides algorithm services for the model.
[0054] Optionally, monitoring the status change of the service deployment set and calling back the status change includes: monitoring the status change of the Deployment and the container, and calling back the status change through a request message.
[0055] Optionally, the apparatus further comprises a processing unit configured to, in response to receiving a model processing request, call the algorithm service based on information indicated by the request.
[0056] Optionally, the device further comprises a service status adjustment unit configured to adjust a maximum number of replicas and a minimum number of replicas of a service in response to a request to modify a service status deployed in a service deployment set.
[0057] Optionally, the device also deletes the service deployment set upon receiving a deletion request for the service deployment set.
[0058] The present application also provides a computer readable medium, which stores a computer program, which can be used to execute the above Figure 1 Based on the method provided.
[0059] This application also provides Figure 4 The one shown corresponds to Figure 1 A schematic diagram of the electronic device. Figure 3 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, this application does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0060] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0061] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0062] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0063] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0064] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0065] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0066] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0067] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0068] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0069] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0070] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0071] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0072] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0073] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer media, including storage devices.
[0074] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0075] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A cloud-edge AI model management method, comprising: In response to a configuration request of an edge device, the edge device is configured based on the content indicated by the configuration request, including: in response to a creation request of the edge device, completing the creation of the edge device by creating a field; in response to a request to add the edge device to a cluster, generating an addition script, and joining the cluster after the script is called and executed to complete the activation of the created edge device; deploying a daemon process for the edge device; and also including: in response to a deletion request of the edge device, exiting the daemon process and deleting the edge device as a node; wherein the state changes of the edge device can be monitored, and the state information of the node can be sent to Kafka, so as to further present the heartbeat and consumption of the node on the AI platform; Based on the received model service deployment request, configure the configuration information required for service deployment and create a service deployment set in Kubernetes; Monitor the status change of the service deployment set and call back the status change; Creating an AI model service deployment set in Kubernetes includes: creating a Deployment in Kubernetes, where the download container is used to download the model, the algorithm container provides algorithm services for the model, and the edge node pod is created through the Kubernetes controller; Monitoring the state change of the service deployment set and calling back the state change includes: monitoring the state change of the Deployment and the container, and calling back the state change through a request message.
2. The cloud-edge AI model management method according to claim 1 is characterized in that: The method further comprises: In response to receiving the model processing request, the algorithm service is called based on the information indicated by the request.
3. The cloud-edge AI model management method according to claim 1, characterized in that: The method further comprises: In response to a request to modify a state of a service deployed in a service deployment set, a maximum number of replicas and a minimum number of replicas of the service are adjusted.
4. The cloud-edge AI model management method according to claim 1, characterized in that: The method further comprises: Upon receiving a deletion request for a service deployment set, the service deployment set is deleted.
5. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 4 is implemented.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 4 is implemented.
