Container-based cloud edge fusion model automatic packaging deployment method
Through a container-based method, the cloud-edge convergence model is automatically packaged and deployed, and the complexity and security of the model is solved, efficient compatibility and security of the model on different devices is achieved, and the management process of the cloud and edge end is simplified.
Patent Information
- Application Number
- CN202411280150.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-12-13
AI Technical Summary
During the cloud-edge convergence model deployment process, it is difficult for the existing technology to achieve automated packaging and rapid deployment of models, especially in terms of hardware capabilities and data security of different edge devices, and the model synchronization and update are complex.
Adopting a container-based approach, providing online visual model building tools, automates the packaging of model files and operating environment information, generates image files using container image building instructions, and deploys and manages models through the integrated mirror warehouse of the cloud platform to ensure that the model is in the cloud and Compatibility and security of edge ends.
This enables easy migration and compatibility improvement of models, avoids environmental conflicts, improves deployment efficiency and security through container management tools, and simplifies the synchronization and update process of models.
Smart Images
Figure CN119127226A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of cloud-edge fusion model deployment, and specifically relates to a container-based cloud-edge fusion model automatic packaging and deployment method. Background Art
[0002] With the continuous popularization and development of the Internet of Things and 5G communication technologies, the amount of data and computing power have exploded, and terminal devices can no longer handle processing work alone. Cloud computing processes data in cloud data centers, so it has strong computing power and sufficient resources, but it cannot meet the increasing demand for real-time data and may cause data security issues; the emerging edge computing is performed on edge devices closer to the data source, which can reduce latency and increase response speed, and ensure data real-time, but the computing power on the edge side is limited and cannot run complex model algorithms. Therefore, edge-cloud collaborative technology, which combines the advantages of edge computing and cloud computing, has received considerable attention.
[0003] Cloud-edge fusion model training adopts the idea of distributed training, using the powerful computing power and data storage capacity of cloud computing to perform large-scale model training; at the same time, the real-time characteristics of edge computing data are used to perform parameter updates, model inference and other operations on the pre-trained model obtained by cloud computing to generate the final model. In the deployment process of cloud-edge fusion intelligent models, how to encapsulate the model and its operating environment and quickly deploy it on the edge device has become a difficulty in this process. To address this problem, the current mainstream approach is to slice the model after the model is trained in the cloud, then encapsulate it into an executable file or a lightweight model format, and then distribute it from the cloud to the edge device. The update and deployment of the model is achieved through OTA (over-the-air) technology. However, the distributed deployment of edge devices brings challenges to model synchronization, making the synchronization and update of the model complicated; at the same time, due to the large differences in hardware capabilities of different edge devices, it is impossible to guarantee that each device can efficiently execute the same model; and when running models on edge devices, data privacy and security may become a problem, especially in scenarios involving personal data. Containerization technology is a lightweight virtualization technology that can package application code, data processing frameworks, and various dependencies into an independent container to achieve rapid deployment while reducing resource waste and costs. In order to meet the huge demand for cloud-edge collaborative training models and facilitate users of different technical levels to use cloud-edge fusion ideas to train the required models, it is necessary to explore how to automate the deployment process of cloud-edge fusion models. Based on this, the present invention proposes a container-based cloud-edge fusion model automatic packaging and deployment method. Summary of the invention
[0004] In view of the problems existing in the prior art, the present invention proposes the following technical solutions: A container-based cloud-edge fusion model automatic packaging and deployment method includes the following steps: Step S1: Provide an online visual model building tool and a basic model. Users can select the corresponding model and perform model training in the cloud. Step S2: Automatically encapsulate the model trained in the cloud, package the model file and the model's operating environment information, the model's corresponding API, and the model's structural information into a model deployment package, and generate an image file using a container image building instruction (such as Dockerfile); Step S3: Based on the image repository integrated with the cloud platform, the model is published on the cloud platform using the container command; Step S4: Match a suitable model from the cloud platform, download it from the image repository, start the model and open the interface, perform calculations in the model, and obtain the results. Further, in step S1, the online visualization model building tool includes: providing an online visualization model building tool to the user, which has a friendly user interface, provides basic models and data processing tools, and supports cloud training, so as to help the user easily build and train the required model, deploy it to the edge in the cloud, and use it. In the front-end page of the visualization tool, the types of basic models provided are displayed, and the user can select the required model by himself or input the requirements to use the recommended model for training in the cloud.
[0005] Furthermore, in step S2, the automatic packaging process of the model is completed in the cloud.
[0006] Step S2 specifically includes: Step S2-1: During the model training process, specify the operating environment parameters, including the operating system type, programming language version, dependency list, etc. Step S2-2: Access the model training record or log, view the specified parameters or configuration files, and obtain the model operating environment from the model training process; Step S2-3, design the interface and define the interface specification, use the generalized interface to obtain all interfaces and all key field contents of the encapsulated content at one time, use code to implement the designed interface, use the HTTP protocol for API calls, and document the protocol; Step S2-4: Package the above-obtained information into a model deployment package to generate an image file. It includes the model type, the model's operating environment information, the model's corresponding API, and the model's architecture information, such as the model's hyperparameter information, the model's loss function, input and output format requirements, etc. Step S2-5: Use container image building instructions (such as Dockerfile) to package the model deployment package to generate an image file.
[0007] Furthermore, in step S2, the design interface includes obtaining all interfaces of the model through the function get_function, including a model identifier interface, an input data interface, an output data interface, a model operating environment interface, a model deployment method interface, a document link interface, an authentication and authorization information interface, an error handling interface, a version control information interface, and a monitoring and log information interface; for each interface, extract its name, field, parameter, and data type information, convert all interface information into JSON format and return it.
[0008] Further, in step S2, the interface specification is defined, including using HTTP protocol for API calls, and implementing the selected interface protocol in the code. This includes implementing the format of requests and responses, data exchange methods, request headers and response header processing, etc. The implemented interface protocol is documented, and the document includes the purpose of the interface, request format, response format, meaning of request headers and response headers, and error code handling methods, so that other developers can understand the interface. Where the interface is needed, the implemented interface protocol is called, and requests and responses are made in the format specified in the document.
[0009] Furthermore, in step S3, the model publishing process includes: obtaining access rights to the cloud platform integrated image repository, automatically logging into the image repository, using container (such as docker) commands to submit packaged containers to the image repository, building container (such as Docker) images, adding tags to the images, and obtaining containers from the image repository.
[0010] Furthermore, in step S4, the deployment and execution of the model at the edge includes: matching a suitable model from the cloud, downloading it from the image repository, starting the model and opening the interface; if necessary, opening the data cache area and transmitting data, calling the interface, performing calculations in the model, and obtaining results.
[0011] Beneficial Effects The method of the present invention has the following advantages: first, container encapsulation allows the model and its dependencies to be easily migrated to different devices, whether in the cloud or at the edge, greatly improving the compatibility and scalability of the model; second, the container isolates the environment in which the model runs, avoiding environmental conflicts between different edge devices; the container can be started quickly and automated management can be achieved through container management tools such as Kubernetes, improving the efficiency of model deployment. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 A schematic diagram of a process for deploying a container-based cloud-edge fusion model automatic packaging according to an embodiment of the present invention; Figure 2A schematic diagram of a process for automatically packaging a model trained in the cloud according to an embodiment of the present invention; Figure 3 This is an example diagram of key interface fields when encapsulating a model according to an embodiment of the present invention; Figure 4 The figure is a flow chart of the edge deployment process of the model according to the embodiment of the present invention. DETAILED DESCRIPTION
[0013] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment, and other embodiments may also fall within the scope of the present invention as long as they conform to the gist of the present invention.
[0014] An automated packaging and deployment method for a container-based cloud-edge fusion model, such as Figure 1 As shown, the following steps are included: Step S1: Provide an online visual model building tool and a basic model. Users can select the corresponding model and perform model training in the cloud. Step S2: Automatically encapsulate the model trained in the cloud, package the model file, the model's operating environment information, the model's corresponding API, and the model's structural information into a model deployment package, and use Dockerfile to generate an image file; Step S3: Based on the image repository integrated with the cloud platform, the model is published on the cloud platform using the docker command; Step S4: Match a suitable model from the cloud platform, download it from the image repository, start the model and open the interface, perform calculations in the model, and obtain the results.
[0015] Specifically, during the model training process, specify the operating environment parameters, including the operating system type, programming language version, dependency list, etc. By querying the logs of the training process, find relevant information about the operating environment. For example, if the model saved during the training process is based on Pytorch training, the TensorRT basic environment should also exist in the image repository to facilitate model deployment.
[0016] like Figure 2 As shown, step S2 includes: Step S2-1: During the model training process, specify the operating environment parameters, including the operating system type, programming language version, dependency list, etc. Step S2-2: Access the model training record or log, view the specified parameters or configuration files, and obtain the model operating environment from the model training process; Step S2-3, design the interface and define the interface specification, use the generalized interface to obtain all interfaces and all key field contents of the encapsulated content at one time, use code to implement the designed interface, use the HTTP protocol for API calls, and document the protocol; Specifically, when designing an interface, you need to define the specifications and standards of the interface to ensure that different models can interact in accordance with these standards. These specifications include input and output data formats, request and response parameters, and error handling. Then design a general interface that should be able to accept various types of input and return the expected output; then write code to implement the interface and test it to ensure that it works properly. Different models can be integrated into the general interface.
[0017] Furthermore, the HTTP protocol is used for API calls, and the selected interface protocol is implemented in the code. This includes the implementation of the request and response formats, data exchange methods, request header and response header processing, etc. The implemented interface protocol is documented, including the purpose of the interface, request format, response format, request header and response header meaning, and error code processing methods, so that other developers can understand the interface. Where the interface is needed, call the implemented interface protocol and make requests and responses in the format specified in the document.
[0018] Step S2-4: Package the above-obtained information into a model deployment package to generate an image file. It includes the model type, the model's operating environment information, the model's corresponding API, and the model's architecture information, such as the model's hyperparameter information, the model's loss function, input and output format requirements, etc. Step S2-5: Use Dockerfile to package the model deployment package to generate an image file.
[0019] Figure 3 This is an embodiment of the key fields of the interface described in step S2, described in JSON format, including: model identifier modelIdentifier, input data inputData, output data outputData, model runtime environment modelRuntimeEnvironment, model deployment mode modelDeployment, document link documentLink, authentication and authorization information information, error handling errorHandling, version control information versionControl, and monitoring and log information monitorLog. For this interface, the parameter to be passed in is the model identifier, and the request type is GET.
[0020] Specifically, in setting the generalized interface, you can use a function named get_function to get all the interfaces of the model and describe them in JSON format. This interface can be used to display the model's functions, input parameters, output parameters, and other related information. It traverses all the attributes of the model object, finds the attributes that are callable and not special methods (starting with double underscores), and treats them as interfaces. For each interface, we extract its name, description, parameters, and return value information, and describe them in JSON format. Finally, all interface information is converted into a JSON string and returned.
[0021] like Figure 4 As shown in the figure, the deployment and execution of the model at the edge includes matching the appropriate model from the cloud, downloading it from the image repository, starting the model and opening the interface; if necessary, opening the data cache and transmitting data, calling the interface, performing calculations in the model, and obtaining results.
[0022] Specifically, edge access to the image repository includes: creating your own repository on Docker Hub, storing the image in the registry, and using the dockerlogin command to log in to Docker to pull the docker image for the local project. The docker push command is required to submit the packaged container to the image repository, and the docker pull command is required to obtain the container from the image repository.
[0023] In this embodiment, we can directly call the function to obtain all the interfaces of the model to demonstrate the functions of the model. In addition, we use Docker, a relatively complete image creation and management tool. By managing container images on Docker Hub or other image repositories, and using docker push and docker pull to automatically package and distribute models, the management and deployment efficiency of the model between the cloud and edge devices has been greatly improved. Its advantages include centralized management, version control, automated deployment, cross-platform portability, efficient resource utilization, and security, which can effectively cope with complex requirements in cloud-edge fusion scenarios.
[0024] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.
Claims
1. A container-based cloud-edge fusion model automatic packaging and deployment method, It is characterized in that The following steps are involved: Step S1: Provide an online visual model building tool, provide basic models, and users can select corresponding models to perform model training in the cloud; Step S2: Automatically encapsulate the model trained in the cloud, package the model file, the model's operating environment information, the model's corresponding API, and the model's structural information into a model deployment package, and generate an image file using the container image building instruction; Step S3: Based on the image repository integrated with the cloud platform, the model is published on the cloud platform using the container command; Step S4: Match a suitable model from the cloud platform, download it from the image repository, start the model and open the interface, perform calculations in the model, and obtain the results.
2. The automated packaging deployment method according to claim 1, It is characterized in that In step S1, the online visualization model building tool includes: having a friendly user interface, providing basic models and data processing tools, and supporting cloud training to help users easily build and train the required models, deploy and use them on the edge in the cloud; in the front-end page of the visualization tool, the types of basic models provided are displayed, and users can select the required model or input the requirements to use the recommended model for training in the cloud.
3. The automated packaging deployment method according to claim 1, It is characterized in that Step S2 specifically includes: Step S2-1: During the model training process, specify the operating environment parameters, including the operating system type, programming language version, dependency list, etc. Step S2-2: Access the model training record or log, view the specified parameters or configuration files, and obtain the model operating environment from the model training process; Step S2-3, define the interface specification, use the generalized interface to obtain all interfaces and all key field contents of the encapsulated content at one time, use code to implement the designed interface, use the HTTP protocol for API calls, and document the protocol; Step S2-4: Package the above-obtained information into a model deployment package to generate an image file; the model deployment package includes the model type, the model's operating environment information, the model's corresponding API, and the model's architecture information, such as the model's hyperparameter information, the model's loss function, input and output format requirements, and other information; Step S2-5: Use the container image building instruction to package the model deployment package to generate an image file.
4. The automated packaging deployment method according to claim 1, It is characterized in that In step S3, the model publishing process includes: obtaining access rights to the cloud platform integrated image repository, automatically logging into the image repository, using container commands to submit packaged containers to the image repository, building container images, adding tags to the images, and obtaining containers from the image repository.
5. The automated packaging deployment method according to claim 1, It is characterized in that In step S4, the deployment and execution of the model at the edge includes: matching a suitable model from the cloud, downloading it from the image repository, starting the model and opening the interface; if necessary, opening the data cache area and transmitting data, calling the interface, performing calculations in the model, and obtaining results.
6. The automated packaging deployment method according to claim 3, It is characterized in that Get all interfaces of the model through the get_function function, including model identifier interface, input data interface, output data interface, model operating environment interface, model deployment method interface, document link interface, authentication and authorization information interface, error handling interface, version control information interface, monitoring and log information interface; for each interface, extract its name, field, parameter and data type information, convert all interface information into JSON format and return it.
7. The automated packaging deployment method according to claim 3, It is characterized in that Use HTTP protocol for API calls and implement the selected interface protocol in the code; including the implementation of request and response formats, data exchange methods, request header and response header processing, etc. The implemented interface protocol should be documented, including the purpose of the interface, request format, response format, meaning of request header and response header, and error code handling method, so that other developers can understand the interface; where the interface needs to be used, call the implemented interface protocol, and make requests and responses in the format specified in the document.