Methods, devices, equipment, and media for deploying machine learning models based on Kubernetes

Abstract

This invention discloses a method, apparatus, device, and medium for deploying machine learning models based on Kubernetes (k8s), belonging to the field of cloud computing and machine learning technology. The technical problem this invention aims to solve is how to adapt to different model formats and frameworks, automatically select the runtime environment, and load the model. The technical solution adopted is as follows: S1. Create model service runtime images for different training frameworks; S2. The model service runtime image loads model files from the / mnt / models directory; S3. Build the model service runtime image and push it to a Dockhub or private Harbor image storage platform, generating a corresponding image address for use; S4. Install and configure the Golang environment and the make command-line tool to compile and build the source code; S5. Download and unzip kubebuilder according to system requirements; S6. Use kubebuilder to build a k8s CRD named serverruntime; S7. Package and deploy serverruntime to the cluster; S8. Bind the correspondence between different training frameworks and the model service runtime image.

Description

Technical Field

[0001] This invention relates to the fields of cloud computing and machine learning technology, specifically to a method, apparatus, device, and medium for deploying machine learning models based on Kubernetes. Background Technology

[0002] With the rapid development of artificial intelligence and machine learning, an increasing number of applications rely on the inference and prediction capabilities of machine learning models. Major enterprises and research institutions use various training frameworks (such as Paddle, PyTorch, and TensorFlow) to train and optimize models. These trained models need to be deployed to production environments to provide online inference services to meet the ever-growing user demand. The model deployment process typically involves several complex operations, including selecting an appropriate runtime environment, configuring model storage, setting up inference services, and integrating with other applications or services. Traditionally, machine learning model deployment requires developers to manually write numerous YAML or JSON files to configure the model's runtime image, storage location, network services, and other information. This process is extremely complex, especially when managing tens of thousands of model instances in a production environment; manually maintaining these configuration files is both tedious and error-prone. Machine learning model lifecycle management also faces many challenges. In real-world scenarios, with changing business needs and model version updates, developers need to frequently update models and redeploy services. This repetitive operation places a significant burden on operations and maintenance personnel, especially when unified management of models with different frameworks and formats is required. Traditional manual deployment methods are prone to problems such as version inconsistencies and model configuration errors, severely impacting system stability and efficiency. As the application scale of machine learning models continues to expand, developers' needs for deploying and managing multi-framework, multi-format models are becoming increasingly urgent. A flexible, scalable, and automated model deployment solution can not only reduce development and operations costs but also significantly improve system reliability and responsiveness. Therefore, how to adapt to different model formats and frameworks, automatically select the runtime environment, and load the model is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] The technical objective of this invention is to provide a method, apparatus, device, and medium for deploying machine learning models based on Kubernetes, in order to solve the problem of how to adapt to different model formats and frameworks, automatically select the runtime environment, and load the model.

[0004] The technical objective of this invention is achieved as follows: a method for deploying a machine learning model based on Kubernetes, the specific method of which is as follows:

[0005] S1. Create runtime images of model services for different training frameworks; the training frameworks include Paddle, PyTorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM.

[0006] S2. The runtime image of the model service consistently loads model files from the / mnt / models directory;

[0007] S3. Build the runtime image of the model service and push the image to Dockhub or a private Harbor image storage platform to generate the corresponding image address for use;

[0008] S4. Install and configure the Golang environment and the make command-line tool to compile and build the source code; specifically, on a Linux system, use the package manager to install Golang and make: apt install golang, apt install make;

[0009] S5. Download and extract kubebuilder according to system requirements;

[0010] S6. Use kubebuilder to build a Kubernetes CRD (Custom Resource Definition) named serverruntime; serverruntime is used to bind the model format and the model service runtime image.

[0011] S7. Package the server runtime and deploy it to the cluster;

[0012] S8. Bind the correspondence between different training frameworks and the runtime images of the model service;

[0013] S9. Use kubebuilder to build a Kubernetes CRD (Custom Resource Definition) named modelserver; modelserver is used to provide the necessary information for model deployment.

[0014] S10. Package the modelserver and deploy it to the cluster;

[0015] S11. Create the modelserver resource. Simply specify the model's format and the model storage address (modelAddr) to complete the model deployment.

[0016] As a preferred option, the specific steps in step S1 for creating runtime images of model services for different training frameworks are as follows:

[0017] S101. Select a suitable base image as the base image for the model service runtime image: Select the official service images released by each training framework as the base image for the model service runtime image. The official service images are pre-installed with various dependencies, reducing the complexity of image creation.

[0018] S102. Write Python code app.py, which loads models from a fixed directory ( / mnt / models) and provides HTTP service.

[0019] S103. Copy the code app.py to the / app directory inside the image;

[0020] S104. Set the image's startup command to python / app / app.py;

[0021] S105. Use the `docker build` command to build the image.

[0022] Preferably, kubebuilder in step S5 is a framework for building Kubernetes APIs, which helps developers create custom controllers and custom resource definitions (CRDs). By using kubebuilder, developers can more easily extend the Kubernetes API to meet the needs of specific applications. According to system requirements, the appropriate version needs to be selected based on the operating system (Windows, Linux) and CPU architecture (X86, ARM).

[0023] Preferably, the serverruntime in step S6 includes a format field and an image field;

[0024] The format field is used to identify the model format, and the allowed values ​​are Paddle, PyTorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM.

[0025] The image field is used to identify the runtime image of the model service corresponding to the model format represented by the format field. The allowed value is the image address generated in step S3.

[0026] As a preferred option, before binding the correspondence between different training frameworks and model service runtime images in step S8, serverruntime CRDs with format values ​​of Paddle, Pytorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM are created respectively.

[0027] Preferably, the modelserver in step S9 includes the format field, replicas field, modelAddr field, and standard podSpec field;

[0028] The format field is used to identify the model format. Allowed values ​​are Paddle, Pytorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM. It corresponds to the format value of serverruntime. The corresponding model service runtime image address in serverruntime is found based on the format field value. This image is used as the POD image.

[0029] The replicas field is used to identify the number of PODs created by the Deployment, that is, the number of instances of the model deployment. The default value is 1.

[0030] The modelAddr field is used to identify the model storage address, and supports network paths;

[0031] The standard podSpec field is used to describe basic information about the POD. The podSpec does not include an image field because the image field is determined by the format field.

[0032] More preferably, the process of creating the modelserver resource in step S11 is as follows:

[0033] Create a Kubernetes Deployment resource based on the standard podPsec field. If the standard podSPec field is empty, then build the podSpec yourself. The image field below should be the runtime image address of the matching model service.

[0034] The number of replicas for a Deployment is the value of the replicates field;

[0035] Create an init container to download the image of the model whose storage address is identified in the modelAddr field, and use the hostpath data volume to mount the downloaded model to the / mnt / models directory of the container;

[0036] Create a Kubernetes Service resource to act as a network proxy for the Service to the POD.

[0037] A Kubernetes-based machine learning model deployment device, comprising:

[0038] The creation module is used to create runtime images of model services for different training frameworks; the training frameworks include Paddle, PyTorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM.

[0039] The loading module is used to fix the model service runtime image to load model files from the / mnt / models directory;

[0040] The image building module is used to build the runtime image of the model service and push the image to the Dockhub or private Harbor image storage platform to generate the corresponding image address for use.

[0041] The configuration module is used to install and configure the Golang environment and the make command-line tool to compile and build the source code; specifically, on Linux systems, Golang and make are installed using package managers: `apt install golang` and `apt install make`.

[0042] The decompression module is used to download and decompress kubebuilder according to system requirements;

[0043] The serverruntime build module is used to build a Kubernetes CRD (Custom Resource Definition) named serverruntime using kubebuilder; serverruntime is used to bind the model format and the model service runtime image.

[0044] The serverruntime deployment module is used to package and deploy serverruntime to the cluster;

[0045] The binding module is used to bind the correspondence between different training frameworks and the runtime images of the model service;

[0046] The modelserver building block is used to build a Kubernetes CRD (Custom Resource Definition) named modelserver using kubebuilder; modelserver is used to provide the necessary information for model deployment.

[0047] The modelserver deployment module is used to package the modelserver and deploy it to the cluster;

[0048] The resource creation module is used to create modelserver resources. Simply specify the model's format and the model storage address (modelAddr) to complete the model deployment.

[0049] An electronic device includes: a memory and at least one processor;

[0050] The memory contains computer programs;

[0051] The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the Kubernetes-based machine learning model deployment method described above.

[0052] A computer-readable storage medium storing a computer program that can be executed by a processor to implement the Kubernetes-based machine learning model deployment method described above.

[0053] The Kubernetes-based machine learning model deployment method, apparatus, device, and medium of the present invention have the following advantages:

[0054] (i) This invention achieves automated creation and management of K8s resources by combining CRD and controller. Users do not need to manually write complex YAML files, which simplifies the deployment process. It can adapt to different model formats and frameworks, automatically select the runtime environment and load the model, which not only reduces development and maintenance costs, but also significantly improves the reliability and response speed of the system.

[0055] (ii) This invention creates a Kubernetes Custom Resource Definition (CRD). Users only need to create this CRD to automatically complete the model deployment work, thereby greatly simplifying the model deployment process.

[0056] (iii) This invention simplifies the deployment process and achieves automated deployment by automatically selecting the runtime image and loading the model through CRD;

[0057] (iv) This invention supports a variety of machine learning frameworks and model formats, and can adapt to the model deployment needs of different scenarios, thus achieving the purpose of flexible expansion;

[0058] (v) This invention utilizes the controller to automatically reconcile the state of K8s resources, ensuring that the resources in the K8s cluster are always consistent with the CRD definition, reducing the need for manual intervention, improving operation and maintenance efficiency, that is, ensuring that the model service is consistent with the expected state, reducing the operation and maintenance burden, and achieving efficient management.

[0059] (vi) The present invention only requires specifying the model format and address to complete the model, reducing the difficulty of model deployment and ensuring ease of use;

[0060] (vii) This invention significantly reduces deployment errors caused by manual operation and improves system stability through automated processes and configuration verification mechanisms. Attached Figure Description

[0061] The invention will be further described below with reference to the accompanying drawings.

[0062] Appendix Figure 1 A flowchart illustrating the deployment method for Kubernetes-based machine learning models. Detailed Implementation

[0063] The following detailed description of the Kubernetes-based machine learning model deployment method, apparatus, device, and medium of the present invention, with reference to the accompanying drawings and specific embodiments, is provided.

[0064] Example 1:

[0065] As attached Figure 1 As shown in the figure, this embodiment provides a method for deploying a machine learning model based on Kubernetes, which is as follows:

[0066] S1. Create runtime images of model services for different training frameworks; the training frameworks include Paddle, PyTorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM.

[0067] S2. The runtime image of the model service consistently loads model files from the / mnt / models directory;

[0068] S3. Build the runtime image of the model service and push the image to Dockhub or a private Harbor image storage platform to generate the corresponding image address for use;

[0069] S4. Install and configure the Golang environment and the make command-line tool to compile and build the source code; specifically, on a Linux system, use the package manager to install Golang and make: apt install golang, apt install make;

[0070] S5. Download and extract kubebuilder according to system requirements;

[0071] S6. Use kubebuilder to build a Kubernetes CRD (Custom Resource Definition) named serverruntime; serverruntime is used to bind the model format and the model service runtime image.

[0072] S7. Package the server runtime and deploy it to the cluster;

[0073] S8. Bind the correspondence between different training frameworks and the runtime images of the model service;

[0074] S9. Use kubebuilder to build a Kubernetes CRD (Custom Resource Definition) named modelserver; modelserver is used to provide the necessary information for model deployment.

[0075] S10. Package the modelserver and deploy it to the cluster;

[0076] S11. Create the modelserver resource. Simply specify the model's format and the model storage address (modelAddr) to complete the model deployment.

[0077] In this embodiment, step S1, creating runtime images for model services targeting different training frameworks, is as follows:

[0078] S101. Select a suitable base image as the base image for the model service runtime image: Select the official service images released by each training framework as the base image for the model service runtime image. The official service images are pre-installed with various dependencies, reducing the complexity of image creation.

[0079] S102. Write Python code app.py, which loads models from a fixed directory ( / mnt / models) and provides HTTP service.

[0080] S103. Copy the code app.py to the / app directory inside the image;

[0081] S104. Set the image's startup command to python / app / app.py;

[0082] S105. Use the `docker build` command to build the image.

[0083] In step S5 of this embodiment, kubebuilder is a framework for building Kubernetes APIs, which helps developers create custom controllers and custom resource definitions (CRDs). By using kubebuilder, developers can more easily extend the Kubernetes API to meet the needs of specific applications. According to system requirements, the appropriate version needs to be selected based on the operating system (Windows, Linux) and CPU architecture (X86, ARM).

[0084] In step S6 of this embodiment, serverruntime includes a format field and an image field;

[0085] The format field is used to identify the model format, and the allowed values ​​are Paddle, PyTorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM.

[0086] The image field is used to identify the runtime image of the model service corresponding to the model format represented by the format field. The allowed value is the image address generated in step S3.

[0087] Before binding the correspondence between different training frameworks and model service runtime images in step S8 of this embodiment, serverruntime CRDs with format values ​​of Paddle, Pytorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM are created respectively.

[0088] In this embodiment, the modelserver in step S9 includes the format field, replicas field, modelAddr field, and standard podSpec field;

[0089] The format field is used to identify the model format. Allowed values ​​are Paddle, Pytorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM. It corresponds to the format value of serverruntime. The corresponding model service runtime image address in serverruntime is found based on the format field value. This image is used as the POD image.

[0090] The replicas field is used to identify the number of PODs created by the Deployment, that is, the number of instances of the model deployment. The default value is 1.

[0091] The modelAddr field is used to identify the model storage address, and supports network paths;

[0092] The standard podSpec field is used to describe basic information about the POD. The podSpec does not include an image field because the image field is determined by the format field.

[0093] In step S11 of this embodiment, the creation of the modelserver resource is handled as follows:

[0094] ① Create a k8s Deployment resource based on the standard podPsec field. If the standard podSPec field is empty, then build the podSpec yourself. The image field below should be the runtime image address of the matching model service.

[0095] ②The number of replicas for a Deployment is the value of the `replicas` field;

[0096] ③ Create an init container to download the image of the model whose storage address is identified in the modelAddr field, and use the hostpath data volume to mount the downloaded model to the / mnt / models directory of the container;

[0097] ④ Create a Kubernetes Service resource to complete the network proxy of the Service to the POD.

[0098] Example 2:

[0099] This embodiment provides a machine learning model deployment device based on Kubernetes, the device comprising:

[0100] The creation module is used to create runtime images of model services for different training frameworks; the training frameworks include Paddle, PyTorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM.

[0101] The loading module is used to fix the model service runtime image to load model files from the / mnt / models directory;

[0102] The image building module is used to build the runtime image of the model service and push the image to the Dockhub or private Harbor image storage platform to generate the corresponding image address for use.

[0103] The configuration module is used to install and configure the Golang environment and the make command-line tool to compile and build the source code; specifically, on Linux systems, Golang and make are installed using package managers: `apt install golang` and `apt install make`.

[0104] The decompression module is used to download and decompress kubebuilder according to system requirements;

[0105] The serverruntime build module is used to build a Kubernetes CRD (Custom Resource Definition) named serverruntime using kubebuilder; serverruntime is used to bind the model format and the model service runtime image.

[0106] The serverruntime deployment module is used to package and deploy serverruntime to the cluster;

[0107] The binding module is used to bind the correspondence between different training frameworks and the runtime images of the model service;

[0108] The modelserver building block is used to build a Kubernetes CRD (Custom Resource Definition) named modelserver using kubebuilder; modelserver is used to provide the necessary information for model deployment.

[0109] The modelserver deployment module is used to package the modelserver and deploy it to the cluster;

[0110] The resource creation module is used to create modelserver resources. Simply specify the model's format and the model storage address (modelAddr) to complete the model deployment.

[0111] Example 3:

[0112] This embodiment also provides an electronic device, including: a memory and a processor;

[0113] The memory stores the instructions executed by the computer.

[0114] The processor executes computer execution instructions stored in the memory, causing the processor to execute the Kubernetes-based machine learning model deployment method in any embodiment of the present invention.

[0115] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can be a microprocessor or any conventional processor.

[0116] Memory is used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, memory can also include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart memory cards (SMC), secure digital cards (SD cards), flash memory cards, at least one disk storage device, flash memory devices, or other volatile solid-state storage devices.

[0117] Example 4:

[0118] This embodiment also provides a computer-readable storage medium storing multiple instructions, which are loaded by a processor to cause the processor to execute the Kubernetes-based machine learning model deployment method according to any embodiment of the present invention. Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or apparatus can read and execute the program code stored in the storage medium.

[0119] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0120] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0121] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0122] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for deploying a machine learning model based on Kubernetes, characterized in that, The method is as follows: S1. Create runtime images of model services for different training frameworks; the training frameworks include Paddle, PyTorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM. S2. The runtime image of the model service consistently loads model files from the / mnt / models directory; S3. Build the runtime image of the model service and push the image to Dockhub or a private Harbor image storage platform to generate the corresponding image address for use; S4. Install and configure the Golang environment and the make command-line tool to compile and build the source code; specifically, on a Linux system, use the package manager to install Golang and make: apt install golang, apt install make; S5. Download and extract kubebuilder according to system requirements; S6. Use kubebuilder to build a Kubernetes CRD named serverruntime; serverruntime is used to bind the model format and the model service runtime image. S7. Package the server runtime and deploy it to the cluster; S8. Bind the correspondence between different training frameworks and the runtime images of the model service; S9. Use kubebuilder to build a Kubernetes CRD named modelserver; modelserver is used to provide the necessary information for model deployment. S10. Package the modelserver and deploy it to the cluster; S11. Create a modelserver resource. Simply specify the model's format and the model storage address (modelAddr) to complete the model deployment. In step S9, the modelserver includes the format field, replicas field, modelAddr field, and the standard podSpec field. The format field is used to identify the model format. Allowed values ​​are Paddle, Pytorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM. It corresponds to the format value of serverruntime. The corresponding model service runtime image address in serverruntime is found based on the format field value. This image is used as the POD image. The replicas field is used to identify the number of PODs created by the Deployment, that is, the number of instances of the model deployment. The default value is 1. The modelAddr field is used to identify the model storage address, and supports network paths; The standard podSpec field is used to describe basic information about the POD. The podSpec does not include an image field because the image field is determined by the format field. The process of creating the modelserver resource in step S11 is as follows: Create a Kubernetes Deployment resource based on the standard podPsec field. If the standard podSPec field is empty, then build the podSpec yourself. The image field below should be the runtime image address of the matching model service. The number of replicas for a Deployment is the value of the replicates field; Create an init container to download the image of the model whose storage address is identified in the modelAddr field, and use the hostpath data volume to mount the downloaded model to the / mnt / models directory of the container; Create a Kubernetes Service resource to act as a network proxy for the Service to the POD.

2. The method for deploying a machine learning model based on Kubernetes according to claim 1, characterized in that, The specific steps in step S1 for creating runtime images of model services for different training frameworks are as follows: S101. Select a suitable base image as the base image for the model service runtime image: Select the service images officially released by each training framework as the base image for the model service runtime image. S102. Write Python code app.py, which loads models from a fixed directory ( / mnt / models) and provides HTTP service. S103. Copy the code app.py to the / app directory inside the image; S104. Set the image's startup command to python / app / app.py; S105. Use the `docker build` command to build the image.

3. The method for deploying a machine learning model based on Kubernetes according to claim 1, characterized in that, In step S5, kubebuilder is a framework for building Kubernetes APIs. It helps developers create custom controllers and custom resource definitions. By using kubebuilder, developers can more easily extend the Kubernetes API to meet the needs of specific applications. According to system requirements, the appropriate version needs to be selected based on the operating system and CPU architecture.

4. The method for deploying a machine learning model based on Kubernetes according to claim 1, characterized in that, The serverruntime in step S6 includes the format field and the image field; The format field is used to identify the model format, and the allowed values ​​are Paddle, PyTorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM. The image field is used to identify the runtime image of the model service corresponding to the model format represented by the format field. The allowed value is the image address generated in step S3.

5. The method for deploying a machine learning model based on Kubernetes according to claim 1, characterized in that, Before binding the correspondence between different training frameworks and model service runtime images in step S8, create serverruntime CRDs with format values ​​of Paddle, Pytorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM respectively.

6. A machine learning model deployment device based on Kubernetes, characterized in that, The apparatus is used to implement the Kubernetes-based machine learning model deployment method as described in any one of claims 1 to 5; the apparatus comprises: The creation module is used to create runtime images of model services for different training frameworks; the training frameworks include Paddle, PyTorch, Tensorflow, Scikit-learn, XGBoost, and LightGBM. The loading module is used to fix the model service runtime image to load model files from the / mnt / models directory; The image building module is used to build the runtime image of the model service and push the image to the Dockhub or private Harbor image storage platform to generate the corresponding image address for use. The configuration module is used to install and configure the Golang environment and the make command-line tool to compile and build the source code; specifically, on Linux systems, Golang and make are installed using package managers: `apt install golang` and `apt install make`. The decompression module is used to download and decompress kubebuilder according to system requirements; The serverruntime build module is used to build Kubernetes CRDs using kubebuilder, and is named serverruntime. serverruntime is used to bind the model format to the model service runtime image. The serverruntime deployment module is used to package and deploy serverruntime to the cluster; The binding module is used to bind the correspondence between different training frameworks and the runtime images of the model service; The modelserver building block is used to build a Kubernetes CRD named modelserver using kubebuilder; modelserver provides the necessary information for model deployment. The modelserver deployment module is used to package the modelserver and deploy it to the cluster; The resource creation module is used to create modelserver resources. Simply specify the model's format and the model storage address (modelAddr) to complete the model deployment.

7. An electronic device, characterized in that, include: Memory and at least one processor; The memory contains computer programs; The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the Kubernetes-based machine learning model deployment method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the Kubernetes-based machine learning model deployment method as described in any one of claims 1 to 5.

Images