Container-based service deployment method, device and server

Abstract

This application provides a container-based service deployment method, apparatus, and server, relating to the field of data storage. The method includes: a server training data to obtain a model file; the server uploading the model file to a model repository for storage; the server determining the corresponding model service code based on the model file; the server constructing an image file based on the model service code; the server storing the image file in an image repository; the server pulling the image file corresponding to the service to be deployed from the image repository and downloading the corresponding model file from a third-party model repository to the target server; and the server running the image file and loading the model file within it to deploy the service. This method, by separating the model service code and the model file, significantly reduces the size of the image file and improves the utilization of container storage space.

Description

Technical Field

[0001] This application relates to the field of data storage, and more particularly to a container-based service deployment method, apparatus, and server. Background Technology

[0002] With the continuous development of artificial intelligence (AI) technology, AI services are being applied more and more widely in various fields. Container-based AI service deployment methods can directly port AI services to target servers by limiting and isolating resources, thus enabling the deployment of AI services on the target servers.

[0003] In existing technologies, model training and model service deployment are crucial components of AI service deployment. In these scenarios, servers can obtain different versions of model files during the model training phase based on different versions of training data. Different versions of model files require corresponding versions of model services. Furthermore, when deploying new model services based on containers, the server needs to build different image files based on the different versions of model files to achieve the deployment of different versions of model services.

[0004] However, in the above process, each version of the model file and model service needs to generate a corresponding image file. These image files occupy a lot of storage space, which can easily lead to low storage space utilization. Summary of the Invention

[0005] This application provides a container-based service deployment method, apparatus, and server to solve the problem in the prior art where image files occupy a large amount of storage space, easily leading to low storage space utilization.

[0006] Firstly, this application provides a container-based service deployment method, including:

[0007] The training data is processed and trained multiple times to obtain multiple model files, which are then uploaded to a model repository for storage.

[0008] The corresponding model service code is determined for each model file, and the image file built based on the model service code is stored in the image repository;

[0009] Based on the service to be deployed, extract the corresponding model file and image file to complete the deployment of the service.

[0010] Optionally, the model service code may include at least the service execution instructions for the service to be deployed and the target storage address of the model file on the target server.

[0011] Optionally, based on the service to be deployed, extract the model file and image file corresponding to the service to be deployed, specifically including:

[0012] Based on the storage address of the model initialization image in the deployment orchestration document of the service to be deployed, pull the model initialization image from the image repository to the target server;

[0013] Run the model initialization image so that the model initialization image downloads the model files from the model repository to the target server according to the model address in the deployment orchestration document;

[0014] The model initialization image is used to initialize the data model in the model file so that the data model can be used by the model service image in the image file corresponding to the service to be deployed.

[0015] Based on the image address in the deployment orchestration document, pull the image file from the image repository to the target server;

[0016] Run the model service image from the image file to complete the deployment of the service to be deployed on the target server.

[0017] Optionally, the method also includes:

[0018] Based on the shared storage space in the deployment orchestration document, configure an empty storage volume for the container in the shared storage space for data caching and temporary data storage.

[0019] Optionally, the training data can be processed and trained multiple times to obtain multiple model files, and these model files can be uploaded to a model repository for storage. Specifically, this includes:

[0020] The training data is processed to obtain the processed training data, and the data model is trained using the processed training data.

[0021] Based on the data model and the first preset rule, model version information is generated, which is used to uniquely identify the data model.

[0022] Store the data model and model version information in the model file;

[0023] Store the model file in the model repository, and add the model address of the model file in the model repository to the model file;

[0024] Repeat the above steps until the data model meets the preset requirements.

[0025] Optionally, the image file built based on the model service code can be stored in an image repository, specifically including:

[0026] Build a model service image based on the model service code;

[0027] Based on the model service image and the second preset rule, image version information is generated. The image version information is used to uniquely identify the model service image.

[0028] Store the model service image and image version information in the image file;

[0029] Store the image file in the image repository, and add the image address of the image file in the image repository to the image file.

[0030] Secondly, this application provides a container-based service deployment apparatus, comprising:

[0031] The processing module is used to process and train the training data multiple times to obtain multiple model files, and upload the model files to the model repository for storage; determine the corresponding model service code for each model file, and store the image file built based on the model service code in the image repository;

[0032] The deployment module is used to extract the model file and image file corresponding to the service to be deployed, so as to complete the deployment of the service.

[0033] Optionally, the model service code may include at least the service execution instructions for the service to be deployed and the target storage address of the model file on the target server.

[0034] Optionally, a deployment module is used specifically for:

[0035] Based on the storage address of the model initialization image in the deployment orchestration document of the service to be deployed, pull the model initialization image from the image repository to the target server;

[0036] Run the model initialization image so that the model initialization image downloads the model files from the model repository to the target server according to the model address in the deployment orchestration document;

[0037] The model initialization image is used to initialize the data model in the model file so that the data model can be used by the model service image in the image file corresponding to the service to be deployed.

[0038] Based on the image address in the deployment orchestration document, pull the image file from the image repository to the target server;

[0039] Run the model service image from the image file to complete the deployment of the service to be deployed on the target server.

[0040] Optionally, the deployment module is also used for:

[0041] Based on the shared storage space in the deployment orchestration document, configure an empty storage volume for the container in the shared storage space for data caching and temporary data storage.

[0042] Optionally, the processing module is specifically used for:

[0043] The training data is processed to obtain the processed training data, and the data model is trained using the processed training data.

[0044] Based on the data model and the first preset rule, model version information is generated, which is used to uniquely identify the data model.

[0045] Store the data model and model version information in the model file;

[0046] Store the model file in the model repository, and add the model address of the model file in the model repository to the model file;

[0047] Repeat the above steps until the data model meets the preset requirements.

[0048] Optionally, the processing module is specifically used for:

[0049] Build a model service image based on the model service code;

[0050] Based on the model service image and the second preset rule, image version information is generated. The image version information is used to uniquely identify the model service image.

[0051] Store the model service image and image version information in the image file;

[0052] Store the image file in the image repository, and add the image address of the image file in the image repository to the image file.

[0053] Thirdly, this application provides a server, including: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the container-based service deployment method of the first aspect and any possible design of the first aspect according to the computer program stored in the memory.

[0054] Fourthly, this application provides a computer-readable storage medium storing a computer program, wherein when at least one processor of a server executes the computer program, the server executes the container-based service deployment method of the first aspect and any possible design of the first aspect.

[0055] Fifthly, this application provides a computer program product comprising a computer program that, when executed by at least one processor of a server, enables the server to perform the container-based service deployment method of the first aspect and any possible design of the first aspect.

[0056] The container-based service deployment method, apparatus, and server provided in this application obtain training data by acquiring pre-provided initial data and organizing it; generate corresponding model files based on the training data; upload the model files to a model repository for storage; determine the corresponding model service code based on the model files; construct an image file based on the model service code; store the image file in an image repository; determine the target server based on the service to be deployed; pull the image file corresponding to the service to be deployed from the image repository to the target server through a container client; download the model file corresponding to the service to be deployed from a third-party model repository to the target server; run the image file and load the model file into the image file to deploy the service to be deployed. This method separates the model service code and model files, greatly reducing the size of the image file built using the model service code, improving the utilization of container storage space, and enhancing the efficiency of image file retrieval. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a schematic diagram illustrating a container-based service deployment scenario provided in an embodiment of this application.

[0059] Figure 2 A flowchart illustrating a container-based service deployment method provided in one embodiment of this application;

[0060] Figure 3 A flowchart of a model training process provided in one embodiment of this application;

[0061] Figure 4 A flowchart of a model service process provided in one embodiment of this application;

[0062] Figure 5 A flowchart illustrating the deployment of a model service is provided as an embodiment of this application;

[0063] Figure 6A schematic diagram of a container-based service deployment device provided in an embodiment of this application;

[0064] Figure 7 This is a schematic diagram of the hardware structure of a server provided in one embodiment of this application. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0066] The terms "first," "second," "third," "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate. For example, without departing from the scope of this document, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0067] Depending on the context, the word "if" as used here can be interpreted as "when," "when," or "in response to determination."

[0068] Furthermore, as used herein, the singular forms “a,” “one,” and “the” are intended to also include the plural forms, unless the context indicates otherwise.

[0069] It should be further understood that the terms “comprising” or “including” indicate the presence of features, steps, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups.

[0070] The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Therefore, “A, B, or C” or “A, B, and / or C” means “any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C”. Exceptions to this definition occur only when combinations of elements, functions, steps, or operations are inherently mutually exclusive in some way.

[0071] With the continuous development of artificial intelligence (AI) technology, AI services are being applied more and more widely in various fields. Container-based AI service deployment methods can directly port AI services to target servers by limiting and isolating resources, thus enabling the deployment of AI services on the target servers.

[0072] In existing technologies, AI service deployment can be divided into two stages: AI model training and model service deployment. In these scenarios, raw data accumulates as the model service is used. The server can analyze and process this raw data to obtain training data. As raw data accumulates, the training data iterates continuously. However, the training code for model training and the model service code for model services are rarely or never modified. During model training, the server can use artificial intelligence algorithms to learn and train the training data to generate a data model. That is, the server can obtain model files containing different versions of the data model based on different versions of the training data. When a new version of the model file is obtained, the server can determine its corresponding model service code. Different versions of the model file can correspond to the same model service code, or different versions of the model file can correspond to different model service codes. Therefore, after obtaining a new model file and determining its corresponding model service code, the server needs to build a new image file. This new image file can include the new model file and its corresponding model service code. When deploying a new model service based on a container, the server can determine the image file to download based on the version of the model file to be deployed, thereby achieving the deployment of the model service. Once the model service is deployed, the server can use the model files in the image file to recognize the data. However, this approach suffers from the problem of excessively large image files, increasing the time required to deploy the model service. Furthermore, each update to the model files and the model service code requires regenerating the image file, which increases the complexity of managing image files, wastes storage space, and leads to low storage utilization.

[0073] In containerized model service deployment scenarios, containerized model service deployment typically improves deployment efficiency and subsequent model file update efficiency. However, in the aforementioned model service deployment methods, the image file includes both model files and model service code. When either the model files or the model service code is updated, the server needs to rebuild the image file. With multiple iterations of model file updates and multiple optimizations of the model service code, the number of image files increases continuously. Furthermore, each built image file is not deleted during this process, leading to a significant waste of storage space due to the large number of image files stored on the server. In addition, model files typically occupy a large amount of memory, far exceeding that of the model service code. Building image files using both model files and model service code together can easily result in excessively large image files. During model service deployment, each deployment requires pulling the latest version of the image file to the target server. Excessively large image files can cause the model service to consume more network bandwidth during deployment. Moreover, the time spent pulling this image file during model service deployment is part of the model service startup time; an excessively large image file can also lead to longer model service startup times.

[0074] To address the issue of excessively large image files, a method of pre-storing models in shared storage has been proposed. This method separates model files from model service code. When building the image file, the server does not need to directly add the model files. Therefore, the image file does not need to be rebuilt after model file updates. However, this method is only applicable to clusters with high-level operational privileges. Furthermore, this method requires file upload and file management permissions in the shared storage space. Otherwise, the server will be unable to upload model files to the shared storage space, and the server will be unable to manage model file versions after they are uploaded. In other words, while the server does not need to rebuild the image file every time the model files are updated, it does need to update the version of the model files in the shared storage space. Therefore, the server typically needs to pre-deploy data storage types such as HostPath and PV in the shared storage space. Thus, this method suffers from high deployment complexity and stringent permission requirements.

[0075] To address the aforementioned issues, this application proposes a container-based service deployment method. In this method, the server can store model files generated during version iterations separately in a third-party model repository. These model files can be centrally managed within this repository. Using this model repository avoids model files occupying the container's storage space. Furthermore, for different model file versions, the server only needs to manage the model files themselves. The server no longer needs to build different image files for different versions of model files. During the development and use of model files, centralized management allows for more accurate recording of the model file iteration process. Engineers can use these model files to clearly understand the improvements made in each version and the problems discovered during testing. Therefore, the storage and management of model files in this third-party model repository provides engineers with more convenient conditions for optimizing and analyzing data models. In addition, when the model service code is updated as the algorithm is optimized, the server can use the updated model service code to build an image file. This method effectively reduces the number of image files built and improves the efficiency of image file usage. This image file can be stored in the container's image repository. This image file only contains the model service code. Therefore, this image file occupies relatively little memory. Using this image file can effectively improve the efficiency of pulling image files. In this application, the separation of model files and model service code makes the image file purer, effectively avoiding problems such as large container storage space consumption and long image file pulling time caused by building the image file together with model files and model service code. The server can also initialize model files within the image file using the model initialization image. When deploying services, the server can use the model initialization image to download model files from the model repository to the target server. Furthermore, the model initialization image can initialize the model files to a storage path agreed upon with the image file. After the model initialization image completes the initialization of the model files, the image file can directly call the model files in that directory to complete the deployment of the model service. Using this method can effectively reduce the complexity of image management, save image storage space, and reduce the image loading time when publishing model services.

[0076] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0077] Figure 1 This illustration shows a scenario diagram of a container-based service deployment according to an embodiment of this application. Figure 1As shown, during the development of a model service, engineers first need to select a suitable algorithm based on the model's use case and initial data. After selecting the algorithm, the server can learn from the existing data based on the selected algorithm to obtain a data model capable of processing similar data. This algorithm is typically a deep learning algorithm or a machine learning algorithm. After acquiring the data model, the server generates the corresponding model file. Engineers also need to develop the model service code for the corresponding algorithm based on the data model. The server can then build the developed model service code into an image file. The server can use this image file and the model file to deploy the model service. When this deployed model service is started, it can call the learned data model to intelligently process unfamiliar similar data.

[0078] During development, it's typically impossible to obtain the optimal data model in one go. The data model is then tested and validated, and based on the results, the data is processed multiple times to obtain different versions of training data. The server can use these different versions of training data to train the data model, resulting in multiple versions of model files. Furthermore, after testing and validation, the server can optimize the core algorithm based on the results and develop different versions of model service code. The server can update and deploy the model service after each update to the data model or model service code. Engineers can then obtain a satisfactory and compliant data model and corresponding model service code after multiple tests and validations.

[0079] In this application, a server is used as the execution entity to perform the container-based service deployment method of the following embodiments. Specifically, the execution entity can be a server hardware device, a software application implementing the following embodiments on the server, a computer-readable storage medium on which the software application implementing the following embodiments is installed, or code implementing the software application of the following embodiments.

[0080] Figure 2 A flowchart illustrating a container-based service deployment method according to an embodiment of this application is shown. Figure 1 Based on the illustrated embodiments, as Figure 2 As shown, with the server as the execution entity, the method in this embodiment may include the following steps:

[0081] S101. Process and train the training data multiple times to obtain multiple model files, and upload the model files to the model repository for storage.

[0082] In this embodiment, after obtaining pre-provided initial data, the server can organize the initial data to obtain training data. The server can then use an algorithm selected by the engineer to train the training data to obtain a data model. This algorithm can be a deep learning algorithm. The server can generate a corresponding model file based on the data model. The server can then upload the model file to a model repository for storage. This model repository can be a third-party storage space.

[0083] In one example, the model file may include a model name, storage address, model description information, model version information, and model testing information. The model name facilitates model identification and usage. Engineers can quickly determine the function of the model file and identify it based on the model name. The storage address provides download links for third parties. For example, a model initialization image can download the model file based on this storage address. The model description information includes a general description of the model's usage scenarios. This information helps users understand the scenarios in which the model will be used and the effects and functions it can achieve. The model version information records the version number of the data model during iterative updates. The model testing information records defects found during testing.

[0084] In one example, such as Figure 3 As shown, the process of generating this model file may specifically include the following steps:

[0085] Step 1: The server acquires initial data and generates training data based on it. The server can then train the resulting data model using the training data. During this process, such as... Figure 3 As shown, the server can first determine whether the initial data needs processing. If the initial data needs processing, the server can process it to obtain training data. Otherwise, if the initial data does not need processing, the server can directly generate training data based on the initial data.

[0086] Step 2: The server can generate model version information based on the data model and the first preset rule. The model version information is used to uniquely identify the data model.

[0087] Step 3: The server can store the data model and model version information in the model file.

[0088] Step 4: The server can push the model file to the model repository, thus storing the model file. During this process, the server can also obtain the model address of the model file in the model repository. The server can then add this model address to the model file.

[0089] Step 5: When the server determines that the model's effect does not meet the preset requirements, such as... Figure 3 As shown, the server can continue to process the training data. The server can then use the processed training data to continue training the model, obtaining the data model. That is, the server can repeat the above steps until the data model's performance meets a preset requirement. This preset requirement can be that the data model's test accuracy is greater than a preset value. This preset value can be an empirical value, such as 90%.

[0090] S102. Determine the corresponding model service code for each model file, and store the image file built based on the model service code in the image repository.

[0091] In this embodiment, the server can determine the corresponding model service code based on the algorithm selected by the engineer for the model. This model service code can be used to optimize and process the results of the data model, thereby generating the final result of the service. Engineers can develop different model service codes based on this algorithm, or they can develop different model service codes based on different algorithms. The server can also build an image file based on the model service code. The server can store this image file in an image repository. This image repository can be the memory inside a container.

[0092] In one example, the model service code includes at least the service execution instructions for the service to be deployed and the target storage address of the model file on the target server. This target storage address is the address where the model service code in the image file reads the model file during runtime. This target storage address can be the address agreed upon between the model service code and the model initialization image.

[0093] In one example, the image file may include an image name, image address, image version information, and image description information. The image name is used for easy image identification. The image address includes the image file's location in the image repository; that is, this image address is the download address for the image file. The server can retrieve the image file based on this image address. The image version information indicates the iteration number of the image file during its construction. The image description information includes information such as the use cases and functions of the model service image within the image file.

[0094] In one example, such as Figure 4 The image file generation process shown can correspond to, for example, the process of generating an image file. Figure 3This process involves steps before deployment and after model training. Before performing the following steps, the server needs to generate the corresponding model service code based on the model file. During service testing and verification, if the data processing and / or algorithm of the model file changes, the server will regenerate the model service code based on the changed data processing and / or algorithm. Alternatively, during service testing and verification, if the parameters in the model file change due to iteration or other reasons, but the data processing and algorithm of the model file remain unchanged, the server does not need to regenerate the model service code. The process of the server generating an image file based on the model service code may specifically include the following steps:

[0095] Step 1: The server can build a model service image based on the model service code.

[0096] Step 2: The server can generate image version information based on the model service image and the second preset rule. The image version information is used to uniquely identify the model service image.

[0097] Step 3: The server stores the model service image and image version information into the image file.

[0098] Step 4: The server pushes the image file to the image repository and adds the image address to the image file based on its image address in the repository. The server can then publish the model service by publishing this image file. When the model service needs to be updated, the server can rebuild the model service image based on the updated model file.

[0099] S103. Based on the service to be deployed, extract the model file and image file corresponding to the service to be deployed to complete the deployment of the service.

[0100] In this embodiment, the server can obtain the service to be deployed from the target server. The server can determine the model files and image files to be extracted based on this service. The server can pull image files from an image repository using a container client. The server can also pull a model initialization image from the image repository using a container client. The server can use this model initialization image to download the corresponding model files from a third-party model repository and initialize these model files to the target storage address on the target server. The image file can read the model files based on the target storage address, thereby enabling the deployment of the service to be deployed.

[0101] In one example, the deployment process of the service to be deployed can be as follows: Figure 5 Specifically, it includes the following steps:

[0102] Step 1: The server can determine the storage address of the model initialization image based on the deployment orchestration document of the service to be deployed. The server can then pull the model initialization image from the image repository to the target server based on this storage address. Pulling the model initialization image is implemented by the container client. The model initialization image can be a separate container. The deployment orchestration document can include the model initialization image address, model file address, image file address, and shared storage space.

[0103] Step 2: The server can run the model initialization image within a container on the target server. Once the model initialization image is running, it can obtain the target storage address from the deployment orchestration documentation. The model initialization image can also download model files from a third-party model repository to the target server.

[0104] Step 3: The server can use the model initialization image to initialize the data model in the model file. The initialized data model can be stored in the target storage address of the target server. The image file can directly access the model file from the target storage address of the target server. This target storage address can be a pre-agreed address between the image file and the model initialization image.

[0105] Step 4: The server can pull the image file directly from the image repository to the target server through the container client, based on the image address in the deployment orchestration document.

[0106] Step 5: The server can run the model service image in the image file to complete the deployment of the service to be deployed on the target server.

[0107] In one example, in addition to the steps described above, the following may also be included:

[0108] Step 6: The server can configure an empty storage volume for the container within the shared storage space, based on the shared storage space specified in the deployment orchestration document, for data caching and temporary data storage. This empty storage volume for temporary data storage can be a third-party model repository.

[0109] The container-based service deployment method provided in this application allows the server to obtain training data after acquiring pre-provided initial data. The server can then generate corresponding model files based on this training data. These model files can be uploaded to a model repository for storage. The server can determine the corresponding model service code based on the model files. The server can build an image file based on the model service code. This image file can be stored in an image repository. The server can determine the target server based on the service to be deployed. The server can pull the image file corresponding to the service to be deployed from the image repository to the target server using a container client. The server can also download the model file corresponding to the service to be deployed from a third-party model repository to the target server. The server can run the image file and load the model file within it to deploy the service. This application significantly reduces the size of the image file built using the model service code by separating the model service code from the model files, thereby improving the utilization of container storage space and the efficiency of image file retrieval.

[0110] Based on the above embodiments, this embodiment can use Kubernetes on a server to manage containerized services on multiple hosts in a cloud platform. Kubernetes is an orchestration tool for managing containers on various hosts in a cloud platform. Kubernetes provides containerized deployment for applications. Furthermore, Kubernetes supports container planning, updates, and maintenance. Containers have the advantages of low resource consumption and fast deployment. An application corresponding to a service can be packaged into an image file. The use of the image file in a container does not require combination with other service stacks. This characteristic of containers allows a service to run in a consistent runtime environment from R&D testing to production deployment. The specific process of using Kubernetes on a server to complete the above steps may include the following steps:

[0111] S201. Based on the features of Pods in Kubernetes, the server configures a temporary shared storage space called `emptyDir` within the Pod. This temporary shared storage space is used to store data model files. That is, this temporary shared storage space is the aforementioned model repository.

[0112] In the Kubernetes world, a Pod is the carrier of a service. A Pod can contain multiple containers. A Pod can be viewed as the smallest unit of scheduling in Kubernetes. Within Kubernetes, Pods can share underlying resources. The later stages of a Pod's lifecycle are managed by Kubernetes.

[0113] Here, `emptyDir` is a storage volume. This `emptyDir` is created when the Pod is specified to a target server. `emptyDir` is initially an empty directory and persists for the entire lifecycle of the Pod. When the Pod is removed, the data in `emptyDir` is also deleted. It's important to note that the crash of a container within the Pod does not cause `emptyDir` to be deleted. `emptyDir` can be mounted to any path under any container within the Pod. Furthermore, `emptyDir` can be shared by all containers within the Pod. Currently, the `emptyDir` disk is typically just regular space for the containers within the Pod. This `emptyDir` can be used for temporary data storage. Alternatively, it can serve as a backup point for containers recovering from a crash.

[0114] S202. Servers can use Kubernetes' initContainers to configure model initialization images. A model initialization image is simply an initContainer. A pod can run multiple containers, which can include one or more initContainers. InitiationContainers typically run before application containers within a pod. Once an initContainer starts running, it must run to completion. Furthermore, within a pod, one initContainer must successfully complete before another can run. If an initContainer in a pod fails, Kubernetes will attempt to restart the pod until the initContainer successfully completes. If the pod's restart policy is set to never, the pod will not restart. However, because the initContainer did not successfully complete, other initContainers will not be able to run. Currently, initContainers do not support lifecycle, livenessProbe, readinessProbe, and startupProbe. InitContainers can contain utilities or custom code that are not present in the service container. These utilities or custom code may be used during the installation of the service container. Using initContainers allows for safer execution of these tools, preventing them from compromising the security of the service image. Furthermore, container initialization can be run as root, allowing the execution of high-privilege commands. Once the container initialization process is complete, it exits without posing any security risks to the service container.

[0115] S203. The server can use Kubernetes containers to configure the image file of the model service. After the image file of the model service is deployed, it can provide the model service on the target server.

[0116] S204. The server can mount temporary shared storage space in both initContainers and containers.

[0117] Based on the steps above, when officially starting a service Pod, from the Pod's perspective, the service deployment can specifically include the following steps:

[0118] S301, the kubelet client in Kubernetes can parse deployment orchestration documents and pull image files and model initialization images to the target server.

[0119] S302. The server can create a temporary shared storage space called emptyDir.

[0120] S303. The server runs the model initialization image, downloads the model file from the model repository to the temporary shared storage space, and performs initialization processing on the model file.

[0121] S304. After the model initialization image is completed, it exits. The server continues to run the business container. This business container is the image file of the model service. After the image file of the model service is started, the model service can be deployed on the target server.

[0122] The deployment orchestration document (Deployment) can include multiple parameters. The `apiVersion` parameter specifies the API version. For example, this version could be `apps / v1`, and in the deployment orchestration document, it can be written as `apiVersion:apps / v1`. The value of the `apiVersion` parameter must be found in `kubectl api-versions`. The `kind` parameter specifies the role / type of the resource being created. For example, this role / type could be `Deployment`, and in the deployment orchestration document, it can be written as `kind:Deployment`. The `metadata` parameter indicates the resource's metadata / attributes. The `name` parameter indicates the resource's name. For example, the resource's name could be `model-demo`, and in the deployment orchestration document, it can be written as `name:model-demo`. The value of the `name` parameter must be unique within the same namespace. The `namespace` parameter indicates the namespace in which the deployment is performed. For example, the namespace name could be `default`, and in the deployment orchestration document, it can be written as `namespace:default`. The `labels` parameter sets the resource's labels. The `spec` parameter indicates the resource's specification fields. `replicas` declares the number of replicas. For example, the number of replicas can be 1. In this deployment orchestration document, it can be written as `replicas:1`. The `selector` parameter indicates the selector. The `matchLabels` parameter indicates the matching labels. The `template` parameter indicates the template. The `metadata` parameter indicates the resource's metadata / attributes. The `labels` parameter indicates the labels for the resource. The `spec` parameter indicates the resource specification fields. The `name` parameter indicates the name of the initialization container. For example, the name of the initialization container can be `model-initContainer`. In this deployment orchestration document, it can be written as `name:model-initContainer`. The `image` parameter indicates the container's address. For example, the address can be `registry.example.com / tools / model-initContainer:latest`. In this deployment orchestration document, it can be written as `image:registry.example.com / tools / model-initContainer:latest`. The `command` parameter is the command executed when the container starts. For example, the command can be `['xxx', 'xxx']`. In this deployment orchestration document, it can be written as `command:['xxx', 'xxx']`. The parameter volumeMounts is used to indicate the file mount directory.The mount directory is the mount directory configured within the container. The `volumeMounts` parameter can include the sub-parameter `mountPath`. `mountPath` specifies the directory to be mounted within the container. For example, this directory could be ` / model / path / model`. In this deployment orchestration document, it can be written as `-mountPath: / model / path / model`. The `name` parameter specifies the defined name. For example, this name could be `model-dir`. In this deployment orchestration document, it can be written as `name:model-dir`. This `name` parameter must correspond to the `volume` parameter below. The `name` parameter can also include the sub-parameter `name`. This sub-parameter `name` specifies the name of the container. For example, this name could be `model-service`. In this deployment orchestration document, it can be written as `-name:model-service`. The `image` parameter specifies the image address used by the container. For example, this image address could be `registry.example.com / inference / model-service:latest`. In this deployment orchestration document, it can be written as `image:registry.example.com / inference / model-service:latest`. The `command` parameter specifies the command parameters to execute when the container starts. For example, the command could be ['xxx','xxxx']. In the deployment orchestration document, it can be written as command: ['xxx','xxxx']. The parameter `imagePullPolicy` indicates the image pull strategy for each Pod startup. For example, this strategy could be `IfNotPresent`. In the deployment orchestration document, it can be written as `imagePullPolicy: IfNotPresent`. `IfNotPresent` can specifically include three policies: `Always`, `Never`, and `IfNotPresent`. `Always` checks every time, meaning a new image is pulled every time. `Never` does not check every time. `IfNotPresent` does not check if the image exists locally, and pulls it if it doesn't. When manually testing, if the image exists in a Docker container, `IfNotPresent` can be used. By default, `Always` is usually used. During deployment, if the image repository does not exist, the deployment will fail. The parameter `volumeMounts` indicates the file mount directory. This mount directory is the directory configured inside the container. The parameter `volumeMounts` includes the sub-parameter `mountPath`. The `mountPath` parameter specifies the directory to be mounted within the container. For example, this directory could be ` / model / path / model`.In this deployment orchestration document, it can be written as `-mountPath: / model / path / model`. Additionally, parameters such as `app`, `version`, `initContainers`, and `containers` can be included. The value of the `app` parameter can be `model-demo`, and it can be written as `app:model-demo` in the deployment orchestration document. The value of the `version` parameter can be `v1`, and it can be written as `version:v1` in the deployment orchestration document.

[0123] Figure 6 This application provides a schematic diagram of the structure of a container-based service deployment apparatus according to an embodiment of the present application. Figure 6 As shown, the container-based service deployment device 10 of this embodiment is used to implement the operations corresponding to the server in any of the above method embodiments. The container-based service deployment device 10 of this embodiment includes:

[0124] Processing module 11 is used to process and train the training data multiple times to obtain multiple model files, and upload the model files to a model repository for storage. It determines the corresponding model service code for each model file and stores the image file built based on the model service code in the image repository.

[0125] Deployment module 12 is used to ensure that the model service code includes at least the service execution instructions for the service to be deployed and the target storage address of the model file on the target server, based on the service to be deployed.

[0126] In one example, the model service code includes at least the running instructions for the service to be deployed and the target storage address.

[0127] In one example, deployment module 12 is specifically used for:

[0128] Based on the storage address of the model initialization image in the deployment orchestration document of the service to be deployed, pull the model initialization image from the image repository to the target server.

[0129] Run the model initialization image so that it downloads the model files from the model repository to the target server according to the model address in the deployment orchestration document.

[0130] The model initialization image is used to initialize the data model in the model file so that the data model can be used by the model service image in the image file corresponding to the service to be deployed.

[0131] Based on the image address in the deployment orchestration document, pull the image file from the image repository to the target server.

[0132] Run the model service image from the image file to complete the deployment of the service to be deployed on the target server.

[0133] In one example, deployment module 12 is also used for:

[0134] Based on the shared storage space in the deployment orchestration document, configure an empty storage volume for the container in the shared storage space for data caching and temporary data storage.

[0135] In one example, processing module 11 is specifically used for:

[0136] The training data is processed to obtain the processed training data, and the data model is trained using the processed training data.

[0137] Based on the data model and the first preset rule, model version information is generated, which is used to uniquely identify the data model.

[0138] Store the data model and model version information in the model file.

[0139] Store the model file in the model repository, and add the model address of the model file in the model repository to the model file.

[0140] Repeat the above steps until the data model meets the preset requirements.

[0141] In one example, processing module 11 is specifically used for:

[0142] Build the model service image based on the model service code.

[0143] Based on the model service image and the second preset rule, image version information is generated. The image version information is used to uniquely identify the model service image.

[0144] Store the model service image and image version information in the image file.

[0145] Store the image file in the image repository, and add the image address of the image file in the image repository to the image file.

[0146] The container-based service deployment device 10 provided in this application embodiment can execute the above method embodiment. Its specific implementation principle and technical effect can be found in the above method embodiment, and will not be repeated here.

[0147] Figure 7 A schematic diagram of the hardware structure of a server provided in an embodiment of this application is shown. Figure 7As shown, the server 20 is used to implement the operations corresponding to the server in any of the above method embodiments. The server 20 in this embodiment may include: a memory 21 and a processor 22.

[0148] The memory 21 is used to store computer programs. The memory 21 may include high-speed random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0149] Processor 22 is used to execute computer programs stored in memory to implement the container-based service deployment method in the above embodiments. For details, please refer to the relevant descriptions in the foregoing method embodiments. The processor 22 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0150] Alternatively, the memory 21 can be either standalone or integrated with the processor 22.

[0151] When the memory 21 is a device independent of the processor 22, the server 20 may also include a bus 23. This bus 23 is used to connect the memory 21 and the processor 22. The bus 23 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0152] The server provided in this embodiment can be used to execute the container-based service deployment method described above. Its implementation and technical effects are similar, and will not be repeated here.

[0153] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0154] The computer-readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of a computer program from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a computer-readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the computer-readable storage medium. Of course, the computer-readable storage medium can also be a component of the processor. The processor and the computer-readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the ASIC can reside in a user equipment. Of course, the processor and the computer-readable storage medium can also exist as discrete components in a communication device.

[0155] Specifically, the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0156] This application also provides a computer program product comprising a computer program stored in a computer-readable storage medium. At least one processor of the device can read the computer program from the computer-readable storage medium, and the at least one processor executes the computer program to cause the device to implement the methods provided in the various embodiments described above.

[0157] This application also provides a chip including a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device with the chip installed performs the methods described in the various possible implementations above.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0159] The modules can be physically separate, for example, installed in different locations within a single device, installed on different devices, distributed across multiple network units, or distributed across multiple processors. Alternatively, the modules can be integrated, for example, installed in the same device, or integrated into a single codebase. The modules can exist in hardware form, software form, or a combination of both. This application can select some or all of the modules to achieve the objectives of this embodiment based on actual needs.

[0160] When the various modules are implemented as integrated software functional modules, they can be stored in a computer-readable storage medium. The aforementioned software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0161] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A container-based service deployment method, characterized by, The method includes: The training data is processed and trained multiple times to obtain multiple model files, which are then uploaded to a model repository for storage; the model repository is a third-party storage space. The corresponding model service code is determined for each model file, and the image file built based on the model service code is stored in the image repository; the image repository is the memory of the container and is used to store the model service code; the model service code includes at least the service execution instructions of the service to be deployed and the target storage address of the model file on the target server; According to the storage address of the model initialization image in the deployment orchestration document of the service to be deployed, pull the model initialization image from the image repository to the target server; Run the model initialization image so that the model initialization image downloads the model file from a third-party model repository to the target server according to the model address in the deployment orchestration document; The model initialization image is used to initialize the data model in the model file so that the data model can be used by the model service image in the image file corresponding to the service to be deployed; After the model initialization image is completed, the image file is pulled from the image repository to the target server according to the image address in the deployment orchestration document; Run the model service image in the image file to complete the deployment of the service to be deployed on the target server.

2. The method according to claim 1, characterized in that, The method further includes: According to the shared storage space in the deployment orchestration document, an empty storage volume for data caching and temporary data storage is configured for the container in the shared storage space.

3. The method according to claim 1 or 2, characterized in that, The process of processing and training the training data multiple times to obtain multiple model files, and then uploading the model files to a model repository for storage, specifically includes: The training data is processed to obtain the processed training data, and the data model is trained using the processed training data. Based on the data model and the first preset rule, model version information is generated, and the model version information is used to uniquely identify the data model. The data model and the model version information are stored in the model file; The model file is stored in the model repository, and the model address of the model file in the model repository is added to the model file; Repeat the above steps until the model effect of the data model meets the preset requirements.

4. The method according to any one of claims 1-3, characterized in that, The step of storing the image file built based on the model service code into the image repository specifically includes: Based on the model service code, construct the model service image; Based on the model service image and the second preset rule, image version information is generated, and the image version information is used to uniquely identify the model service image. The model service image and the image version information are stored in the image file; The image file is stored in the image repository, and the image address of the image file in the image repository is added to the image file.

5. A container-based service deployment device, characterized in that, The device includes: The processing module is used to process and train the training data multiple times to obtain multiple model files, and upload the model files to a model repository for storage; generate a model service code based on each model file, and store the image file built based on the model service code in an image repository; the model repository is a third-party storage space, and the image repository is the memory of a container used to store the model service code; the model service code includes at least the service execution instructions of the service to be deployed and the target storage address of the model file on the target server; The deployment module is used to: pull the model initialization image from the image repository to the target server according to the storage address of the model initialization image in the deployment orchestration document of the service to be deployed; run the model initialization image so that it downloads the model file from a third-party model repository to the target server according to the model address in the deployment orchestration document; initialize the data model in the model file using the model initialization image so that the data model can be used by the model service image in the image file corresponding to the service to be deployed; after the model initialization image is executed, pull the image file from the image repository to the target server according to the image address in the deployment orchestration document; and run the model service image in the image file to complete the deployment of the service to be deployed on the target server.

6. A server, characterized in that, The server includes: a memory and a processor; the memory is used to store computer programs; the processor is used to implement the container-based service deployment method as described in any one of claims 1 to 4 according to the computer programs stored in the memory.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, is used to implement the container-based service deployment method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the container-based service deployment method according to any one of claims 1 to 4.

Images