Pod creation method and device, computer equipment and storage medium

Through the pod creation method that automatically checks and matches container images, the problem of AI application version mismatch in Kubernetes clusters is solved, which improves deployment efficiency and success rate, and simplifies the user configuration process.

CN119002937APending Publication Date: 2024-11-22JINAN INSPUR DATA TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410989205.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In Kubernetes cluster, the version mismatch between the deep learning framework, CUDA library and GPU driver version of AI application causes the AI ​​application to fail to start. Users need to configure the correct mirror version through trial and error loops, which is inefficient.

Method used

Provide a pod creation method that automatically checks and matches container images. By receiving pod creation requests from AI applications, obtaining configuration information, obtaining target container images, and using preset dependency mapping tables to check dependency conflicts, automatically configure node affinity information. , ensure that the mirror version is compatible.

Benefits of technology

It simplifies the user configuration process, reduces the risk of errors caused by version mismatch, improves the deployment efficiency and success rate of AI applications on K8S clusters, and reduces troubleshooting and maintenance work.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002937A_ABST
    Figure CN119002937A_ABST
Patent Text Reader

Abstract

The invention relates to a pod creation method and device, computer equipment and a storage medium. The method comprises the following steps: receiving a pod creation request of an AI application; in response to the fact that the to-be-created pod is not configured with mirror image information, obtaining configuration information of the to-be-created pod; obtaining a target container mirror image from a preset mirror image warehouse according to the configuration information, and modifying mirror image information of the target container mirror image into the configuration information; judging whether the mirror image information of the target container mirror image and the configuration information of the to-be-created pod have dependency conflicts or not according to a preset dependency mapping table; if not, obtaining a matched node according to the configuration information, configuring node affinity information for the to-be-created pod, and adding the node affinity information to the configuration information; and creating the pod according to the configuration information. By adopting the method, before pod creation, the cluster automatically selects the container mirror image meeting the requirement for the user, and checks the dependency relationship in the configuration information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a pod creation method, apparatus, computer device and storage medium. Background Art

[0002] In the current container cloud platform environment, Kubernetes (K8S) clusters provide powerful resource management and scheduling capabilities for model development, training, and reasoning in the AI ​​field. Users start the runtime environment by defining pods, where the configuration of container images is crucial, including the runtime environment and dependent libraries of the application. For AI development, training, and reasoning, the most important dependent library is the deep learning framework.

[0003] like Figure 1 As shown in the figure, users use K8S to execute AI applications. However, with the rapid development of AI technology, GPU drivers, CUDA libraries, and deep learning frameworks are also continuously upgraded. New versions are often not backward compatible, which causes AI applications to rely on a specific version or a certain range of versions of the deep learning framework, and the deep learning framework relies on a specific version or version range of CUDA and GPU drivers, especially the GPU driver version of the host where the runtime container is located. When the dependencies at the above levels do not match, the user's AI application will fail to start or will not work properly.

[0004] The current solution requires users to study and understand these complex version dependencies on their own to select the correct image configuration. Once the configuration is improper, users must go through a trial-and-error cycle, waiting for their pod to be created, scheduled, and started, and then their AI application will start running abnormally before they can get information feedback about the configuration error, and then modify the relevant image configuration again to enter the next round of debugging. It is time-consuming, labor-intensive, and inefficient to constantly modify the image debugging to select a version that meets the needs. Summary of the invention

[0005] Based on this, it is necessary to provide a pod creation method, device, computer equipment and storage medium that can automatically check and match container images to address the above technical problems.

[0006] On the one hand, a pod creation method is provided, the method comprising:

[0007] Receive pod creation requests from AI applications;

[0008] In response to the pod to be created not being configured with image information, obtaining configuration information of the pod to be created;

[0009] Obtain a target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information;

[0010] Check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to the preset dependency mapping table;

[0011] If not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information;

[0012] The pod is created according to the configuration information.

[0013] In one embodiment, the configuration information of the pod to be created includes at least one of image information and deep learning framework version information, and the method further includes:

[0014] Call the admission control plug-in to intercept the pod to be created, and check whether the pod to be created is configured with image information;

[0015] In response to the pod configuration image information to be created, the image information is extracted as the target container image.

[0016] In one embodiment, the configuration information of the pod to be created also includes CUDA version information and / or GPU driver version information; and obtaining the target container image from a preset image repository according to the configuration information includes:

[0017] Create the image repository according to the matching relationship between the deep learning framework version information, CUDA version information, GPU driver version information and the container image;

[0018] According to one or more of the deep learning framework version information, the CUDA version information, and the GPU driver version information, a target container image that matches the configuration information of the pod to be created is obtained from the image repository.

[0019] In one of the embodiments, after obtaining the target container image from the preset image repository according to the configuration information and modifying the image information of the target container image into the configuration information, it also includes:

[0020] Call the admission control plug-in to intercept the pod to be created, and check whether the pod to be created is configured with image information;

[0021] If not, the creation fails.

[0022] In one of the embodiments, checking whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to the preset dependency mapping table includes:

[0023] Building a dependency mapping table based on the dependency relationship between the deep learning framework version and the CUDA version, and the dependency relationship between the CUDA version and the GPU driver version, and storing the dependency mapping table in the admission control plug-in;

[0024] In response to the presence of CUDA version information and / or GPU driver version information in the configuration information of the pod to be created and the image information of the target container image, checking whether there is a dependency conflict between the image information and the configuration information according to the dependency mapping table;

[0025] If yes, the creation fails.

[0026] In one embodiment, acquiring a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding the node affinity information to the configuration information includes:

[0027] Obtaining GPU driver version information of each node, and reporting the GPU driver version information to the admission control plug-in to form a node information table;

[0028] In response to the configuration information of the pod to be created including GPU driver version information, matching the GPU driver version information with the node information table to obtain a matching result;

[0029] Determine, according to the matching result, whether there is a node matching the pod to be created;

[0030] If not, the creation fails;

[0031] If yes, configure node affinity information for the pod to be created according to the matching result;

[0032] In response to the pod to be created not being configured with GPU driver version information, acquiring, according to the dependency mapping table, a set of GPU driver version information that matches the configuration information of the pod to be created;

[0033] Matching the GPU driver version information set with the node information table to obtain a matching result;

[0034] Determine, according to the matching result, whether there is a node matching the pod to be created;

[0035] If not, the creation fails;

[0036] If yes, node affinity information is configured for the pod to be created according to the matching result.

[0037] In one embodiment, creating the pod according to the configuration information includes:

[0038] Scheduling the pod to be created to a matching node according to the node affinity;

[0039] Pull the target container image, create and start the pod according to the configuration information, and execute the AI ​​application.

[0040] On the other hand, a pod creation device is provided, the device comprising:

[0041] The receiving module is used to receive the pod creation request of the AI ​​application;

[0042] An acquisition module, configured to acquire configuration information of the pod to be created in response to the pod to be created not being configured with image information;

[0043] A first matching module, configured to obtain a matching target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information;

[0044] A checking module is used to check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to a preset dependency mapping table;

[0045] A second matching module is used for, if not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information;

[0046] A creation module is used to create the pod according to the configuration information.

[0047] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0048] Receive pod creation requests from AI applications;

[0049] In response to the pod to be created not being configured with image information, obtaining configuration information of the pod to be created;

[0050] Obtain a target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information;

[0051] Check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to the preset dependency mapping table;

[0052] If not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information;

[0053] The pod is created according to the configuration information.

[0054] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0055] Receive pod creation requests from AI applications;

[0056] In response to the pod to be created not being configured with image information, obtaining configuration information of the pod to be created;

[0057] Obtain a target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information;

[0058] Check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to the preset dependency mapping table;

[0059] If not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information;

[0060] The pod is created according to the configuration information.

[0061] The above-mentioned pod creation method, device, computer equipment and storage medium, by modifying the pod definition configuration, changes the user's configuration of the image into the user's configuration of the demand. When the user submits a pod related to the creation of an AI application, there is no need to pay attention to the dependency between the deep learning framework version and the CUDA version, GPU driver version, nor to directly specify the image. Instead, the cluster automatically selects a target container image that meets the requirements for the user from the image repository, thereby simplifying the user's learning cost and workflow. The dependency mapping table can automatically check the dependency between the pod configuration to be created and the target container image, as well as the matching relationship between the pod to be created and each node. If an incorrect configuration with a dependency conflict occurs, it can immediately fail and return in the creation stage, reducing the risk of errors caused by version mismatches, while reducing the troubleshooting and maintenance work caused by version mismatches, thereby improving the deployment efficiency and success rate of AI applications on the K8S cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A flowchart of the process of using K8S to perform AI tasks in the existing technology;

[0063] Figure 2 A schematic diagram of a process of creating a pod in one embodiment;

[0064] Figure 3 A schematic diagram of a process of creating a pod in another embodiment;

[0065] Figure 4 A structural block diagram of a pod creation device in one embodiment;

[0066] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] Kubernetes: An open source container orchestration platform, referred to as K8S. It can manage a group of hosts running Linux containers to form a K8S cluster, and automatically deploy, manage, and scale containerized applications.

[0069] Pod: It is the basic unit of resource scheduling and management in Kubernetes. It consists of one or more containers (such as Docker containers). Usually these containers share storage, network, and operating specifications.

[0070] Container image: A lightweight, portable software packaging method that packages an application and all its runtime environments, system libraries, and dependencies into a single file. This file can be loaded and run by containerized platforms (such as Docker, Kubernetes, etc.), enabling rapid deployment and execution of applications.

[0071] CUDA: A GPU parallel computing platform and programming model launched by NVIDIA. Developers can develop high-performance parallel programs running on GPU hardware based on the CUDA tool library. Currently, NVIDIA GPU is the most mainstream hardware used for model training and reasoning in the field of artificial intelligence. To use NVIDIA GPU software, the underlying system needs to have support from graphics card drivers and CUDA libraries.

[0072] Deep learning framework: A software framework used in the field of artificial intelligence to simplify and accelerate the development, training and deployment of deep learning models. It provides advanced programming interfaces and tools, allowing developers to use high-level abstractions to define neural network structures, process data, optimize model parameters, etc., without having to write underlying complex computing code. The current mainstream deep learning frameworks include tensorflow, pytorch, mxnet, etc.

[0073] AI application: After the user creates a pod in the K8S cluster, the model development and debugging, training and reasoning services and other related program applications run in the pod.

[0074] In one embodiment, Figure 2 and Figure 3 As shown, a pod creation method is provided, including the following steps:

[0075] Step S1, receiving a pod creation request of an AI application.

[0076] When using K8S for development, training, or inference tasks, users generally need to define one or a group of pods to start the runtime environment, and one of the most important field attributes in the pod definition is the container image configuration. This method optimizes K8S's original description of the pod, and no longer requires container image configuration as a required item. When the user initiates a pod creation request for an AI application, one or more of the image information, deep learning framework version information, CUDA version information, and GPU driver version information can be selected in the pod definition, and at least one of the image information and deep learning framework version information must be filled in. Therefore, users can configure the pod image information in the original way, or configure the deep learning framework version information instead of the image information. Optionally, users can additionally configure the CUDA version and GPU driver version information based on their own knowledge.

[0077] In the above pod creation method, users no longer need to understand and specify specific container images in detail when configuring pods, thereby reducing users' learning costs and configuration difficulty. At the same time, it avoids compatibility issues caused by version conflicts between image versions, framework versions, CUDA versions, or GPU versions, reduces failures caused by incorrect configurations, and improves configuration efficiency and success rate.

[0078] Step S2: In response to the pod not being configured with image information, obtaining configuration information of the pod.

[0079] In one possible implementation, the configuration information of the pod to be created includes at least one of image information and deep learning framework version information, and the method further includes: calling an access control plug-in to intercept the pod to be created, and checking whether the pod to be created is configured with image information; in response to the pod to be created being configured with image information, extracting the image information as the target container image.

[0080] Specifically, create a set of Admission Webhook admission control plug-ins and develop Admission Webhookmutate interface. After the K8S apiserver receives the pod creation request, call the mutate interface to intercept the pod creation and check the pod configuration. If the pod configuration explicitly configures the container image, it will be released without modification; if the container image is not configured in the pod, query the image repository for available container images with the pod configuration information as a parameter, and modify it to the pod image configuration.

[0081] In the above pod creation method, the configuration of the pod can be checked before the pod is created. If the container image information is not configured, the webhook mechanism can be used to help the user complete the image configuration information according to the configuration information. If there is any situation that may cause failure, the creation failure is triggered in advance, rather than waiting for the pod to be created and scheduled to start before the execution fails.

[0082] Step S3: Obtain a target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information.

[0083] In one possible implementation, the configuration information of the pod to be created also includes CUDA version information and / or GPU driver version information; the specific steps of obtaining the target container image from the preset image repository according to the configuration information include: creating the image repository according to the matching relationship between the deep learning framework version information, CUDA version information and GPU driver version information and the container image; obtaining the target container image that matches the configuration information of the pod to be created from the preset image repository according to one or more of the deep learning framework version information, CUDA version information and GPU driver version information.

[0084] It should be noted that when the pod is not configured with a container image, the query parameter information can be entered through the query interface of the image warehouse to return the available container images to complete the image information of the pod. The query parameters here can be one or more of the deep learning framework version information, CUDA version information and GPU driver version information, where CUDA version information and GPU driver version information are optional configuration information. The image warehouse can adopt a one-to-many mapping strategy, that is, each specific deep learning framework version may have multiple container images to choose from. At this time, entering the query parameter can query multiple corresponding container images, and each container image meets the requirements. The first container image is selected as the target container image by default, and it can also be selected according to other strategies; the container image in the image warehouse can also be mapped one-to-one with the deep learning framework, that is, the unique corresponding container image can be obtained from the image warehouse with the deep learning framework version as a parameter. At this time, the container image is directly extracted as the target container image, and its image information is added to the configuration information of the pod to ensure that the pod can correctly reference the image when it is created.

[0085] Preferably, based on the historical startup data and usage frequency information of the image repository, a usage frequency model of the container image is constructed using a machine learning algorithm;

[0086] Based on the usage frequency model, predict the usage probability of the container images in the image repository, and take the container images with a predicted probability greater than a preset threshold as high-frequency images;

[0087] Pre-pulling and caching the high-frequency image to a local node;

[0088] When the image repository adopts a one-to-many mapping strategy, multiple matching candidate container images can be obtained from the image repository based on the pod configuration information;

[0089] Obtain historical performance data of the candidate container image, including but not limited to startup time, resource consumption, image size, and dependency complexity, where startup time refers to the events required to start the container each time, resource consumption refers to CPU and memory usage during runtime, image size refers to the time and network bandwidth required to download and load the image, and dependency complexity refers to the number and types of dependencies contained in the image;

[0090] Based on the historical performance data, a linear combination or a machine learning model is used to calculate the priority score of the candidate container image;

[0091] The candidate container image with the highest priority score is selected as the target container image.

[0092] In the above pod creation method, by creating an image repository and parsing the deep learning framework version and other information of the pod configuration to obtain the matching target container image, the image information in the pod configuration can be automatically completed without manual intervention, reducing the time and learning cost of manual configuration. In addition, by matching the deep learning framework, CUDA and GPU driver versions, it can ensure that the target container image is fully compatible with the pod's operating environment, avoiding operating errors or performance problems caused by version mismatches, and improving deployment efficiency and accuracy.

[0093] Step S4: determine whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to the preset dependency mapping table.

[0094] It should be noted that image information refers to the manifest information of the container image, which contains the key information required to build and manage the container image. This application adds the deep learning framework version information to the image manifest file used by the AI ​​application. Optionally, it also adds the CUDA version information and GPU driver version information that the deep learning framework depends on. Since these image information and the dependent library versions in the pod configuration information may conflict, it is necessary to perform a dependency matching check on the two. It should be noted that since the CUDA version information and GPU driver version information are optional redundant information, when the pod and container image are not configured with this information, there is no need to check the dependency matching relationship between the two.

[0095] In one possible implementation, after obtaining the target container image from the preset image repository according to the configuration information and modifying the image information of the target container image into the configuration information, it also includes: calling the admission control plug-in to intercept the pod to be created, and checking whether the pod to be created is configured with image information; if not, the creation fails.

[0096] Specifically, the Admission Webhook validate interface is developed. Before checking whether there is a dependency conflict, the pod configuration is checked through the validate interface. If the container image is not configured in the pod configuration, allowed: false is returned, causing the pod creation to fail immediately. If the pod is configured with a container image, a dependency matching check is performed on the dependent library version in the image information to check whether there is a dependency conflict. If there is a conflict, the creation fails directly. If there is no conflict, the pod creation is allowed.

[0097] In the above pod creation method, the admission control plug-in's advance check mechanism is used to check before the pod is created to ensure that the pod has correctly obtained the target container image from the image repository and added the image information to the configuration information, preventing the creation of a pod without image information, thereby avoiding operation errors and resource waste.

[0098] In one possible implementation, checking whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to a preset dependency mapping table includes: building a dependency mapping table based on the dependency relationship between the deep learning framework version and the CUDA version, and the dependency relationship between the CUDA version and the GPU driver version, and storing the dependency mapping table in the access control plug-in; in response to the presence of CUDA version information and / or GPU driver version information in the configuration information of the pod to be created and the image information of the target container image, checking whether there is a dependency conflict between the image information and the configuration information according to the dependency mapping table; if so, the creation fails.

[0099] Specifically, a possible structure example of a dependency mapping table is as follows:

[0100]

[0101] Among them, a deep learning framework version tensorflow can match multiple versions of CUDA framework, and a CUDA version can match multiple versions of GPU driver versions. Therefore, based on the dependency matching relationship between the deep learning framework version, CUDA version and GPU driver version, a dependency mapping table is established and maintained in the admission control plug-in. The pod is intercepted and its configuration is checked through the validate interface. At the same time, the manifest of the target container image is obtained to obtain the deep learning framework version information, CUDA version information and GPU driver version information in the image information, and the dependency matching check is performed with the version information in the pod definition. The dependency mapping table is used to determine whether there is a dependency conflict between the two configurations. If there is a conflict, the creation fails directly. If there is no conflict, the pod creation is released.

[0102] In the above pod creation method, the pre-check mechanism of the admission control plug-in checks the dependency consistency between the container image and the pod configuration before the pod is created, which can prevent runtime errors. By maintaining a detailed dependency mapping table, it ensures that the pod configuration information fully matches the dependency library version declared in the container image, avoiding pod startup failures due to environmental incompatibility.

[0103] Step S5: If not, obtain a node matching the pod to be created according to the configuration information, configure node affinity information for the pod to be created and add it to the configuration information.

[0104] In one possible implementation, the specific steps of obtaining the node matching the pod to be created according to the configuration information, configuring the node affinity information for the pod to be created and adding it to the configuration information include: obtaining the GPU driver version information of each node, and reporting the GPU driver version information to the admission control plug-in to form a node information table; in response to the configuration information of the pod to be created including the GPU driver version information, matching the GPU driver version information with the node information table to obtain a matching result; judging whether there is a node matching the pod to be created according to the matching result; if not, the creation fails; if so, configuring the node affinity information for the pod to be created according to the matching result; in response to the pod to be created not being configured with the GPU driver version information, obtaining a set of GPU driver version information matching the configuration information of the pod to be created according to the dependency mapping table; matching the set of GPU driver version information with the node information table to obtain a matching result; judging whether there is a node matching the pod to be created according to the matching result; if not, the creation fails; if so, configuring the node affinity information for the pod to be created according to the matching result.

[0105] Specifically, we develop a GPUDriverSniffer daemonset and run a log collector on each node to centralize log data, collect the GPU driver version of each node and the availability information of the node, and report it to the admission control plug-in to form a node information table.

[0106] When a pod creation request arrives, the admission control plug-in intercepts and parses the pod configuration information. If the pod is configured with GPU version information, the version is matched with the node information table to find out whether there is a node with the same GPU driver version. If not, the creation fails. If so, it is released. If the GPU version information is not configured in the pod configuration, the feasible GPU driver version range is obtained based on the dependency mapping relationship between the deep learning framework version and the CUDA version, and matched with the node information table to find out whether there are nodes with intersections. If at least one matching node is found, the node affinity is modified for the pod through the admission control plug-in to ensure that the pod is scheduled to a matching node. If no node matches the GPU driver version or the corresponding GPU driver version range configured in the pod, the pod creation fails and an error message is returned to the user.

[0107] In the above pod creation method, the node information table is formed by reporting the GPU driver version information of the node to the admission control difference set, and the GPU driver version on the pod and the node is accurately matched, which can avoid resource waste due to driver incompatibility. For pods that do not specify the GPU driver version, the allowed GPU driver version range will be found according to the dependency mapping table, and matching nodes will be selected from it to ensure that each pod can run on the most suitable node, thereby improving the effective utilization of GPU and other hardware resources.

[0108] Step S6: Create the pod according to the configuration information.

[0109] In one possible implementation, the specific steps of creating the pod according to the configuration information include: scheduling the pod to be created to a matching node according to the node affinity; pulling the target container image, creating and starting the pod according to the configuration information to execute the AI ​​application.

[0110] For example, when creating a pod, the user defines the deep learning framework version as 1.0.0 and the CUDA version as 8.0, but does not specify the GPU driver version. First, a compatible container image is obtained from the image repository through the deep learning version and CUDA version, and then a set of GPU driver versions compatible with CUDA8.0 is obtained according to the dependency mapping table, such as version 347.62 and version 353.66. Then, the GPU driver version set corresponding to the pod is matched with the node information table through the admission control plug-in. It is found that the GPU driver versions of two nodes in the cluster are 347.62 and 353.66 respectively. Therefore, the node affinity is modified for the pod to ensure that it will be scheduled to one of the two nodes.

[0111] In the above pod creation method, when users submit pods related to AI applications, they do not need to pay attention to the version dependencies between the deep learning framework and the CUDA library and GPU driver, nor do they need to specify the image. Instead, they can directly specify the deep learning framework version, and the cluster will automatically select a container image that meets the requirements for the user; or users can configure one or more of the container image, deep learning framework version, CUDA version, GPU driver version, etc. according to their knowledge. The system will check the matching of the above configurations of the user and the GPU driver version of the node. If an incorrect configuration with a dependency conflict occurs, it will immediately fail and return in the creation stage, reducing the troubleshooting and maintenance work caused by version mismatch.

[0112] It should be understood that although Figure 2-3The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-3 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0113] In one embodiment, Figure 4 As shown, a pod creation device is provided, including: a receiving module, an acquisition module, a first matching module, a checking module, a second matching module and a creation module, wherein:

[0114] The receiving module is used to receive the pod creation request of the AI ​​application;

[0115] An acquisition module, configured to acquire configuration information of the pod to be created in response to the pod to be created not being configured with image information;

[0116] A first matching module, configured to obtain a matching target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information;

[0117] A checking module is used to check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to a preset dependency mapping table;

[0118] A second matching module is used for, if not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information;

[0119] A creation module is used to create the pod according to the configuration information.

[0120] In one embodiment, the acquisition module is also used to call the admission control plug-in to intercept the pod, and check whether the pod is configured with image information through the admission control plug-in; in response to the pod being configured with image information, extract the image information as the target container image.

[0121] In one embodiment, the first matching module is further used to create the image repository according to the matching relationship between the deep learning framework version information, CUDA version information and GPU driver version information and the container image; and obtain the target container image that matches the configuration information of the pod from the preset image repository according to one or more of the deep learning framework version information, CUDA version information and GPU driver version information.

[0122] In one embodiment, the judgment module is further used to call the admission control plug-in to intercept the pod, and check whether the pod is configured with image information through the admission control plug-in; if not, the pod creation fails.

[0123] In one embodiment, the judgment module is also used to construct a dependency mapping table based on the dependency relationship between the deep learning framework version and the CUDA version, and the dependency relationship between the CUDA version and the GPU driver version, and store the dependency mapping table in the access control plug-in; in response to the presence of CUDA version information and / or GPU driver version information in the configuration information of the pod and the image information of the target container image, determine whether there is a dependency conflict between the image information and the configuration information according to the dependency mapping table; if so, the pod creation fails.

[0124] In one embodiment, the second matching module is further used to obtain the GPU driver version information of each node, and report the GPU driver version information to the admission control plug-in to form a node information table; in response to the configuration information of the pod including the GPU driver version information, the GPU driver version information is matched with the node information table to obtain a matching result; according to the matching result, it is determined whether there is a node matching the pod; if not, the pod creation fails; if so, the node affinity is modified for the pod according to the matching result; in response to the pod not being configured with the GPU driver version information, a set of GPU driver version information matching the target image container is obtained according to the dependency mapping table; the GPU driver version information set is matched with the node information table to obtain a matching result; according to the matching result, it is determined whether there is a node matching the pod; if not, the pod creation fails; if so, the node affinity is modified for the pod according to the matching result.

[0125] In one embodiment, the creation module is also used to schedule the pod to be created to a matching node according to the node affinity; pull the target container image, create and start the pod according to the configuration information to execute the AI ​​application.

[0126] For the specific limitations of the pod creation device, please refer to the limitations of the pod creation method above, which will not be repeated here. Each module in the above-mentioned pod creation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0127] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store mapping relationship data between dependent library versions. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a pod creation method is implemented.

[0128] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0129] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0130] Receive pod creation requests from AI applications;

[0131] In response to the pod to be created not being configured with image information, obtaining configuration information of the pod to be created;

[0132] Obtain a target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information;

[0133] Check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to the preset dependency mapping table;

[0134] If not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information;

[0135] The pod is created according to the configuration information.

[0136] In one embodiment, when the processor executes the computer program, the following steps are also implemented: calling the admission control plug-in to intercept the pod, and checking whether the pod is configured with image information through the admission control plug-in; in response to the pod being configured with image information, extracting the image information as the target container image.

[0137] In one embodiment, when the processor executes the computer program, the following steps are also implemented: creating the image repository according to the matching relationship between the deep learning framework version information, the CUDA version information, and the GPU driver version information and the container image; and obtaining a target container image that matches the configuration information of the pod from the preset image repository according to one or more of the deep learning framework version information, the CUDA version information, and the GPU driver version information.

[0138] In one embodiment, when the processor executes the computer program, the following steps are also implemented: calling the admission control plug-in to intercept the pod, and checking whether the pod is configured with image information through the admission control plug-in; if not, the pod creation fails.

[0139] In one embodiment, the processor further implements the following steps when executing the computer program: constructing a dependency mapping table based on the dependency relationship between the deep learning framework version and the CUDA version, and the dependency relationship between the CUDA version and the GPU driver version, and storing the dependency mapping table in the access control plug-in; in response to the presence of CUDA version information and / or GPU driver version information in the configuration information of the pod and the image information of the target container image, determining whether there is a dependency conflict between the image information and the configuration information according to the dependency mapping table; if so, the pod creation fails.

[0140] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the GPU driver version information of each node, and reporting the GPU driver version information to the admission control plug-in to form a node information table; in response to the configuration information of the pod including the GPU driver version information, matching the GPU driver version information with the node information table to obtain a matching result; judging whether there is a node matching the pod according to the matching result; if not, the pod creation fails; if so, modifying the node affinity for the pod according to the matching result; in response to the pod not being configured with the GPU driver version information, obtaining a set of GPU driver version information matching the target image container according to the dependency mapping table; matching the GPU driver version information set with the node information table to obtain a matching result; judging whether there is a node matching the pod according to the matching result; if not, the pod creation fails; if so, modifying the node affinity for the pod according to the matching result.

[0141] In one embodiment, when the processor executes the computer program, the following steps are also implemented: scheduling the pod to a matching node according to the node affinity; pulling the target container image and starting the pod to execute the AI ​​application.

[0142] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0143] Receive pod creation requests from AI applications;

[0144] In response to the pod to be created not being configured with image information, obtaining configuration information of the pod to be created;

[0145] Obtain a target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information;

[0146] Check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to the preset dependency mapping table;

[0147] If not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information;

[0148] The pod is created according to the configuration information.

[0149] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: calling the admission control plug-in to intercept the pod, and checking whether the pod is configured with image information through the admission control plug-in; in response to the pod being configured with image information, extracting the image information as the target container image.

[0150] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: creating the image repository based on the matching relationship between the deep learning framework version information, CUDA version information, and GPU driver version information and the container image; and obtaining a target container image that matches the configuration information of the pod from the preset image repository based on one or more of the deep learning framework version information, CUDA version information, and GPU driver version information.

[0151] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: calling the admission control plug-in to intercept the pod, and checking whether the pod is configured with image information through the admission control plug-in; if not, the pod creation fails.

[0152] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: constructing a dependency mapping table based on the dependency relationship between the deep learning framework version and the CUDA version, and the dependency relationship between the CUDA version and the GPU driver version, and storing the dependency mapping table in the access control plug-in; in response to the presence of CUDA version information and / or GPU driver version information in the configuration information of the pod and the image information of the target container image, determining whether there is a dependency conflict between the image information and the configuration information according to the dependency mapping table; if so, the pod creation fails.

[0153] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the GPU driver version information of each node, and reporting the GPU driver version information to the admission control plug-in to form a node information table; in response to the configuration information of the pod including the GPU driver version information, matching the GPU driver version information with the node information table to obtain a matching result; judging whether there is a node matching the pod according to the matching result; if not, the pod creation fails; if so, modifying the node affinity for the pod according to the matching result; in response to the pod not being configured with the GPU driver version information, obtaining a set of GPU driver version information matching the target image container according to the dependency mapping table; matching the GPU driver version information set with the node information table to obtain a matching result; judging whether there is a node matching the pod according to the matching result; if not, the pod creation fails; if so, modifying the node affinity for the pod according to the matching result.

[0154] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: scheduling the pod to a matching node according to the node affinity; pulling the target container image and starting the pod to execute the AI ​​application.

[0155] In one embodiment, a computer product is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0156] Receive pod creation requests from AI applications;

[0157] In response to the pod to be created not being configured with image information, obtaining configuration information of the pod to be created;

[0158] Obtain a target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information;

[0159] Check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to the preset dependency mapping table;

[0160] If not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information;

[0161] The pod is created according to the configuration information.

[0162] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: calling the admission control plug-in to intercept the pod, and checking whether the pod is configured with image information through the admission control plug-in; in response to the pod being configured with image information, extracting the image information as the target container image.

[0163] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: creating the image repository based on the matching relationship between the deep learning framework version information, CUDA version information, and GPU driver version information and the container image; and obtaining a target container image that matches the configuration information of the pod from the preset image repository based on one or more of the deep learning framework version information, CUDA version information, and GPU driver version information.

[0164] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: calling the admission control plug-in to intercept the pod, and checking whether the pod is configured with image information through the admission control plug-in; if not, the pod creation fails.

[0165] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: constructing a dependency mapping table based on the dependency relationship between the deep learning framework version and the CUDA version, and the dependency relationship between the CUDA version and the GPU driver version, and storing the dependency mapping table in the access control plug-in; in response to the presence of CUDA version information and / or GPU driver version information in the configuration information of the pod and the image information of the target container image, determining whether there is a dependency conflict between the image information and the configuration information according to the dependency mapping table; if so, the pod creation fails.

[0166] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the GPU driver version information of each node, and reporting the GPU driver version information to the admission control plug-in to form a node information table; in response to the configuration information of the pod including the GPU driver version information, matching the GPU driver version information with the node information table to obtain a matching result; judging whether there is a node matching the pod according to the matching result; if not, the pod creation fails; if so, modifying the node affinity for the pod according to the matching result; in response to the pod not being configured with the GPU driver version information, obtaining a set of GPU driver version information matching the target image container according to the dependency mapping table; matching the GPU driver version information set with the node information table to obtain a matching result; judging whether there is a node matching the pod according to the matching result; if not, the pod creation fails; if so, modifying the node affinity for the pod according to the matching result.

[0167] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: scheduling the pod to a matching node according to the node affinity; pulling the target container image and starting the pod to execute the AI ​​application.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0169] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0170] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A pod creation method, characterized in that: include: Receive pod creation requests from AI applications; In response to the pod to be created not being configured with image information, obtaining configuration information of the pod to be created; Obtain a target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information; Check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to the preset dependency mapping table; If not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information; The pod is created according to the configuration information.

2. The pod creation method according to claim 1, characterized in that: The configuration information of the pod to be created includes at least one of image information and deep learning framework version information, and the method further includes: Call the admission control plug-in to intercept the pod to be created, and check whether the pod to be created is configured with image information; In response to the pod configuration image information to be created, the image information is extracted as the target container image.

3. The pod creation method according to claim 2, characterized in that: The configuration information of the pod to be created also includes CUDA version information and / or GPU driver version information; and obtaining the target container image from a preset image repository according to the configuration information includes: Create the image repository according to the matching relationship between the deep learning framework version information, CUDA version information, GPU driver version information and the container image; According to one or more of the deep learning framework version information, the CUDA version information, and the GPU driver version information, a target container image that matches the configuration information of the pod to be created is obtained from the image repository.

4. The pod creation method according to claim 2, characterized in that: After obtaining the target container image from the preset image repository according to the configuration information and modifying the image information of the target container image into the configuration information, the method further includes: Call the admission control plug-in to intercept the pod to be created, and check whether the pod to be created is configured with image information; If not, the creation fails.

5. The pod creation method according to claim 3, characterized in that: The checking, according to the preset dependency mapping table, whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created includes: Building a dependency mapping table based on the dependency relationship between the deep learning framework version and the CUDA version, and the dependency relationship between the CUDA version and the GPU driver version, and storing the dependency mapping table in the admission control plug-in; In response to the presence of CUDA version information and / or GPU driver version information in the configuration information of the pod to be created and the image information of the target container image, checking whether there is a dependency conflict between the image information and the configuration information according to the dependency mapping table; If yes, the creation fails.

6. The pod creation method according to claim 2, characterized in that: The acquiring, according to the configuration information, a node matching the pod to be created, configuring node affinity information for the pod to be created and adding the node affinity information to the configuration information includes: Obtaining GPU driver version information of each node, and reporting the GPU driver version information to the admission control plug-in to form a node information table; In response to the configuration information of the pod to be created including GPU driver version information, matching the GPU driver version information with the node information table to obtain a matching result; Determine, according to the matching result, whether there is a node matching the pod to be created; If not, the creation fails; If yes, configure node affinity information for the pod to be created according to the matching result; In response to the pod to be created not being configured with GPU driver version information, acquiring a set of GPU driver version information matching the configuration information of the pod to be created according to the dependency mapping table; Matching the GPU driver version information set with the node information table to obtain a matching result; Determine, according to the matching result, whether there is a node matching the pod to be created; If not, the creation fails; If yes, node affinity information is configured for the pod to be created according to the matching result.

7. The pod creation method according to claim 1, characterized in that: The creating the pod according to the configuration information includes: Scheduling the pod to be created to a matching node according to the node affinity; Pull the target container image, create and start the pod according to the configuration information, and execute the AI ​​application.

8. A pod creation device, characterized in that: The device comprises: The receiving module is used to receive the pod creation request of the AI ​​application; An acquisition module, configured to acquire configuration information of the pod to be created in response to the pod to be created not being configured with image information; A first matching module, configured to obtain a matching target container image from a preset image repository according to the configuration information, and modify the image information of the target container image into the configuration information; A checking module is used to check whether there is a dependency conflict between the image information of the target container image and the configuration information of the pod to be created according to a preset dependency mapping table; A second matching module is used for, if not, obtaining a node matching the pod to be created according to the configuration information, configuring node affinity information for the pod to be created and adding it to the configuration information; A creation module is used to create the pod according to the configuration information.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Method and system for automatically configuring CUDA environment and testing GPU pressure

    CN119922183A