Graphic processor allocation method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides an allocation method and device of a graphics processing unit, a storage medium and electronic equipment, and relates to the field of computers.The allocation method of the graphics processing unit comprises the steps that through a resource management component, based on a first annotation of a container group in a database component, a GPU scheduling program plug-in is called to allocate a computing node and a GPU for the container group, obtaining a second annotation of the target computing node and the container group; through a node manager in the target computing node, under the condition that it is monitored that the container group is allocated to the target computing node, a computing container is created for the container group when the container in the target computing node is called to run based on the equipment file path; and adding or deleting the GPU for the computing container according to the second annotation of the container group through the equipment plug-in in the target computing node.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a method and device for allocating a graphics processor, a storage medium, and an electronic device. Background Art

[0002] The popularization of the Internet and the acceleration of the digitalization process have generated a huge amount of data, providing rich materials for the training of artificial intelligence. K8s is a powerful container orchestration tool. It uses container groups (Pods) as logical concepts, packages containers, and then manages containers in a unified resource view. It has excellent resource management and scheduling capabilities and has become the preferred technology stack for artificial intelligence development platforms. In the process of artificial intelligence development, there are often scenarios where accelerator cards need to be added or removed from containers to cope with the dynamic demand for computing resources. However, containers do not support hot update of graphics processor (Graphics Processing Unit, referred to as GPU) devices, that is, GPU devices cannot be dynamically added or removed during container operation. GPU devices can only be mounted to containers and access rights can be added when creating containers. Once the container is created, the GPU device in the container cannot be updated. If the user needs to add or reduce GPU devices to the container, he can only package the container into an image first, then use the packaged image to create a new container, and reallocate the required GPU device when creating it. In the field of artificial intelligence development, images are usually tens of GB, and the amount of data is very large, which makes the process of packaging containers into images and then recreating containers very time-consuming and wasteful of resources.

[0003] In the related technology, the container does not support hot update of GPU devices, resulting in the inability to dynamically add or remove GPU devices during container operation. No effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method and device for allocating a graphics processor, a storage medium, and an electronic device to at least solve the problem that a container group does not support hot update of GPU devices, resulting in an inability to dynamically add or remove GPU devices during the operation of the container group.

[0005] According to one embodiment of the present application, a method for allocating a graphics processor is provided, which is applied to a container orchestration platform, wherein the container orchestration platform includes a management node and multiple computing nodes, wherein the management node has a database component and a resource management component, wherein the resource management component has a graphics processor GPU scheduler plug-in for realizing dynamic allocation of GPUs; wherein the computing node has a node manager, a container runtime, and multiple GPUs; wherein the node manager has a device plug-in for realizing the management function of the GPU, including: through the resource management component, based on the first annotation of the container group in the database component, calling the GPU scheduler plug-in to allocate computing nodes and GPUs to the container group, obtaining a second annotation of the target computing node and the container group, wherein the first annotation is used to indicate the number of GPUs requested by the container group, and the second annotation is used to indicate the number of GPUs requested by the container group. The second annotation has a universal unique identification code of N allocated GPUs, the N GPUs are GPUs in the target computing node, and the multiple computing nodes include the target computing node; when the first annotation of the container group changes, the second annotation of the container group changes; through the node manager in the target computing node, when monitoring that the container group is allocated to the target computing node, based on the device file path, calling the container runtime in the target computing node to create a computing container for the container group, wherein the device file corresponding to the device file path is the file required for adding all GPUs on the target computing node to the computing container, and the container group includes the computing container; through the device plug-in in the target computing node, according to the second annotation of the container group, add or delete GPUs for the computing container.

[0006] In an exemplary embodiment, before calling the container runtime of the target computing node based on the device file path to create a computing container for the container group, the method also includes: through the node manager in the target computing node, when monitoring that the container group is assigned to the target computing node, calling the device plug-in in the target computing node to obtain the device file path.

[0007] In an exemplary embodiment, the computing node also has a GPU driver; before the resource management component is used to call the GPU scheduler plug-in based on the first annotation of the container group to allocate computing nodes and GPUs to the container group, and obtains the second annotation of the target computing node and the container group, the method also includes: calling the GPU driver in the i-th computing node through the device plug-in in the i-th computing node among the multiple computing nodes to obtain a GPU list of the i-th computing node, wherein the GPU list has device information of all GPUs in the i-th computing node, and the device information includes: an identifier of the GPU, a universal unique identifier of the GPU, and a status of the GPU; and adding the GPU list of the i-th computing node to the third annotation of the i-th computing node through the device plug-in in the i-th computing node, wherein the third annotation is used to indicate the GPU information in the computing node.

[0008] In an exemplary embodiment, the resource management component is used to call the GPU scheduler plug-in based on the first annotation of the container group in the database component to allocate computing nodes and GPUs to the container group, and obtain the target computing node and the second annotation of the container group, including: the resource management component is used to call the GPU scheduler plug-in based on the first annotation of the container group in the database component to determine the target computing node when the creation event of the container group in the database component is monitored, wherein the number of idle GPUs in the target computing node is greater than the number of GPUs requested in the first annotation, and the number of idle GPUs in the target computing node is determined according to the third annotation of the target computing node, and the third annotation is used to indicate the GPU information in the computing node, and the GPU scheduler plug-in obtains the third annotation of each computing node in the multiple computing nodes during the initialization phase; the resource management component is used to call the GPU scheduler plug-in to select M GPUs from the idle GPU list of the target computing node, and add the universal unique identification codes of the M GPUs to the second annotation of the container group, wherein M is equal to the number of GPUs indicated by the first annotation; the resource management component is used to call the GPU scheduler plug-in to select M GPUs from the idle GPU list of the target computing node, and add the universal unique identification codes of the M GPUs to the second annotation of the container group, wherein M is equal to the number of GPUs indicated by the first annotation; component, calling the GPU scheduler plug-in to delete the M GPUs from the list of idle GPUs of the target computing node; and / or through the resource management component, in the case of monitoring that the first annotation of the container group in the database component changes, determining that the computing node allocated by the container group is the target computing node; when P is greater than Z and there are PZ idle GPUs in the target computing node, calling the GPU scheduler plug-in through the resource management component, selecting PZ GPUs from the list of idle GPUs of the target computing node, and adding the universal unique identification codes of the PZ GPUs to the target computing node. Add to the second annotation of the container group, wherein Z is equal to the number of GPUs recorded in the second annotation of the container group, and P is equal to the number of GPUs indicated after the first annotation changes; call the GPU scheduler plug-in through the resource management component to delete the PZ GPUs from the list of idle GPUs of the target computing node; when P is less than Z, call the GPU scheduler plug-in through the resource management component to delete the universal unique identification codes of the ZP GPUs from the second annotation of the container group, and add the ZP GPUs to the list of idle GPUs of the target computing node.

[0009] In an exemplary embodiment, the method further includes: when P is greater than Z and there are no PZ idle GPUs on the target computing node, calling the GPU scheduler plug-in through the resource management component, selecting X GPUs from the idle GPU list of the target computing node, and adding the universal unique identification codes of the X GPUs to the second annotation of the container group, wherein X is the number of current idle GPUs of the target computing node, and X is an integer greater than or equal to 0 and less than PZ; calling the GPU scheduler plug-in through the resource management component to delete the X GPUs from the idle GPU list of the target computing node; and sending a prompt message through the resource management component, wherein the prompt message is used to indicate that the container group still needs to add a GPU.

[0010] In an exemplary embodiment, adding or deleting a GPU for the computing container according to the second annotation of the container group through the device plug-in in the target computing node includes: entering the namespace of the computing container through the device plug-in in the target computing node, upon listening to the creation event of the computing container, and deleting the GPU in the computing container that is not recorded in the second annotation according to the universal unique identifier of the GPU indicated in the second annotation; entering the namespace of the computing container through the device plug-in in the target computing node, upon listening to the change of the second annotation of the container group, and adding or deleting a GPU for the computing container according to the universal unique identifier of the GPU indicated in the second annotation, so that the GPU currently allowed to be used by the computing container is the GPU indicated in the second annotation.

[0011] In an exemplary embodiment, after adding the GPU list of the i-th computing node to the third annotation of the i-th computing node through the device plug-in in the i-th computing node, the method also includes: registering virtual device resources to the node manager of the i-th computing node through the device plug-in in the i-th computing node; recording in the database component through the node manager of the i-th computing node that the i-th computing node has the virtual device resources, wherein the virtual device resources are used to indicate that the computing node has the ability to dynamically change the number of GPUs used by the container; calling the GPU scheduler plug-in through the resource management component based on the first annotation of the container group in the database component to allocate computing nodes and GPUs to the container group, including: calling the GPU scheduler plug-in through the resource management component based on the first annotation of the container group in the database component, allocating computing nodes to the container group from the computing nodes with the virtual device resources among the multiple computing nodes, and correspondingly allocating GPUs.

[0012] According to another embodiment of the embodiment of the present application, a graphics processor allocation device is also provided, which is applied to a container orchestration platform, wherein the container orchestration platform includes a management node and multiple computing nodes, wherein the management node has a database component and a resource management component, wherein the resource management component has a graphics processor GPU scheduler plug-in for realizing dynamic allocation of GPUs; the computing node has a node manager, a container runtime and multiple GPUs; the node manager has a device plug-in for realizing the management function of the GPU, including: an allocation module, which is used to call the GPU scheduler plug-in to allocate computing nodes and GPUs to the container group based on the first annotation of the container group in the database component through the resource management component, and obtain a second annotation of the target computing node and the container group, wherein the first annotation is used to indicate the number of GPUs requested by the container group, and the second annotation is used to indicate the number of GPUs requested by the container group, and the second annotation is used to indicate the number of GPUs requested by the container group. The solution has a universal unique identification code of N allocated GPUs, the N GPUs are GPUs in the target computing node, and the multiple computing nodes include the target computing node; when the first annotation of the container group changes, the second annotation of the container group changes; a creation module is used to call the container runtime in the target computing node based on the device file path to create a computing container for the container group through the node manager in the target computing node, when monitoring that the container group is allocated to the target computing node, wherein the device file corresponding to the device file path is the file required for adding all GPUs on the target computing node to the computing container, and the container group includes the computing container; a first processing module is used to add or delete a GPU for the computing container according to the second annotation of the container group through the device plug-in in the target computing node.

[0013] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0014] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0015] According to another embodiment of the present application, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0016] Through the present application, a graphics processor GPU scheduler plug-in is added to the resource management component, and a device plug-in is added to the node manager. Since the resource management component can call the GPU scheduler plug-in to determine the second annotation of the container group according to the first annotation of the container group, the node manager of the computing node dynamically adds or deletes GPUs for the computing containers of the container group according to the second annotation by calling the device plug-in, and there is no need to recreate the container to reallocate the GPU device, thereby solving the problem that the container does not support hot update of GPU devices, resulting in the inability to dynamically add or remove GPU devices during container operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a hardware structure block diagram of a server device of a method for allocating a graphics processor according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of a method for allocating a graphics processor according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of the overall architecture of an optional method for allocating a graphics processor according to an embodiment of the present application;

[0021] Figure 4 is an optional timing diagram reported by a device according to an embodiment of the present application;

[0022] Figure 5 is a timing diagram of an optional container creation according to an embodiment of the present application;

[0023] Figure 6 is an optional timing diagram of a container updating a GPU according to an embodiment of the present application;

[0024] Figure 7 is a schematic diagram of a container graphics card pass-through process according to an embodiment of the present application;

[0025] Figure 8 is a structural block diagram of a distribution device of a graphics processor according to an embodiment of the present application;

[0026] Fig. 9 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0029] The graphics processor allocation method embodiment provided in the embodiment of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a method for allocating a graphics processor according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the allocation method of the graphics processor in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0032] In order to solve the above problems, a method for allocating a graphics processor is provided in this embodiment, which is applied to a container orchestration platform. The container orchestration platform includes a management node and multiple computing nodes. The management node has a database component and a resource management component. The resource management component has a graphics processor GPU scheduler plug-in for realizing dynamic allocation of GPUs; the computing node has a node manager, a container runtime and multiple GPUs; the node manager has a device plug-in for realizing GPU management functions, such as Figure 2 As shown, the process includes the following steps S202-S206:

[0033] Step S202: through the resource management component, based on the first annotation of the container group in the database component, calling the GPU scheduler plug-in to allocate a computing node and a GPU to the container group, and obtaining a second annotation of the target computing node and the container group, wherein the first annotation is used to indicate the number of GPUs requested by the container group, and the second annotation has universal unique identification codes of N allocated GPUs, the N GPUs are GPUs in the target computing node, and the multiple computing nodes include the target computing node; when the first annotation of the container group changes, the second annotation of the container group changes;

[0034] Alternatively, if Figure 3 As shown, the container orchestration platform (such as K8s) has management nodes and computing nodes, and the management nodes and computing nodes can be physical servers, virtual machines, etc.;

[0035] The management node includes the following parts: the database component (etcd) records all resources in K8s; the access proxy server (kube-apiserver) in the management node is the access proxy of the application programming interface (API). All requests to interact with etcd must be initiated to the kube-apiserver. After format verification, authentication, filtering and other operations, the kube-apiserver forwards the request to etcd; the resource management component (kube-Scheduler) is the scheduler in K8s, which performs resource scheduling in a unified resource view. The resource management component has a graphics processor GPU scheduler plug-in (gpuscheduler), which is responsible for the dynamic allocation of GPUs;

[0036] The computing node includes the following parts (taking computing node 1 as an example): the node manager (kubelet) is an agent component on the computing node. Its main function is to monitor the node status and maintain the container life cycle. Kubelet has a device plug-in (deviceplugin), which provides the necessary interface for kubelet to call; the container runtime (docker) provides container management functions; as well as a GPU driver and several GPUs.

[0037] Optionally, when the first annotation of the container group changes, the second annotation of the container group changes, and the target computing node remains unchanged.

[0038] Optionally, the annotation information and meanings of the first annotation, the second annotation, and the following third annotation used in the embodiment of the present application are as shown in Table 1:

[0039] Table 1

[0040]

[0041] Step S204: When monitoring that the container group is assigned to the target computing node, the node manager in the target computing node calls the container runtime in the target computing node based on the device file path to create a computing container for the container group, wherein the device file corresponding to the device file path is a file required for adding all GPUs on the target computing node to the computing container, and the container group includes the computing container;

[0042] It should be noted that the node manager has already obtained the device file path before invoking the container runtime in the target computing node to create a computing container for the container group based on the device file path.

[0043] It should be noted that when the node manager calls the container runtime in the target computing node based on the device file path to create a computing container for the container group, all GPUs have been added to the computing container.

[0044] In an exemplary embodiment, before the container runtime of the target computing node is called based on the device file path to create a computing container for the container group, the method further includes: through the node manager in the target computing node, when it is monitored that the container group is assigned to the target computing node, calling the device plug-in in the target computing node to obtain the device file path.

[0045] Optionally, the container group (specifically, the computing container in the container group) may apply for a fake device resource;

[0046] Optionally, when the node manager in the target computing node monitors that the container group is assigned to the target computing node, the device plug-in in the target computing node is called to obtain the device file path, including: when the node manager in the target computing node monitors that the container group is assigned to the target computing node and the container group applies for virtual device resources, the device plug-in in the target computing node is called to obtain the device file path.

[0047] It should be noted that the node manager obtains the device file path by calling the device plug-in, which enhances the accuracy and timeliness of device allocation and avoids inaccurate allocation caused by changes in device status. The device plug-in can monitor the device status in real time to ensure that the device file path obtained when the computing container is created is valid and available. By accurately obtaining the device file path before the computing container is created, the allocation and use of resources can be optimized to avoid waste of resources. At the same time, by avoiding unnecessary resource reallocation and container reconstruction, the overall efficiency of the system is improved, especially in large-scale artificial intelligence development and training scenarios. This effect is particularly obvious.

[0048] Optionally, the creation process of the computing container is as follows Figure 5 As shown, the created computing container meets the following three characteristics:

[0049] 1. The computing container disables the permission to create file entry points (mknod permission);

[0050] It should be noted that if the computing container has the mknod permission, the computing container can actively search and guess all available GPUs, which may cause confusion in GPU management and even lead to system instability or security vulnerabilities. In K8s, device allocation and management are completed through components such as kubelet and deviceplugin. If the computing container can add GPUs by itself, it bypasses the resource management and scheduling mechanism of K8s, resulting in inconsistent resource status and chaotic management. For example, the container may create redundant GPUs, and these nodes will not be recognized by K8s, making it impossible to uniformly allocate and recycle resources. Therefore, disabling the mknod permission ensures that operations within the container will not interfere with the necessary measures of the host's GPU management and K8s resource management mechanism, so that the addition and deletion of GPU devices can be centrally controlled, which not only ensures security, but also maintains the uniformity and effectiveness of resource management.

[0051] 2. The computing container will apply for a virtual device (fakedevice) resource;

[0052] It should be noted that only when the computing container requests fakedevice resources, the node manager in the target computing node will call the deviceplugin plug-in allocation (Allocate) interface. The call of the Allocate interface allows the deviceplugin plug-in to perform specific operations. If the computing container applies for virtual device (fakedevice) resources, it means that the computing container needs to dynamically change the number of GPUs it uses.

[0053] 3. With annotation request / gpus.

[0054] Step S206: adding or deleting a GPU for the computing container according to the second annotation of the container group through the device plug-in in the target computing node.

[0055] Optionally, the device plugin adds or removes a GPU for the compute container in etcd according to the second annotation of the container group.

[0056] In the above steps S202-S206, a graphics processor GPU scheduler plug-in is added to the resource management component, and a device plug-in is added to the node manager. Since the resource management component can call the GPU scheduler plug-in to determine the second annotation of the container group according to the first annotation of the container group, the node manager of the computing node dynamically adds or deletes GPUs for the computing containers of the container group according to the second annotation by calling the device plug-in, and there is no need to recreate the container to reallocate the GPU device, thereby solving the problem that the container does not support hot update of GPU devices, resulting in the inability to dynamically add or remove GPU devices during container operation.

[0057] In an exemplary embodiment, the computing node further includes a GPU driver; before the method allocates computing nodes and GPUs to the container group through the resource management component based on the first annotation of the container group and obtains the second annotation of the target computing node and the container group, the method further includes the following steps S11-S12:

[0058] Step S11: acquiring a GPU list of the i-th computing node by calling a GPU driver in the i-th computing node through a device plug-in in the i-th computing node among the multiple computing nodes, wherein the GPU list has device information of all GPUs of the i-th computing node, and the device information includes: an identifier of the GPU, a universal unique identification code of the GPU, and a status of the GPU;

[0059] Step S12: adding the GPU list of the i-th computing node to the third annotation of the i-th computing node through the device plug-in in the i-th computing node, wherein the third annotation is used to indicate the GPU information in the computing node.

[0060] In an exemplary embodiment, after adding the GPU list of the i-th computing node to the third annotation of the i-th computing node through the device plug-in in the i-th computing node, the method further includes the following steps: registering virtual device resources with the node manager of the i-th computing node through the device plug-in in the i-th computing node; recording in the database component through the node manager of the i-th computing node that the i-th computing node has the virtual device resources, wherein the virtual device resources are used to indicate that the computing node has the ability to dynamically change the number of GPUs used by the container;

[0061] In an exemplary embodiment, the resource management component is used to call the GPU scheduler plug-in based on the first annotation of the container group in the database component to allocate computing nodes and GPUs to the container group, including: calling the GPU scheduler plug-in based on the first annotation of the container group in the database component through the resource management component, allocating computing nodes to the container group from computing nodes with the virtual device resources in the multiple computing nodes, and correspondingly allocating GPUs.

[0062] That is to say, the computing node with fakedevice resources has the ability to dynamically change the number of GPUs used by the container, so that the resource management component can allocate computing nodes to the container group from the computing nodes with the virtual device resources in the computing nodes, and allocate GPUs accordingly, to ensure that GPU resources are not wasted.

[0063] Optionally, if a node manager of a computing node has a device plug-in, the computing node has a virtual device resource.

[0064] Alternatively, if Figure 4 As shown in the figure, deviceplugin calls the GPU driver to obtain all GPU devices, and then adds the GPU device list and device health status to the node's gpus annotation. At the same time, deviceplugin registers fakedevice resources with kubelet, and the number of resources is set to the maximum number of Pods that can be accommodated by kubelet configuration. Kubelet will report fakedevice resources to kube-apiserver. Finally, deviceplugin calls the GPU driver every 10 seconds to obtain the latest status of the device. If the device status changes, the device status is updated to the node annotation.

[0065] It should be noted that through the above steps, comprehensive monitoring and information reporting of GPU devices on the computing nodes are achieved, providing accurate basic data for subsequent dynamic allocation of GPUs. Deviceplugin can periodically call the GPU driver to obtain the real-time status of the GPU, including the GPU's identification, uuid, and health status. This ensures the real-time and accuracy of GPU information and provides a reliable data source for dynamic allocation of GPUs. By updating GPU information in real time at the node level, the system can quickly respond to changes in GPU status, such as device failures or new devices, ensuring that the latest GPU resource status is taken into account when allocating resources, enhancing the flexibility and reliability of the system.

[0066] In an exemplary embodiment, the step of calling the GPU scheduler plug-in to allocate a computing node and a GPU to the container group based on the first annotation of the container group in the database component through the resource management component to obtain the second annotation of the target computing node and the container group includes the following steps S21 and / or S22:

[0067] Step S21: through the resource management component, in the case of monitoring the creation event of the container group in the database component, the GPU scheduler plug-in is called based on the first annotation of the container group in the database component to determine the target computing node, wherein the number of idle GPUs in the target computing node is greater than the number of GPUs requested in the first annotation, and the number of idle GPUs in the target computing node is determined according to the third annotation of the target computing node, the third annotation is used to indicate the GPU information in the computing node, and the GPU scheduler plug-in obtains the third annotation of each computing node in the multiple computing nodes during the initialization phase; through the resource management component, the GPU scheduler plug-in is called to select M GPUs from the idle GPU list of the target computing node, and the universal unique identification codes of the M GPUs are added to the second annotation of the container group, wherein M is equal to the number of GPUs indicated by the first annotation; through the resource management component, the GPU scheduler plug-in is called to delete the M GPUs from the idle GPU list of the target computing node;

[0068] Alternatively, if Figure 5 As shown, after kube-scheduler listens to the pod creation event, it will schedule it. kube-scheduler is a plug-in architecture, and the main structure can be simply described as three processes: pre-selecting nodes, selecting the best nodes, and binding nodes. Each plug-in can register hook functions to these processes, and these hook functions will be called when executing a specific process, thereby affecting the processing results of each process or making some marks. The node pre-selection process is to select nodes that meet the pod requirements based on some requirements of the pod for the nodes. The optimal node selection process is to select the most suitable node from the nodes that meet the pod requirements based on certain strategies. The node binding process is to bind the Pod to the most suitable node. In addition to enabling the default plug-in, kube-scheduler also starts the gpuscheduler plug-in;

[0069] The gpuscheduler plug-in registers hook functions in the pre-selection process and the binding process to achieve the following functions: 1. When the gpuscheduler plug-in is initialized, it reads the gpus annotation on each node, obtains all GPU devices on the node, initializes them into the idle device list of each node, and then caches them. Whenever a pod is bound to this node, the GPU device recorded in the bound_gpus annotation of the pod is deleted from the idle device list; when the pod is deleted from the node, the GPU device recorded in the bound_gpus annotation of the pod is added to the idle device list of this node. 2. In the pre-selection process, the number of healthy GPU devices in the idle device list is compared with the number of GPU devices requested by the request / gpus annotation of the pod. If the number of idle GPU devices is less than the number of GPU devices requested by the pod, the scheduling to this node is rejected. 3. In the process of binding nodes, the corresponding number of GPUs are randomly allocated from the idle GPU device list and added to the bound_gpus annotation of the pod. Then bind to the corresponding node.

[0070] After the scheduling is completed, the kubelet of the corresponding node will monitor that the pod is bound to this node. Because the pod applies for fakedevice resources, it will trigger kubelet to call the Allocate interface of deviceplugin, so that the GPU driver files, deep learning library files, and device files of all GPU devices on the host can be mounted. When calling docker to create a pod, kubelet will mount these files into the pod. After the container is successfully started, kubelet will update the Pod status to Running.

[0071] Step S22: through the resource management component, when it is monitored that the first annotation of the container group in the database component changes, determine that the computing node allocated by the container group is the target computing node; when P is greater than Z and there are PZ idle GPUs in the target computing node, through the resource management component, call the GPU scheduler plug-in, select PZ GPUs from the idle GPU list of the target computing node, and add the universal unique identification codes of the PZ GPUs to the second annotation of the container group, wherein Z is equal to the number of GPUs recorded in the second annotation of the container group, and P is equal to the number of GPUs indicated after the first annotation changes; through the resource management component, call the GPU scheduler plug-in to delete the PZ GPUs from the idle GPU list of the target computing node; when P is less than Z, call the GPU scheduler plug-in through the resource management component to delete the universal unique identification codes of ZP GPUs from the second annotation of the container group, and add the ZP GPUs to the idle GPU list of the target computing node.

[0072] It should be noted that the above step S21 is executed when a new GPU is added, and step S22 is executed when the GPU is updated. There is no specific order in which step S21 and step S22 are executed.

[0073] Alternatively, if Figure 6 As shown in the figure, after modifying the request / gpus annotation value of the computing container to the actual required value, the gpuscheduler will monitor the change of the request / gpus annotation value of the Pod, and compare the request / gpus annotation value (the number of requested devices) with the number of devices in the bound_gpus annotation value (the number of allocated devices), and then reallocate the GPU.

[0074] Optionally, if the number of requested devices is equal to the number of allocated devices, no processing is performed; if the number of requested devices is less than the number of allocated devices, the devices in the bound_gpus annotation value are randomly deleted to make the number of allocated devices consistent with the number of requested devices; if the number of requested devices is greater than the number of allocated devices, devices are randomly allocated to the Pod from the node's free device list and appended to the bound_gpus annotation value.

[0075] It should be noted that the above steps realize the optimization of dynamic allocation of GPU resources of pod. Step S2 1 When the pod is created, according to the number of GPUs requested by the pod, GPU resources are efficiently selected and allocated from the list of free GPUs of the target computing node, avoiding the problem of resource waste and uneven allocation, ensuring that each pod can obtain the required GPU resources, thereby improving resource utilization and overall system performance. Step S22 When the first annotation of the pod changes, the GPU resource allocation can be adjusted in real time, whether it is to increase or decrease the number of GPUs, it can respond quickly and make corresponding resource adjustments. This flexibility enables the platform to adapt to changing computing needs, especially in scenarios such as deep learning and AI development where the demand for GPU resources fluctuates greatly, which can significantly reduce the time and cost required for resource adjustment. The GPU scheduler plug-in obtains the third annotation of each computing node in the initialization phase, monitors and maintains the list of free GPUs on the computing node in real time, and ensures the accuracy of resource status. After resource allocation or recovery, the second annotation of the pod in the database component and the list of free GPUs of the target computing node can be updated in time. This real-time performance is crucial for large-scale cluster management and scheduling. By dynamically updating GPU resources, the tedious process of recreating containers or exporting images from containers in traditional methods is avoided, greatly shortening the resource adjustment time.

[0076] In an exemplary embodiment, the method further includes the following steps S31 to S33:

[0077] Step S31: when P is greater than Z and there are no PZ idle GPUs in the target computing node, the GPU scheduler plug-in is called through the resource management component to select X GPUs from the idle GPU list of the target computing node, and the universal unique identification codes of the X GPUs are added to the second annotation of the container group, where X is the number of current idle GPUs of the target computing node, and X is an integer greater than or equal to 0 and less than PZ;

[0078] Step S32: calling the GPU scheduler plug-in through the resource management component to delete the X GPUs from the idle GPU list of the target computing node;

[0079] Step S33: Send a prompt message through the resource management component, wherein the prompt message is used to indicate that the container group still needs to add a GPU.

[0080] Alternatively, if Figure 6 As shown in the figure, when the number of requested devices is greater than the number of allocated devices, but it is not enough to allocate all idle devices to the Pod, the gpuscheduler will issue an event to indicate that the Pod needs more GPUs.

[0081] It should be noted that, through the above steps, when the number of GPUs P requested by the container group is greater than the number of GPUs currently available on the target computing node Z, the GPU scheduler plug-in can intelligently select as many GPUs as possible from the node's free GPU list (X, where X is the maximum number of available GPUs on the current node, and X satisfies 0≤X<PZ), and add the universal unique identifiers (uuids) of these GPUs to the second annotation of the container group. This strategy ensures that when resources are limited, the container group can obtain as many GPU resources as possible, thereby meeting its computing needs to a certain extent and avoiding idleness and waste of resources. In step S32, the GPU scheduler plug-in is called through the resource management component to delete the X GPUs allocated to the container group from the free GPU list of the target computing node, and synchronize the resource status of the node. This operation can ensure that the resource view in the K8s system is consistent with the actual physical resource status, avoid resource conflicts during subsequent container scheduling, and improve the accuracy and efficiency of resource scheduling.

[0082] In an exemplary embodiment, adding or deleting a GPU for the computing container according to the second annotation of the container group through the device plug-in in the target computing node includes the following steps S41 to S42:

[0083] Step S41: when a creation event of the computing container is monitored, the device plug-in in the target computing node enters the namespace of the computing container and deletes the GPU in the computing container that is not recorded in the second annotation according to the universal unique identifier of the GPU indicated in the second annotation;

[0084] It should be noted that deviceplugin will continue to monitor container startup events in docker, because the device files of all GPU devices on the host are mounted when Allocate is called, so when the pod starts successfully, all GPU devices can be seen and have device access rights. When monitoring the startup event of the pod, deviceplugin will get the pod to which the pod belongs from the pod's label information, and then get the uuid value of the GPU from the bound_gpus annotation of the pod, and query the major and minor device numbers of the GPU corresponding to the uuid. Finally, deviceplugin will enter the namespace of the pod, uninstall and delete the GPU devices not included in bound_gpus according to the major and minor device numbers. Because bound_gpus is added by gpuscheduler and represents the GPU assigned to the pod, after this operation, only the GPU specified by bound_gpus will remain in the pod.

[0085] Step S42: Through the device plug-in in the target computing node, when it is monitored that the second annotation of the container group changes, enter the namespace of the computing container, and add or delete a GPU for the computing container according to the universal unique identification code of the GPU indicated in the second annotation, so that the GPU currently allowed to be used by the computing container is the GPU indicated in the second annotation.

[0086] It should be noted that when the GPU redistribution is completed, the deviceplugin will monitor the change in the bound_gpus annotation value of the Pod. Then it will enter the namespace of the Pod's computing container and add or delete GPU devices according to the actual situation. In the above process, the deviceplugin enters the namespace of the Pod's computing container to uninstall, delete, and add GPU devices. The commands used are as follows:

[0087] #Check the correspondence between uuid and GPU device;

[0088] nvidia-smi-L;

[0089] #Check the primary and secondary device numbers of the device;

[0090] ls -l / dev / nvidia*;

[0091] #Query the main process number of the computing container;

[0092] pid=$(docker inspect--format'{{.State.Pid}}' <container-id>);

[0093] #nvidiaX represents the GPU device with index number X;

[0094] #Enter the namespace of the computing container to uninstall a GPU device;

[0095] nsenter-t$pid-a umount / dev / nvidiaX;

[0096] #Enter the namespace of the computing container and delete a GPU device;

[0097] nsenter-t$pid-a rm / dev / nvidiaX;

[0098] #Enter the namespace of the computing container and add a GPU device;

[0099] #major and minor are the major and minor device numbers of nvidiaX respectively;

[0100] nsenter -t$pid -a mknod --mode 0666 / dev / nvidiaX c major minor.

[0101] In an exemplary embodiment, when the resource management component fails to call the GPU scheduler plug-in to allocate computing nodes and GPUs to the container group based on the first annotation of the container group in the database component, the method further includes the following steps: controlling the container's access rights to the graphics card character device through cgroup.devices.allow, granting the container with the rwm permission the major device number and minor device number corresponding to the graphics card character device file on the host, allowing the container to access and operate the graphics card character device. In step S202, the graphics card character device file is mapped to the container through mount.entry, so that the container has a device file consistent with the host graphics card device and can access and use the graphics card device. In this way, it is possible to add or delete a GPU device to a container.

[0102] Alternatively, if Figure 7 As shown in the figure, the device permission control capability of the devices subsystem in the cgroup v1 version is used. By writing the graphics card device access permissions to devices.allow and devices.deny under the cgroup instance, the container can have access to these devices. cgroup v2 is a brand new cgroup version that has been redesigned to address the defects of cgroup v1 and has many excellent features. In many Linux distributions that use high-version kernels, cgroup v2 has become the default option.

[0103] In addition, to use graphics card devices in containers, you also need to install necessary files such as drivers and deep learning libraries. There are two ways to install them: one is to install these files on the host machine, and then let nvidia-container-runtime mount the container that has applied for the GPU, and the other is to install these files directly in the container.

[0104] It should be noted that the present application also has the following advantages: it realizes the dynamic addition or deletion of GPU devices to containers in the artificial intelligence development platform without the need to export the container as an image, without the need to delete and rebuild the container, and the process of adding or deleting GPU devices is relatively short.

[0105] It should be noted that, in addition to GPU, this method is also applicable to other acceleration card devices, such as: Neural Processing Unit (NPU), Machine Learning Unit (MLU), etc.

[0106] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0107] In this embodiment, a graphics processor allocation device is also provided, which is applied to a container orchestration platform. The container orchestration platform includes a management node and multiple computing nodes. The management node has a database component and a resource management component. The resource management component has a graphics processor GPU scheduler plug-in for realizing dynamic allocation of GPUs; the computing node has a node manager, a container runtime and multiple GPUs; the node manager has a device plug-in for realizing the management function of the GPU, which is used to implement the above-mentioned embodiments and preferred implementation methods. Those that have been explained will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware that implements a predetermined function. Although the modules described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0108] Figure 8 is a structural block diagram of a distribution device of a graphics processor according to an embodiment of the present application, the device comprising:

[0109] The allocation module 82 is used to call the GPU scheduler plug-in to allocate computing nodes and GPUs to the container group based on the first annotation of the container group in the database component through the resource management component, and obtain the target computing node and the second annotation of the container group, wherein the first annotation is used to indicate the number of GPUs requested by the container group, and the second annotation has universal unique identification codes of N allocated GPUs, the N GPUs are GPUs in the target computing node, and the multiple computing nodes include the target computing node; when the first annotation of the container group changes, the second annotation of the container group changes;

[0110] A creation module 84 is used to, through a node manager in the target computing node, call a container runtime in the target computing node based on a device file path to create a computing container for the container group when monitoring that the container group is assigned to the target computing node, wherein the device file corresponding to the device file path is a file required for adding all GPUs on the target computing node to the computing container, and the container group includes the computing container;

[0111] The first processing module 86 is used to add or delete a GPU for the computing container according to the second annotation of the container group through the device plug-in in the target computing node.

[0112] The above-mentioned device adds a graphics processor GPU scheduler plug-in in the resource management component and a device plug-in in the node manager. Since the resource management component can call the GPU scheduler plug-in to determine the second annotation of the container group according to the first annotation of the container group, the node manager of the computing node dynamically adds or deletes the GPU for the computing container of the container group according to the second annotation by calling the device plug-in, and there is no need to recreate the container to reallocate the GPU device, thereby solving the problem that the container does not support hot update of GPU devices, resulting in the inability to dynamically add or remove GPU devices during container operation.

[0113] In an exemplary embodiment, the above-mentioned device also includes: an acquisition module, which is used to call the device plug-in in the target computing node to obtain the device file path through the node manager in the target computing node when it monitors that the container group is assigned to the target computing node before calling the container runtime of the target computing node based on the device file path to create a computing container for the container group.

[0114] In an exemplary embodiment, the above-mentioned device also includes: a second processing module, which is used to call the GPU driver in the i-th computing node through the device plug-in in the i-th computing node among the multiple computing nodes to obtain the GPU list of the i-th computing node before calling the GPU scheduler plug-in to allocate computing nodes and GPUs to the container group based on the first annotation of the container group through the resource management component to obtain the target computing node and the second annotation of the container group, wherein the GPU list has device information of all GPUs of the i-th computing node, and the device information includes: GPU identification, GPU universal unique identification code and GPU status; through the device plug-in in the i-th computing node, add the GPU list of the i-th computing node to the third annotation of the i-th computing node, wherein the third annotation is used to indicate the GPU information in the computing node.

[0115] In an exemplary embodiment, the allocation module 82 is also used to call the GPU scheduler plug-in to determine the target computing node based on the first annotation of the container group in the database component through the resource management component when monitoring the creation event of the container group in the database component, wherein the number of idle GPUs in the target computing node is greater than the number of GPUs requested in the first annotation, and the number of idle GPUs in the target computing node is determined according to the third annotation of the target computing node, and the third annotation is used to indicate the GPU information in the computing node, and the GPU scheduler plug-in obtains the third annotation of each computing node in the multiple computing nodes during the initialization phase; call the GPU scheduler plug-in through the resource management component, select M GPUs from the idle GPU list of the target computing node, and add the universal unique identification codes of the M GPUs to the second annotation of the container group, wherein M is equal to the number of GPUs indicated by the first annotation; call the GPU scheduler plug-in through the resource management component to select the M GPUs from the idle GPU list of the target computing node, and add the universal unique identification codes of the M GPUs to the second annotation of the container group, wherein M is equal to the number of GPUs indicated by the first annotation; The target computing node is determined as the target computing node by the resource management component when the first annotation of the container group in the database component is monitored to change. When P is greater than Z and there are PZ idle GPUs in the target computing node, the GPU scheduler plug-in is called by the resource management component to select PZ GPUs from the idle GPU list of the target computing node, and the universal unique identification codes of the PZ GPUs are added to the second annotation of the container group, wherein Z is equal to the number of GPUs recorded in the second annotation of the container group, and P is equal to the number of GPUs indicated after the first annotation changes. The GPU scheduler plug-in is called by the resource management component to delete the PZ GPUs from the idle GPU list of the target computing node. When P is less than Z, the GPU scheduler plug-in is called by the resource management component to delete the universal unique identification codes of ZP GPUs from the second annotation of the container group, and add the ZP GPUs to the idle GPU list of the target computing node.

[0116] In an exemplary embodiment, the allocation module 82 is further used to, when P is greater than Z and there are no PZ idle GPUs in the target computing node, call the GPU scheduler plug-in through the resource management component to select X GPUs from the idle GPU list of the target computing node, and add the universal unique identification codes of the X GPUs to the second annotation of the container group, where X is the number of current idle GPUs of the target computing node, and X is an integer greater than or equal to 0 and less than PZ; call the GPU scheduler plug-in through the resource management component to delete the X GPUs from the idle GPU list of the target computing node; and send a prompt message through the resource management component, where the prompt message is used to indicate that the container group still needs to add a GPU.

[0117] In an exemplary embodiment, the first processing module 86 is also used to, through the device plug-in in the target computing node, enter the namespace of the computing container when a creation event of the computing container is monitored, and delete the GPU in the computing container that is not recorded in the second annotation according to the universal unique identifier of the GPU indicated in the second annotation; through the device plug-in in the target computing node, when a change in the second annotation of the container group is monitored, enter the namespace of the computing container, and add or delete a GPU for the computing container according to the universal unique identifier of the GPU indicated in the second annotation, so that the GPU currently allowed to be used by the computing container is the GPU indicated in the second annotation.

[0118] In an exemplary embodiment, the second processing module is also used for, after adding the GPU list of the i-th computing node to the third annotation of the i-th computing node through the device plug-in in the i-th computing node, registering the virtual device resources with the node manager of the i-th computing node through the device plug-in in the i-th computing node; recording in the database component through the node manager of the i-th computing node that the i-th computing node has the virtual device resources, wherein the virtual device resources are used to indicate that the computing node has the ability to dynamically change the number of GPUs used by the container; and calling the GPU scheduler plug-in to allocate computing nodes and GPUs to the container group based on the first annotation of the container group in the database component through the resource management component, including: calling the GPU scheduler plug-in based on the first annotation of the container group in the database component through the resource management component, allocating computing nodes to the container group from the computing nodes with the virtual device resources among the multiple computing nodes, and correspondingly allocating GPUs.

[0119] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0120] Optionally, in this embodiment, the computer program may be configured to perform the following steps by means of the computer program:

[0121] S1, through the resource management component, based on the first annotation of the container group in the database component, calling the GPU scheduler plug-in to allocate computing nodes and GPUs to the container group, and obtaining the target computing node and the second annotation of the container group, wherein the first annotation is used to indicate the number of GPUs requested by the container group, and the second annotation has universal unique identification codes of N allocated GPUs, the N GPUs are GPUs in the target computing node, and the multiple computing nodes include the target computing node; when the first annotation of the container group changes, the second annotation of the container group changes;

[0122] S2, through the node manager in the target computing node, when monitoring that the container group is assigned to the target computing node, calling the container runtime in the target computing node based on the device file path to create a computing container for the container group, wherein the device file corresponding to the device file path is a file required for adding all GPUs on the target computing node to the computing container, and the container group includes the computing container;

[0123] S3: Add or delete a GPU for the computing container according to the second annotation of the container group through the device plug-in in the target computing node.

[0124] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0125] The embodiment of the present application also provides an electronic device, such as Fig. 9 As shown, the electronic device includes a memory 902 and a processor 904. The memory 902 stores a computer program, and the processor 904 is configured to execute the steps in any of the above method embodiments through the computer program.

[0126] Optionally, in this embodiment, the processor 904 may be configured to perform the following steps through a computer program:

[0127] S1, through the resource management component, based on the first annotation of the container group in the database component, calling the GPU scheduler plug-in to allocate computing nodes and GPUs to the container group, and obtaining the target computing node and the second annotation of the container group, wherein the first annotation is used to indicate the number of GPUs requested by the container group, and the second annotation has universal unique identification codes of N allocated GPUs, the N GPUs are GPUs in the target computing node, and the multiple computing nodes include the target computing node; when the first annotation of the container group changes, the second annotation of the container group changes;

[0128] S2, through the node manager in the target computing node, when monitoring that the container group is assigned to the target computing node, calling the container runtime in the target computing node based on the device file path to create a computing container for the container group, wherein the device file corresponding to the device file path is a file required for adding all GPUs on the target computing node to the computing container, and the container group includes the computing container;

[0129] S3: Add or delete a GPU for the computing container according to the second annotation of the container group through the device plug-in in the target computing node.

[0130] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0131] Alternatively, a person skilled in the art may understand that: Fig. 9 The structure shown is for illustration only. Fig. 9 The structure of the electronic device is not limited. For example, the electronic device may also include Fig. 9 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig. 9 Different configurations shown.

[0132] Among them, the memory 902 can be used to store software programs and modules, such as the program instructions / modules corresponding to the allocation method of the graphics processor and the allocation device of the graphics processor in the embodiment of the present application. The processor 904 executes various functional applications and data processing by running the software programs and modules stored in the memory 902, that is, realizing the above-mentioned allocation method of the graphics processor. The memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 902 may further include a memory remotely located relative to the processor 904, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 902 can be specifically, but not limited to, used to store information such as system configuration files. As an example, such as Fig. 9 As shown, the memory 902 may include, but is not limited to, the allocation module 82, the creation module 84, and the first processing module 86 in the allocation device of the graphics processor. In addition, it may also include, but is not limited to, other module units in the allocation device of the graphics processor, which will not be repeated in this example.

[0133] Optionally, the transmission device 906 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one example, the transmission device 906 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 906 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0134] In addition, the electronic device further includes: a display 908; and a connection bus 910, which is used to connect various module components in the electronic device.

[0135] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0136] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0137] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the above method embodiments.

[0138] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0139] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for allocating a graphics processor, characterized in that: Applied to a container orchestration platform, the container orchestration platform includes a management node and multiple computing nodes, the management node has a database component and a resource management component, the resource management component has a graphics processor GPU scheduler plug-in for realizing dynamic allocation of GPUs; The computing node has a node manager, a container runtime and multiple GPUs; The node manager has a device plug-in for implementing the management function of the GPU; include: By means of the resource management component, based on the first annotation of the container group in the database component, the GPU scheduler plug-in is called to allocate a computing node and a GPU to the container group, and a second annotation of the target computing node and the container group is obtained, wherein the first annotation is used to indicate the number of GPUs requested by the container group, and the second annotation has universal unique identification codes of N allocated GPUs, the N GPUs are GPUs in the target computing node, and the multiple computing nodes include the target computing node; when the first annotation of the container group changes, the second annotation of the container group changes; By means of a node manager in the target computing node, when monitoring that the container group is assigned to the target computing node, a container runtime in the target computing node is called based on a device file path to create a computing container for the container group, wherein the device file corresponding to the device file path is a file required for adding all GPUs on the target computing node to the computing container, and the container group includes the computing container; A GPU is added or deleted for the computing container according to the second annotation of the container group through the device plug-in in the target computing node.

2. The method according to claim 1, characterized in that Before calling the container runtime of the target computing node based on the device file path to create a computing container for the container group, the method further includes: By means of the node manager in the target computing node, when it is monitored that the container group is allocated to the target computing node, the device plug-in in the target computing node is called to obtain the device file path.

3. The method according to claim 1, characterized in that The computing node also has a GPU driver; Before the resource management component is used to call the GPU scheduler plug-in based on the first annotation of the container group to allocate a computing node and a GPU to the container group, and obtains the second annotation of the target computing node and the container group, the method further includes: Calling a GPU driver in an i-th computing node through a device plug-in in the i-th computing node among the multiple computing nodes to obtain a GPU list of the i-th computing node, wherein the GPU list has device information of all GPUs of the i-th computing node, and the device information includes: an identifier of the GPU, a universal unique identification code of the GPU, and a status of the GPU; Adding the GPU list of the i-th computing node to the third annotation of the i-th computing node through the device plug-in in the i-th computing node, wherein the third annotation is used to indicate GPU information in the computing node.

4. The method according to claim 1, characterized in that: The method of calling the GPU scheduler plug-in to allocate a computing node and a GPU to the container group based on the first annotation of the container group in the database component through the resource management component to obtain a second annotation of the target computing node and the container group includes: Through the resource management component, in the case of monitoring the creation event of the container group in the database component, the GPU scheduler plug-in is called based on the first annotation of the container group in the database component to determine the target computing node, wherein the number of idle GPUs in the target computing node is greater than the number of GPUs requested in the first annotation, and the number of idle GPUs in the target computing node is determined according to the third annotation of the target computing node, and the third annotation is used to indicate the GPU information in the computing node, and the GPU scheduler plug-in obtains the third annotation of each computing node in the multiple computing nodes during the initialization phase; through the resource management component, the GPU scheduler plug-in is called to select M GPUs from the idle GPU list of the target computing node, and the universal unique identification codes of the M GPUs are added to the second annotation of the container group, wherein M is equal to the number of GPUs indicated by the first annotation; through the resource management component, the GPU scheduler plug-in is called to delete the M GPUs from the idle GPU list of the target computing node; and / or Through the resource management component, when it is monitored that the first annotation of the container group in the database component changes, it is determined that the computing node allocated by the container group is the target computing node; when P is greater than Z and there are PZ idle GPUs in the target computing node, through the resource management component, the GPU scheduler plug-in is called to select PZ GPUs from the idle GPU list of the target computing node, and the universal unique identification codes of the PZ GPUs are added to the second annotation of the container group, wherein Z is equal to the number of GPUs recorded in the second annotation of the container group, and P is equal to the number of GPUs indicated after the first annotation changes; through the resource management component, the GPU scheduler plug-in is called to delete the PZ GPUs from the idle GPU list of the target computing node; when P is less than Z, the GPU scheduler plug-in is called to delete the universal unique identification codes of ZP GPUs from the second annotation of the container group, and the ZP GPUs are added to the idle GPU list of the target computing node.

5. The method according to claim 4, characterized in that The method further comprises: When P is greater than Z and there are no PZ idle GPUs in the target computing node, the GPU scheduler plug-in is called through the resource management component to select X GPUs from the idle GPU list of the target computing node, and the universal unique identifiers of the X GPUs are added to the second annotation of the container group, where X is the number of current idle GPUs of the target computing node, and X is an integer greater than or equal to 0 and less than PZ; By calling the GPU scheduler plug-in through the resource management component, the X GPUs are deleted from the idle GPU list of the target computing node; and Prompt information is sent through the resource management component, wherein the prompt information is used to indicate that a GPU needs to be added to the container group.

6. The method according to claim 1, characterized in that The adding or deleting a GPU for the computing container according to the second annotation of the container group through the device plug-in in the target computing node includes: When a creation event of the computing container is monitored, the device plug-in in the target computing node enters the namespace of the computing container and deletes the GPU in the computing container that is not recorded in the second annotation according to the universal unique identifier of the GPU indicated in the second annotation; Through the device plug-in in the target computing node, when it is monitored that the second annotation of the container group changes, the namespace of the computing container is entered, and a GPU is added or deleted for the computing container according to the universal unique identifier of the GPU indicated in the second annotation, so that the GPU currently allowed to be used by the computing container is the GPU indicated in the second annotation.

7. The method according to claim 3, characterized in that After adding the GPU list of the i-th computing node to the third annotation of the i-th computing node through the device plug-in in the i-th computing node, the method further includes: registering virtual device resources with the node manager of the i-th computing node through the device plug-in in the i-th computing node; recording in the database component through the node manager of the i-th computing node that the i-th computing node has the virtual device resources, wherein the virtual device resources are used to indicate that the computing node has the ability to dynamically change the number of GPUs used by the container; The method comprises: calling the GPU scheduler plug-in based on the first annotation of the container group in the database component through the resource management component to allocate computing nodes and GPUs to the container group, including: calling the GPU scheduler plug-in based on the first annotation of the container group in the database component through the resource management component, allocating computing nodes to the container group from computing nodes with the virtual device resources in the multiple computing nodes, and correspondingly allocating GPUs.

8. A device for allocating a graphics processor, applied to a container orchestration platform, the container orchestration platform comprising a management node and a plurality of computing nodes, the management node comprising a database component and a resource management component, the resource management component comprising a graphics processor GPU scheduler plug-in for implementing dynamic allocation of GPUs; the computing node comprising a node manager, a container runtime and a plurality of GPUs; the node manager comprising a device plug-in for implementing a management function of the GPU, characterized in that: include: an allocation module, configured to call the GPU scheduler plug-in to allocate computing nodes and GPUs to the container group based on the first annotation of the container group in the database component through the resource management component, and obtain a second annotation of the target computing node and the container group, wherein the first annotation is used to indicate the number of GPUs requested by the container group, and the second annotation has universal unique identification codes of N allocated GPUs, the N GPUs are GPUs in the target computing node, and the multiple computing nodes include the target computing node; when the first annotation of the container group changes, the second annotation of the container group changes; A creation module, configured to, through a node manager in the target computing node, call a container runtime in the target computing node based on a device file path to create a computing container for the container group when monitoring that the container group is assigned to the target computing node, wherein the device file corresponding to the device file path is a file required for adding all GPUs on the target computing node to the computing container, and the container group includes the computing container; The first processing module is used to add or delete a GPU for the computing container according to the second annotation of the container group through a device plug-in in the target computing node.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Images