Gpu resource scheduling method and device, equipment and storage medium

Abstract

The application relates to the technical field of clouds, and discloses a GPU resource scheduling method, device and equipment and a storage medium. The method comprises the following steps: obtaining registration information of a target container for GPU resources, and creating socket information of a simulated card slot of the target container in the GPU resources according to the registration information; based on the socket information, performing virtualization processing on the target container on a K8S cluster to obtain node resource information corresponding to the target container on the K8S cluster; dividing the GPU resources according to the node resource information, and allocating the divided GPU resources to the target container through the K8S cluster; obtaining an operation instruction for the target container on the K8S cluster, and scheduling GPU resources corresponding to the operation instruction in the GPU resources allocated to the target container according to the socket information. The application realizes that multiple containers share the resources of one GPU card, and improves the flexibility of the use of GPU resource requirements.

Description

Technical Field

[0001] This invention relates to the field of cloud technology, and in particular to a GPU resource scheduling method, apparatus, device, and storage medium. Background Technology

[0002] As business continues to grow, maintaining the stability and reliability of business has brought increasing challenges to the maintenance, management, and elastic scaling of application services. To cope with the ever-changing operational management and development needs, many businesses have chosen to containerize their services and then manage them using Kubernetes (K8S, a container cluster management system). Relying on Kubernetes, containers can be managed flexibly, easily, efficiently, and securely, and development versions can be rapidly iterated and deployed. This is especially true for GPU-intensive computing services.

[0003] Because Kubernetes nodes cannot manage GPU resources natively like CPU and memory resources, they need to rely on third-party official device plugins, such as those from mainstream AMD and NVIDIA. However, relying on these has the following limitations: 1) When setting GPU resource requirements, only limits can be specified, unlike CPU and memory where request limits can be set. This means GPU resources cannot be flexibly allocated; only the requested amount is used. 2) Currently, GPUs cannot be shared between containers, nor can they be over-allocated; that is, one GPU can only belong to one container. In short, this results in low efficiency in GPU resource allocation and invocation. Summary of the Invention

[0004] The main objective of this invention is to solve the technical problem of low efficiency in allocating and calling GPU resources.

[0005] The first aspect of this invention provides a GPU resource scheduling method, comprising: acquiring registration information of a target container for GPU resources, and creating socket information for the target container to simulate a card slot in the GPU resources based on the registration information; virtualizing the target container on a Kubernetes cluster based on the socket information to obtain node resource information corresponding to the target container in the Kubernetes cluster; partitioning the GPU resources according to the node resource information, and allocating the partitioned GPU resources to the target container through the Kubernetes cluster; acquiring operation instructions for the target container on the Kubernetes cluster, and scheduling the GPU resources corresponding to the operation instructions from the GPU resources allocated to the target container with reference to the socket information.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the step of creating socket information for the target container to simulate a card slot in the GPU resource based on the registration information includes: determining the video memory capacity to be registered by the target container in the GPU resource and the registration name of the simulated card slot based on the registration information; creating a socket for the target container in the GPU resource using the registration name according to a preset format, and generating socket information based on the socket and the video memory capacity.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of virtualizing the target container on a K8S cluster based on the socket information to obtain the node resource information corresponding to the target container in the K8S cluster includes: determining the process information of the target container and calculating the video memory requirements of each process in the process information; determining the resource information of each GPU card associated with the K8S cluster, and allocating node resources of each process on the corresponding GPU card based on the resource information and the video memory requirements; and generating the node resource information of the target container in the K8S cluster according to the allocated node resources and the socket information.

[0008] Optionally, in a third implementation of the first aspect of the present invention, generating the node resource information of the target container in the K8S cluster based on the allocated node resources and the socket information includes: binding the node resources of the corresponding GPU cards allocated to each process in the K8S cluster with the socket information based on the allocated node resources; and generating the node resource information of the target container in the K8S cluster based on the binding result.

[0009] Optionally, in the fourth implementation of the first aspect of the present invention, after dividing the GPU resources according to the node resource information and allocating the divided GPU resources to the target container through the K8S cluster, the method further includes: monitoring the video memory usage of each process in the target container, and determining whether the monitored video memory usage exceeds the video memory requirement of the corresponding process; if it exceeds, then executing a preset restart strategy for the target container in the K8S cluster, and reallocating the node resources of each process on the corresponding GPU card according to the video memory usage.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of scheduling the GPU resource corresponding to the operation instruction from the GPU resources allocated to the target container with reference to the socket information includes: determining the emulator slot pointed to by the operation instruction with reference to the socket information, and sending the operation instruction to the GPU partition corresponding to the GPU resource allocated to the target container through the emulator slot; traversing the GPU partition using the operation instruction, and determining the GPU resource operated by the operation instruction based on the traversal result; and scheduling the GPU resource operated by the operation instruction to the target container.

[0011] A second aspect of the present invention provides a GPU resource scheduling apparatus, comprising: a creation module, configured to acquire registration information of a target container for GPU resources, and create socket information for the target container to simulate a card slot in the GPU resources based on the registration information; a virtualization module, configured to virtualize the target container on a Kubernetes cluster based on the socket information to obtain node resource information corresponding to the target container in the Kubernetes cluster; a partitioning module, configured to partition the GPU resources according to the node resource information, and allocate the partitioned GPU resources to the target container through the Kubernetes cluster; and a scheduling module, configured to acquire operation instructions for the target container on the Kubernetes cluster, and schedule the GPU resources corresponding to the operation instructions from the GPU resources allocated to the target container with reference to the socket information.

[0012] Optionally, in a first implementation of the second aspect of the present invention, the creation module includes: a determining unit, configured to determine, based on the registration information, the video memory capacity to be registered by the target container in the GPU resources and the registration name of the simulated card slot; and a creation unit, configured to create a socket for the target container in the GPU resources according to a preset format using the registration name, and generate socket information based on the socket and the video memory capacity.

[0013] Optionally, in a second implementation of the second aspect of the present invention, the virtualization module includes: a statistics unit, configured to determine the process information of the target container and to count the video memory requirements of each process in the process information; an allocation unit, configured to determine the resource information of each GPU card associated with the K8S cluster, and to allocate node resources of each process on the corresponding GPU card based on the resource information and the video memory requirements; and a generation unit, configured to generate node resource information of the target container in the K8S cluster according to the allocated node resources and the socket information.

[0014] Optionally, in a third implementation of the second aspect of the present invention, the generation unit is further configured to: bind the node resources of the corresponding GPU card allocated to each process in the K8S cluster with the socket information according to the allocated node resources; and generate the node resource information of the target container in the K8S cluster based on the binding result.

[0015] Optionally, in the fourth implementation of the second aspect of the present invention, the GPU resource scheduling device further includes a restart module, configured to: monitor the memory usage of each process in the target container, and determine whether the monitored memory usage exceeds the memory requirement of the corresponding process; if it exceeds, execute a preset restart strategy on the target container in the K8S cluster, and reallocate the node resources of each process on the corresponding GPU card according to the memory usage.

[0016] Optionally, in a fifth implementation of the second aspect of the present invention, the scheduling module includes: a sending unit, configured to determine the simulated card slot pointed to by the operation instruction by referring to the socket information, and send the operation instruction to the GPU partition corresponding to the GPU resource allocated to the target container through the simulated card slot; a traversal unit, configured to traverse the GPU partition using the operation instruction, and determine the GPU resource operated by the operation instruction based on the traversal result; and a scheduling unit, configured to schedule the GPU resource operated by the operation instruction to the target container.

[0017] A third aspect of the present invention provides a GPU resource scheduling device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the GPU resource scheduling device to execute the GPU resource scheduling method described above.

[0018] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the GPU resource scheduling method described above.

[0019] In the technical solution provided by this invention, socket information is generated by simulating a card slot in GPU resources for the target container. This allows containers to share the resources of the same GPU card in the simulated card slot, or to allocate multiple processes of the container to multiple GPU cards. Then, based on the socket information, the target container can be virtualized on the K8S cluster to obtain the node resource information corresponding to the target container in the K8S cluster. The GPU resources are then allocated and distributed to the target container. Subsequently, when an operation command for the target container is obtained, the GPU resources corresponding to the operation command can be scheduled from the GPU resources allocated to the target container according to the socket information. This enables multiple containers to share the resources of a single GPU card, improving the flexibility of GPU resource usage. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the first embodiment of the GPU resource scheduling method in this invention;

[0021] Figure 2 This is a schematic diagram of a second embodiment of the GPU resource scheduling method in this invention;

[0022] Figure 3 This is a schematic diagram of a third embodiment of the GPU resource scheduling method in this invention;

[0023] Figure 4 This is a schematic diagram of one embodiment of the GPU resource scheduling device in this invention;

[0024] Figure 5 This is a schematic diagram of another embodiment of the GPU resource scheduling device in this invention;

[0025] Figure 6 This is a schematic diagram of one embodiment of the GPU resource scheduling device in this invention. Detailed Implementation

[0026] This invention provides a GPU resource scheduling method, apparatus, device, and storage medium. The method involves acquiring the registration information of a target container for GPU resources, and based on this registration information, creating socket information for the target container to simulate a GPU slot within the GPU resources. Based on this socket information, the target container is virtualized on a Kubernetes (K8S) cluster to obtain the node resource information corresponding to the target container in the K8S cluster. According to the node resource information, GPU resources are partitioned, and the partitioned GPU resources are allocated to the target container via the K8S cluster. Operation instructions for the target container on the K8S cluster are acquired, and, referring to the socket information, the corresponding GPU resources are scheduled from the GPU resources allocated to the target container. This invention enables multiple containers to share the resources of a single GPU card, improving the flexibility of GPU resource usage.

[0027] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the GPU resource scheduling method in this invention includes:

[0029] 101. Obtain the registration information of the target container for the GPU resources, and based on the registration information, create the socket information of the target container simulating the card slot in the GPU resources;

[0030] It is understood that the executing entity of this invention can be a GPU resource scheduling device, a terminal, or a server; no specific limitation is made here. This embodiment of the invention will be described using a server as an example.

[0031] In this embodiment, a device plugin corresponding to this GPU resource scheduling server is added between the Kubernetes cluster and the GPU resources. Subsequently, the Kubernetes cluster can schedule GPU resources through this device plugin. First, socket information for the target container to interface with the GPU resources is created. The Kubernetes cluster then specifically uses this socket information on the device plugin to schedule GPU resources.

[0032] Specifically, when it is necessary to register GPU resources for a target container, its registration information is obtained, including the basic information of the device where the target container is located, such as the video memory capacity required by the target container for the GPU card, the card slot path and capacity serial number, etc., to determine the required GPU resource area, and to create the socket information of the simulated card slot in a pre-set format, such as "GPU card slot"-"capacity serial number", so as to determine the GPU resource interface and allocation area.

[0033] 102. Based on the socket information, the target container is virtualized on the K8S cluster to obtain the node resource information of the target container in the K8S cluster.

[0034] In this embodiment, the socket information of the simulated card slot is used to connect to the GPU resources required by the target container, and then the target container is registered with the K8S cluster. Since the user actually controls the GPU resources through K8S, the target container needs to be virtualized on the K8S cluster. The virtual socket in the socket information is used to connect to the virtual card slot corresponding to the GPU resources that the K8S cluster needs to operate when controlling the target container, so as to realize the conversion from virtual socket to real physical GPU hardware.

[0035] Additionally, when the target container is virtualized to the K8S cluster, although the socket information is created by simulating a card slot, it is not a physical card slot but is simulated by path naming. Therefore, when applying, the resource name can be requested by the memory capacity through the socket. Thus, when the target container's service requests GPU resources in the K8S cluster, it requests GPU resources with the memory capacity rather than the number of GPU cards. The registration of each process of the target container virtualization in the GPU resources is represented as node resource information.

[0036] 103. Based on the node resource information, the GPU resources are divided, and the divided GPU resources are allocated to the target container through the K8S cluster;

[0037] In this embodiment, the node resource information includes: Kubernetes cluster nodes (node_name); Kubernetes cluster pod resources (pod_name); Kubernetes cluster container resources (container_name); the internal network IP address (cluster_ip) corresponding to the pod within the Kubernetes cluster (virtual_ip_host); GPU resources (gpu_resource), given by GPU ID, number of CUDA cores (cuda_core), and CUDA storage (cuda_memory); memory (memory), given by size (e.g., 108Gi); and volume storage (volume), given by size (e.g., 500Gi). Each node's corresponding pod is associated with the socket of the corresponding target container. The pod can allocate, distribute, and call GPU resources according to the socket. Therefore, node resources can be used here to achieve the allocation and distribution of GPU resources to the target container through the Kubernetes cluster.

[0038] 104. Obtain the operation instructions for the target container on the K8S cluster, and schedule the GPU resources corresponding to the operation instructions from the GPU resources allocated to the target container, referring to the socket information.

[0039] In this embodiment, after the target container registers the required GPU resources on K8S through socket information, when operating on the target container on the K8S cluster to operate on GPU resources, the GPU resources are actually scheduled indirectly through the created socket fields. At the same time, when it is determined that the service is executed on the GPU card, it is executed according to the GPU resources allocated to the target container, realizing the local scheduling of GPU resources for the target container. Other containers allocated GPU resources on the same GPU card can also schedule the corresponding GPU resources, realizing that multiple containers share the GPU resources on the same GPU card.

[0040] In this embodiment of the invention, by creating socket information in the GPU resources to simulate a card slot for the target container, the container can share the resources of the same GPU card in the simulated card slot, or the multiple processes of the container can be allocated to multiple GPU cards. Then, based on the socket information, the target container can be virtualized on the Kubernetes cluster to obtain the node resource information corresponding to the target container in the Kubernetes cluster. The GPU resources are then allocated and distributed to the target container. Subsequently, when an operation command for the target container is obtained, the GPU resources corresponding to the operation command can be scheduled from the GPU resources allocated to the target container according to the socket information. This enables multiple containers to share the resources of a single GPU card, improving the flexibility of GPU resource usage.

[0041] Please see Figure 2 The second embodiment of the GPU resource scheduling method in this invention includes:

[0042] 201. Obtain the registration information of the target container for GPU resources, and determine the video memory capacity to be registered by the target container in the GPU resources and the registration name of the simulated card slot based on the registration information;

[0043] 202. According to the preset format, create a socket for the target container in the GPU resources using the registered name, and generate socket information based on the socket and the video memory capacity;

[0044] In this embodiment, performing specific initialization by the GPU vendor can determine the basic information of the device where the target container is located, which may include the video memory capacity of the GPU card. After determining the video memory capacity (in GB), a corresponding number of sockets (specifically virtual Unix sockets) can be created. The format can be referred to as “GPU card slot path”-“serial number”, and a preset format is used to indicate whether the virtual Unix socket is used. A layer of virtualization is used to achieve resource refinement of a GPU card.

[0045] By associating the GPU resource's memory capacity with the socket, the memory usage of the corresponding virtual card slot is determined and encapsulated in the socket information. When it is necessary to schedule the simulated card slot of the GPU resource through the plug-in device that extends this GPU resource scheduling server, the specific simulated card slot to be operated can be obtained through the virtual Unix socket, and the required GPU resources can be adjusted to realize the conversion from virtual Unix socket to real physical GPU hardware.

[0046] 203. Determine the process information of the target container and count the video memory requirements of each process in the process information;

[0047] 204. Determine the resource information of each GPU card associated with the K8S cluster, and allocate node resources of each process on the corresponding GPU card based on the resource information and the memory requirements.

[0048] 205. Based on the allocated node resources and the socket information, generate the node resource information of the target container in the K8S cluster;

[0049] In this embodiment, when allocating resources, priority is given to grouping virtual Unix sockets with the same "GPU slot path" part together. If they cannot be allocated to the same GPU card, the required GPU resources can be split across multiple GPU cards. The target typically includes one or more processes, forming process information; here, multiple processes are allocated across multiple GPU cards to achieve GPU resource allocation.

[0050] Additionally, when allocating multiple processes across multiple GPU cards, the GPU resource allocation limits for each process are broken down accordingly to ensure that the allocated GPU resources on each GPU card are adequately allocated. Specifically, the GPU memory usage (i.e., memory requirement) of each process in the target container can be determined through the hardware manufacturer's driver, and the corresponding number of node resources for each GPU card associated with the Kubernetes cluster can be randomly selected.

[0051] Specifically, when registering the GPU resources required by a target container on a Kubernetes cluster, registration can be done through the gRPC service in Kubernetes. In addition to the Unix virtual socket, the registration also includes the local device plugin API version and the ResourceName defined based on the Unix virtual socket. The ResourceName is the name published in Kubernetes for GPU resource management.

[0052] 206. Based on the node resource information, the GPU resources are divided, and the divided GPU resources are allocated to the target container through the K8S cluster;

[0053] 207. Obtain the operation instructions for the target container on the K8S cluster, and schedule the GPU resources corresponding to the operation instructions from the GPU resources allocated to the target container, referring to the socket information.

[0054] In this embodiment of the invention, by converting the physical GPU card into a virtual Unix socket in the format of "GPU card slot path"-"serial number" and registering it in K8S, a single GPU card is divided into more granular resources for allocation and management. This allows the K8S cluster to manage a single GPU card that can be shared between containers and improves the flexibility of GPU card resource allocation.

[0055] Please see Figure 3 The third embodiment of the GPU resource scheduling method in this invention includes:

[0056] 301. Obtain the registration information of the target container for the GPU resources, and based on the registration information, create the socket information of the target container simulating the card slot in the GPU resources;

[0057] 302. Determine the process information of the target container and count the video memory requirements of each process in the process information;

[0058] 303. Determine the resource information of each GPU card associated with the K8S cluster, and allocate node resources of each process on the corresponding GPU card based on the resource information and the video memory requirement;

[0059] 304. Based on the allocated node resources, bind the node resources of the corresponding GPU cards allocated to each process with the socket information in the K8S cluster;

[0060] 305. Based on the binding result, generate the node resource information of the target container in the K8S cluster;

[0061] In this embodiment, the node resources allocated to each process in the target container are associated and bound with the virtual Unix socket assigned to the target container to generate node resource information. In addition, processes can be marked as in use to ensure that resource requests are less than or equal to the request limit, while also guaranteeing that the total GPU resource requests for node resources are less than the total number of managed virtual Unix sockets.

[0062] It should be noted that the number of target containers here can be one container or multiple containers, without specific limitations. That is, one container can include multiple processes, or one container can be one process, and GPU resources can be allocated to the processes corresponding to multiple containers.

[0063] 306. Based on the node resource information, the GPU resources are divided, and the divided GPU resources are allocated to the target container through the K8S cluster;

[0064] In this embodiment, after allocating GPU resources to the target container, to ensure that each process in the target container requests less GPU resources than its allocated resources, the GPU memory usage of each pod can be monitored. Specifically, the memory usage of the corresponding process can be monitored. When the GPU memory usage of a single pod exceeds the limit set when the process is registered in the Kubernetes cluster (memory requirement), a restart policy is automatically issued for that pod. This ensures the isolation of multiple processes occupying GPU resources after GPU card virtualization, and the pod is associated with a socket, effectively reallocating GPU resources for each process in the corresponding target container. The specific steps are as follows:

[0065] 1) Monitor the video memory usage of each process in the target container, and determine whether the monitored video memory usage exceeds the video memory requirement of the corresponding process.

[0066] 2) If the limit is exceeded, a preset restart policy is executed on the target container in the K8S cluster, and the node resources of each process on the corresponding GPU card are reallocated according to the memory usage.

[0067] Additionally, it can monitor the GPU service resource usage of each process in a format consistent with Prometheus and output GPU memory usage data for each pod.

[0068] 307. Obtain the operation instructions for the target container on the K8S cluster;

[0069] 308. Referring to the socket information, determine the simulated card slot to which the operation command points, and send the operation command to the GPU partition corresponding to the GPU resources allocated to the target container through the simulated card slot;

[0070] 309. Use the operation instructions to traverse the GPU partitions, and determine the GPU resources operated by the operation instructions based on the traversal results;

[0071] 310. Schedule the GPU resources operated by the operation instruction to the target container.

[0072] In this embodiment, socket information is used to simulate the path pointing to the card slot and obtain the GPU partition corresponding to the card slot. Multiple containers can select the corresponding GPU partition to share the GPU resources in the same GPU card. Then, after the operation command points to the GPU partition allocated to the target container, the GPU resources in that GPU partition can be scheduled according to the identifier in the operation command.

[0073] Specifically, before the K8S cluster can operate on the target container, the GPU resource allocation service needs to be started. First, the initialization settings are executed, and the gRPC service is started using the previously generated Unix virtual socket. The call path can be / var / lib / kubelet / device-plugins / . Then, it registers itself with the K8S cluster. The call path can be / var / lib / kubelet / device-plugins / kubelet.sock. After that, it runs in a service model and continuously monitors the running status and reports to the K8S cluster.

[0074] In this embodiment of the invention, a restart strategy is executed after the video memory usage of a pod in the K8S cluster exceeds the GPU resource usage limit. This ensures virtual resource isolation of multiple pods using the same GPU card and guarantees that each pod is provided with sufficient GPU resources under the video memory limit. At the same time, the GPU video memory usage of each pod is output, which improves the stability of GPU resource scheduling under K8S cluster management.

[0075] The GPU resource scheduling method in the embodiments of the present invention has been described above. The GPU resource scheduling apparatus in the embodiments of the present invention will be described below. Please refer to [link / reference]. Figure 4 One embodiment of the GPU resource scheduling device in this invention includes:

[0076] A creation module 401 is used to obtain the registration information of the target container for GPU resources, and create the socket information of the target container simulating the card slot in the GPU resources according to the registration information;

[0077] The virtualization module 402 is used to virtualize the target container on the K8S cluster based on the socket information to obtain the node resource information of the target container in the K8S cluster.

[0078] The partitioning module 403 is used to partition the GPU resources according to the node resource information, and allocate the partitioned GPU resources to the target container through the K8S cluster;

[0079] The scheduling module 404 is used to obtain the operation instructions for the target container on the K8S cluster, and, with reference to the socket information, schedule the GPU resources corresponding to the operation instructions from the GPU resources allocated to the target container.

[0080] In this embodiment of the invention, by creating socket information in the GPU resources to simulate a card slot for the target container, the container can share the resources of the same GPU card in the simulated card slot, or the multiple processes of the container can be allocated to multiple GPU cards. Then, based on the socket information, the target container can be virtualized on the Kubernetes cluster to obtain the node resource information corresponding to the target container in the Kubernetes cluster. The GPU resources are then allocated and distributed to the target container. Subsequently, when an operation command for the target container is obtained, the GPU resources corresponding to the operation command can be scheduled from the GPU resources allocated to the target container according to the socket information. This enables multiple containers to share the resources of a single GPU card, improving the flexibility of GPU resource usage.

[0081] Please see Figure 5 Another embodiment of the GPU resource scheduling device in this invention includes:

[0082] A creation module 401 is used to obtain the registration information of the target container for GPU resources, and create the socket information of the target container simulating the card slot in the GPU resources according to the registration information;

[0083] The virtualization module 402 is used to virtualize the target container on the K8S cluster based on the socket information to obtain the node resource information of the target container in the K8S cluster.

[0084] The partitioning module 403 is used to partition the GPU resources according to the node resource information, and allocate the partitioned GPU resources to the target container through the K8S cluster;

[0085] The scheduling module 404 is used to obtain the operation instructions for the target container on the K8S cluster, and, with reference to the socket information, schedule the GPU resources corresponding to the operation instructions from the GPU resources allocated to the target container.

[0086] Specifically, the creation module 401 includes:

[0087] The determining unit 4011 is used to determine, based on the registration information, the video memory capacity to be registered in the GPU resources of the target container and the registration name of the simulated card slot;

[0088] Creation unit 4012 is used to create a socket for the target container in the GPU resources according to a preset format using the registered name, and to generate socket information based on the socket and the video memory capacity.

[0089] Specifically, the virtualization module 402 includes:

[0090] The statistics unit 4021 is used to determine the process information of the target container and to count the video memory requirements of each process in the process information.

[0091] The allocation unit 4022 is used to determine the resource information of each GPU card associated with the K8S cluster, and allocate the node resources of each process on the corresponding GPU card based on the resource information and the video memory requirement.

[0092] The generation unit 4023 is used to generate node resource information of the target container in the K8S cluster based on the allocated node resources and the socket information.

[0093] Specifically, the generation unit 4023 is further used for:

[0094] Based on the allocated node resources, the node resources of the corresponding GPU cards allocated to each process are bound to the socket information in the K8S cluster;

[0095] Based on the binding result, the node resource information of the target container in the K8S cluster is generated.

[0096] Specifically, the GPU resource scheduling device further includes a restart module 405, used for:

[0097] Monitor the video memory usage of each process in the target container, and determine whether the monitored video memory usage exceeds the video memory requirement of the corresponding process.

[0098] If the limit is exceeded, a preset restart policy is executed on the target container in the K8S cluster, and the node resources of each process on the corresponding GPU card are reallocated according to the memory usage.

[0099] Specifically, the scheduling module 404 includes:

[0100] The sending unit 4041 is used to refer to the socket information to determine the simulated card slot to which the operation instruction points, and send the operation instruction to the GPU partition corresponding to the GPU resources allocated to the target container through the simulated card slot.

[0101] Traversal unit 4042 is used to traverse the GPU partition using the operation instruction, and determine the GPU resource operated by the operation instruction based on the traversal result;

[0102] The scheduling unit 4043 is used to schedule the GPU resources operated by the operation instruction to the target container.

[0103] In this embodiment of the invention, by converting the physical GPU card into a virtual Unix socket in the format of "GPU slot path"-"serial number" and registering it in Kubernetes, a single GPU card is divided into more granular resources for allocation and management. This allows the Kubernetes cluster to manage a single GPU card that can be shared among containers and improves the flexibility of GPU card resource allocation. In addition, by executing a restart policy after the GPU memory usage of a pod in the Kubernetes cluster exceeds the GPU resource usage limit, the virtual isolation of resources among multiple pods using the same GPU card is ensured, guaranteeing that each pod is provided with sufficient GPU resources under the memory limit. At the same time, the GPU memory usage of each pod is output, improving the stability of GPU resource scheduling under Kubernetes cluster management.

[0104] above Figure 4 and Figure 5 The GPU resource scheduling device in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The GPU resource scheduling device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0105] Figure 6 This is a schematic diagram of a GPU resource scheduling device 600 provided in an embodiment of the present invention. The GPU resource scheduling device 600 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 610 (e.g., one or more processors) and memory 620, and one or more storage media 630 (e.g., one or more mass storage devices) for storing application programs 633 or data 632. The memory 620 and storage media 630 can be temporary or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the GPU resource scheduling device 600. Furthermore, the processor 610 may be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the GPU resource scheduling device 600.

[0106] The GPU resource scheduling device 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input / output interfaces 660, and / or one or more operating systems 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 6 The GPU resource scheduling device structure shown does not constitute a limitation on the GPU resource scheduling device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0107] The present invention also provides a GPU resource scheduling device, wherein the computer device includes a memory and a processor, the memory storing computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor performs the steps of the GPU resource scheduling method in the above embodiments.

[0108] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the GPU resource scheduling method.

[0109] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0110] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A GPU resource scheduling method, characterized in that, The GPU resource scheduling method includes: The process involves obtaining the target container's registration information for GPU resources. This registration information includes basic information about the device where the target container resides: the required video memory capacity, slot path, and capacity serial number for the GPU card, thus determining the required GPU resource area. Based on this registration information, the process creates socket information for the target container to simulate a slot within the GPU resources. This creation includes: determining the video memory capacity to be registered for the target container within the GPU resources and the registration name of the simulated slot based on the registration information; creating a socket for the target container within the GPU resources using the registration name according to a preset format; and generating socket information based on the socket and the video memory capacity. Based on the socket information, the target container is virtualized on the K8S cluster to obtain the node resource information of the target container in the K8S cluster. Based on the node resource information, the GPU resources are divided, and the divided GPU resources are allocated to the target container through the K8S cluster; Obtain the operation instructions for the target container on the Kubernetes cluster, and schedule the GPU resources corresponding to the operation instructions from the GPU resources allocated to the target container, referring to the socket information. The process of scheduling the GPU resources corresponding to the operation instructions from the GPU resources allocated to the target container, referring to the socket information, includes: determining the emulator slot pointed to by the operation instructions, and sending the operation instructions to the GPU partition corresponding to the GPU resources allocated to the target container through the emulator slot; traversing the GPU partitions using the operation instructions, and determining the GPU resources operated by the operation instructions based on the traversal results; and scheduling the GPU resources operated by the operation instructions to the target container.

2. The GPU resource scheduling method according to claim 1, characterized in that, Based on the socket information, the target container is virtualized on the Kubernetes cluster to obtain the node resource information corresponding to the target container in the Kubernetes cluster, including: Determine the process information of the target container and count the video memory requirements of each process in the process information; Determine the resource information of each GPU card associated with the K8S cluster, and allocate node resources for each process on the corresponding GPU card based on the resource information and the memory requirements. Based on the allocated node resources and the socket information, the node resource information of the target container in the K8S cluster is generated.

3. The GPU resource scheduling method according to claim 2, characterized in that, The step of generating the node resource information of the target container in the K8S cluster based on the allocated node resources and the socket information includes: Based on the allocated node resources, the node resources of the corresponding GPU cards allocated to each process are bound to the socket information in the K8S cluster; Based on the binding result, the node resource information of the target container in the K8S cluster is generated.

4. The GPU resource scheduling method according to claim 2, characterized in that, After partitioning the GPU resources according to the node resource information and allocating the partitioned GPU resources to the target container through the K8S cluster, the method further includes: Monitor the video memory usage of each process in the target container, and determine whether the monitored video memory usage exceeds the video memory requirement of the corresponding process. If the limit is exceeded, a preset restart policy is executed on the target container in the K8S cluster, and the node resources of each process on the corresponding GPU card are reallocated according to the memory usage.

5. A GPU resource scheduling device, characterized in that, The GPU resource scheduling device includes: A creation module is used to obtain the registration information of the target container for GPU resources. The registration information includes basic information of the device where the target container is located: the video memory capacity required by the target container for the GPU card, the card slot path, and the capacity serial number, and to determine the required GPU resource area. Based on the registration information, the module creates socket information for the target container to simulate the card slot in the GPU resources. Based on the registration information, the module determines the video memory capacity to be registered by the target container in the GPU resources and the registration name of the simulated card slot. According to a preset format, the module creates a socket for the target container in the GPU resources using the registration name, and generates socket information based on the socket and the video memory capacity. The virtualization module is used to virtualize the target container on the K8S cluster based on the socket information, so as to obtain the node resource information of the target container in the K8S cluster. The partitioning module is used to partition the GPU resources according to the node resource information, and allocate the partitioned GPU resources to the target container through the K8S cluster; The scheduling module is used to obtain operation instructions for the target container on the K8S cluster, and schedule the GPU resources corresponding to the operation instructions from the GPU resources allocated to the target container with reference to the socket information; determine the simulated card slot pointed to by the operation instructions with reference to the socket information, and send the operation instructions to the GPU partition corresponding to the GPU resources allocated to the target container through the simulated card slot; traverse the GPU partitions with the operation instructions, and determine the GPU resources operated by the operation instructions based on the traversal results; and schedule the GPU resources operated by the operation instructions to the target container.

6. The GPU resource scheduling device according to claim 5, characterized in that, The GPU resource scheduling device also includes a restart module, used for: Monitor the video memory usage of each process in the target container, and determine whether the monitored video memory usage exceeds the video memory requirement of the corresponding process. If the limit is exceeded, a preset restart policy is executed on the target container in the K8S cluster, and the node resources of each process on the corresponding GPU card are reallocated according to the memory usage.

7. A GPU resource scheduling device, characterized in that, The GPU resource scheduling device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the GPU resource scheduling device to perform the steps of the GPU resource scheduling method as described in any one of claims 1-4.

8. A computer-readable storage medium storing instructions thereon, characterized in that, When the instruction is executed by the processor, it implements the steps of the GPU resource scheduling method as described in any one of claims 1-4.

Images