Method, device and storage medium for creating a load implementation method by designating a GPU

Abstract

The application discloses a specified GPU creation load implementation method and device, equipment and storage medium, the method is through deploying gRPC transparent agent service between Kubelet and GPU device plug-in, and scheduling the target Pod to the specified node through the Volcano scheduler;When Kubelet detects the GPU resource requirement of the target Pod, sends Allocate gRPC request to gRPC transparent agent service;Based on gRPC transparent agent service, the modified Allocate gRPC request is sent to the specified target GPU device plug-in, and the specified GPU device is bound to the target Pod, so that the load runs on the specified GPU device, which can realize accurate control and allocation of GPU resources, has compatibility, scalability and flexibility, improves resource utilization efficiency and user experience, and improves the speed and efficiency of GPU resource management.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of resource management in containerization platforms, and in particular to a method and device for creating a load on a specified GPU, as well as an equipment and a storage medium. BACKGROUND

[0002] The GPU device plugin of Nvidia is a typical implementation of the Kubernetes device plugin mechanism. It creates a UNIX socket under the host path / var / lib / kubelet / device-plugins / and starts a gRPC service to communicate with kubelet. The plugin can register GPU devices on the node to Kubernetes and respond to allocation requests, thereby supporting containers using GPU resources.

[0003] gRPC is a high-performance RPC framework based on HTTP / 2 protocol, widely used in microservice architecture for communication between services. In microservice systems, the Proxy pattern is a common design pattern that intercepts and processes requests between services to achieve load balancing, authentication, monitoring, and other functions. The advantage of gRPC Proxy is its transparency, which can enhance service functions without modifying existing service code.

[0004] In the device plugin scenario of Kubernetes, gRPC communication is the core interaction between kubelet and device plugins. Therefore, by using the characteristics of gRPC Proxy, an intermediate layer can be inserted on the existing communication link to extend the function without affecting the original implementation of the plugin.

[0005] The existing GPU DevicePlugin technology is used in the Kubernetes ecosystem to manage GPU resources, but its design and implementation have the following significant shortcomings:

[0006] 1. No support for specifying specific GPUs

[0007] Users cannot specify the use of certain specific GPU cards on a node when creating a load (such as a Pod). The existing plugin only provides availability information for GPU resources, but does not have the ability to select specific GPUs based on user needs. Users cannot predict or control the allocated GPU, which may result in inconsistent task performance or the inability to utilize the unique features of certain specific GPUs (such as higher computing power or support for specific versions).

[0008] 2. Randomness of allocation

[0009] In the process of allocating requests, the kubelet randomly selects a batch of devices from the GPU list provided by the device plugin, rather than allocating according to the specific needs of the user; this leads to a lack of flexibility in resource utilization, especially in scenarios that require fine control of GPU resources (such as multi-task parallel training or workloads optimized for specific hardware), where users cannot fully exploit the potential of the hardware.

[0010] 3. Lack of scalability and flexibility

[0011] The design of existing plugins is relatively fixed and difficult to quickly adapt to new GPU hardware or different resource management strategies; as GPU technology rapidly develops, plugins may not be able to support new features or optimizations in a timely manner, leading to technical lag and limiting their application in diverse scenarios. SUMMARY

[0012] The main purpose of the present application is to provide a method, device, and storage medium for creating a load on a specified GPU, aiming to solve the technical problems of existing GPU resource management not supporting the designation of specific GPUs, having random allocation, lacking flexibility in resource utilization, and existing plugins being unable to support new features or optimizations in a timely manner.

[0013] In a first aspect, the present application provides a method for creating a load on a specified GPU, which includes the following steps:

[0014] A gRPC transparent proxy service is deployed between Kubelet and GPU device plugins, and a target Pod is scheduled to a specified node through a Volcano scheduler;

[0015] When the Kubelet detects the GPU resource needs of the target Pod, it sends an Allocate gRPC request to the gRPC transparent proxy service;

[0016] Based on the gRPC transparent proxy service, the modified Allocate gRPC request is sent to the specified target GPU device plugin, and the specified GPU device is bound to the target Pod, so that the load runs on the specified GPU device.

[0017] Optionally, the gRPC transparent proxy service is deployed between Kubelet and GPU device plugins, and a target Pod is scheduled to a specified node through a Volcano scheduler, including:

[0018] accessing a gRPC transparent proxy service between the Kubelet and the GPU device plugin, intercepting and processing communication between the Kubelet and the GPU device plugin through the gRPC transparent proxy service, the gRPC transparent proxy service communicating with the Kubelet and the GPU device plugin through a UNIX socket;

[0019] adding a preset scheduling plugin in the Volcano scheduler, and scheduling a target Pod to a specified node according to the preset scheduling plugin.

[0020] Optionally, the adding of the preset scheduling plugin in the Volcano scheduler and the scheduling of the target Pod to the specified node according to the preset scheduling plugin comprises:

[0021] adding a preset scheduling plugin in the Volcano scheduler, and adding a scheduling label in metadata of the Pod when detecting a user-submitted Pod;

[0022] filtering out the specified node according to a GPU number specified by the scheduling label in a Predicates stage of Kubernetes scheduling;

[0023] scheduling the Pod to the specified node through the preset scheduling plugin, and marking the Pod as being scheduled.

[0024] Optionally, the sending of the Allocate gRPC request to the gRPC transparent proxy service when the Kubelet detects the GPU resource requirement of the target Pod comprises:

[0025] performing GUP device registration when detecting that a GPU device plugin is started;

[0026] sending a ListAndWatch request to the gRPC transparent proxy service when the Kubelet detects the GPU resource requirement of the target Pod;

[0027] receiving the ListAndWatch request through the gRPC transparent proxy service, obtaining available GPU devices, forwarding device information of the available GPU devices to the GPU device plugin, and caching the device information of the available GPU devices;

[0028] selecting a GPU device from the available GPU devices randomly through the Kubelet, and sending an Allocate gRPC request to the gRPC transparent proxy service.

[0029] Optionally, the performing of the GUP device registration when detecting that the GPU device plugin is started comprises:

[0030] Upon detecting the GPU device plugin startup, register the available GPU device information to the gRPC transparent proxy service through a UNIX socket;

[0031] Cache the available GPU device information through the gRPC transparent proxy service, forward the registration request to the Kubelet, and simulate the original device plugin behavior.

[0032] Optionally, based on the gRPC transparent proxy service, the modified Allocate gRPC request is sent to the specified target GPU device plugin, and the specified GPU device is bound to the target Pod, so that the load runs on the specified GPU device. Before the specified GPU device is created, the load implementation method further includes:

[0033] Find the nearest scheduled Pod in the marked scheduling queue according to the FIFO principle through the gRPC transparent proxy service;

[0034] Extract the specified GPU number from the metadata of the scheduled Pod, and convert the specified GPU number to the corresponding device ID through the gRPC transparent proxy service;

[0035] According to the device ID, the random device ID in the Allocate gRPC request is covered to obtain the modified Allocate gRPC request, and the device ID is forwarded to the GPU device plugin.

[0036] Optionally, based on the gRPC transparent proxy service, the modified Allocate gRPC request is sent to the specified target GPU device plugin, and the specified GPU device is bound to the target Pod, so that the load runs on the specified GPU device. Including:

[0037] Based on the gRPC transparent proxy service, the modified Allocate gRPC request is sent to the specified target GPU device plugin, and the response information returned by the target GPU device plugin processing the modified Allocate gRPC request is received;

[0038] The response information is returned to the Kubelet through the gRPC transparent proxy service;

[0039] The specified GPU device of the specified node is bound to the target Pod through the Kubelet, so that the load runs on the specified GPU device.

[0040] In a second aspect, to achieve the above object, the application further provides a designated GPU creation load implementation device, which comprises:

[0041] A deployment scheduling module is configured to deploy a gRPC transparent proxy service between the Kubelet and the GPU device plugin, and schedule a target Pod to a designated node through a Volcano scheduler;

[0042] A request sending module is configured to send an Allocate gRPC request to the gRPC transparent proxy service when the Kubelet detects GPU resource requirements of the target Pod;

[0043] A device binding module is configured to send the modified Allocate gRPC request to a designated target GPU device plugin based on the gRPC transparent proxy service, bind the designated GPU device to the target Pod, and run the load on the designated GPU device.

[0044] In a third aspect, to achieve the above object, the application further provides a designated GPU creation load implementation device, which comprises a memory, a processor, and a designated GPU creation load implementation program stored in the memory and executable on the processor, wherein the designated GPU creation load implementation program is configured to implement the steps of the designated GPU creation load implementation method as described above.

[0045] In a fourth aspect, to achieve the above object, the application further provides a storage medium having a designated GPU creation load implementation program stored thereon, wherein the designated GPU creation load implementation program is executable by a processor to implement the steps of the designated GPU creation load implementation method as described above.

[0046] The application provides a method for creating a load by specifying a GPU, which comprises the following steps: deploying a gRPC transparent proxy service between a Kubelet and a GPU device plug-in, scheduling a target Pod to a specified node through a Volcano scheduler; when the Kubelet detects a GPU resource requirement of the target Pod, sending an Allocate gRPC request to the gRPC transparent proxy service; and based on the gRPC transparent proxy service, sending a modified Allocate gRPC request to a specified target GPU device plug-in to bind a specified GPU device to the target Pod, so that the load runs on the specified GPU device, which can realize accurate control and allocation of GPU resources, support users to create a load by specifying a specific GPU device in a containerized platform, and has compatibility, scalability and flexibility, the non-invasive design ensures compatibility with an existing Kubernetes architecture, the flexibility makes it applicable to various GPU hardware environments, which not only improves resource utilization efficiency and user experience, but also provides an efficient and flexible solution for GPU resource management, has significant technical advantages and application value, and improves the speed and efficiency of GPU resource management. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A device structure schematic diagram of a hardware running environment involved in an embodiment of the application is shown in the figure.

[0048] Figure 2 A flowchart of a first embodiment of the method for creating a load by specifying a GPU is shown in the figure.

[0049] Figure 3 A flowchart of a second embodiment of the method for creating a load by specifying a GPU is shown in the figure.

[0050] Figure 4 A flowchart of a third embodiment of the method for creating a load by specifying a GPU is shown in the figure.

[0051] Figure 5 A flowchart of a fourth embodiment of the method for creating a load by specifying a GPU is shown in the figure.

[0052] Figure 6 A flowchart of a fifth embodiment of the method for creating a load by specifying a GPU is shown in the figure.

[0053] Figure 7 A functional module diagram of a first embodiment of the device for creating a load by specifying a GPU is shown in the figure.

[0054] The object, function features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0056] The solution of the embodiment of the present application is mainly: a gRPC transparent proxy service is deployed between Kubelet and a GPU device plugin, a target Pod is scheduled to a specified node through a Volcano scheduler; when the Kubelet detects a GPU resource requirement of the target Pod, an Allocate gRPC request is sent to the gRPC transparent proxy service; based on the gRPC transparent proxy service, a modified Allocate gRPC request is sent to a specified target GPU device plugin, a specified GPU device is bound to the target Pod, and load is run on the specified GPU device, so that accurate control and allocation of GPU resources can be achieved, a user can specify a specific GPU device to create load in a containerized platform, compatibility, scalability and flexibility are achieved, the non-invasive design ensures compatibility with the existing Kubernetes architecture, flexibility makes it suitable for various GPU hardware environments, not only improves resource utilization efficiency and user experience, but also provides an efficient and flexible solution for GPU resource management, has significant technical advantages and application value, improves the speed and efficiency of GPU resource management, and solves the technical problems that the existing technology does not support specifying a specific GPU, has randomness in allocation, resource utilization lacks flexibility, and existing plugins cannot support new functions or optimization in time.

[0057] Reference Figure 1 , Figure 1 The device structure diagram of the hardware running environment involved in the embodiment of the present application is shown.

[0058] As Figure 1 shown, the device can include a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen (Display) and an input unit such as a keyboard (Keyboard). The optional user interface 1003 can also include a standard wired interface and a wireless interface. The network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 can be a high-speed RAM memory or a stable memory (Non-Volatile Memory), such as a magnetic disk memory. The memory 1005 can also be an independent storage device from the aforementioned processor 1001.

[0059] Those skilled in the art can understand, Figure 1The device structure shown in the figure does not constitute a limitation on the device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0060] As shown in Figure 1 The memory 1005 as a storage medium can include an operation device, a network communication module, a user interface module, and a designated GPU creation load implementation program.

[0061] The device of the application calls the designated GPU creation load implementation program stored in the memory 1005 through the processor 1001, and performs the following operations:

[0062] Deploy a gRPC transparent proxy service between Kubelet and GPU device plugins, and schedule target Pods to designated nodes through the Volcano scheduler;

[0063] When the Kubelet detects the GPU resource requirement of the target Pod, an Allocate gRPC request is sent to the gRPC transparent proxy service;

[0064] Based on the gRPC transparent proxy service, the modified Allocate gRPC request is sent to the designated target GPU device plugin, and the designated GPU device is bound to the target Pod, so that the load runs on the designated GPU device.

[0065] The device of the application calls the designated GPU creation load implementation program stored in the memory 1005 through the processor 1001, and performs the following operations:

[0066] Access a gRPC transparent proxy service between Kubelet and GPU device plugins, intercept and process communication between the Kubelet and the GPU device plugin through the gRPC transparent proxy service, and the gRPC transparent proxy service communicates with the Kubelet and the GPU device plugin through a UNIX socket.

[0067] Add a preset scheduling plugin in the Volcano scheduler, and schedule target Pods to designated nodes according to the preset scheduling plugin.

[0068] The device of the application calls the designated GPU creation load implementation program stored in the memory 1005 through the processor 1001, and performs the following operations:

[0069] Add a preset scheduling plugin in the Volcano scheduler, and add a scheduling label in the metadata of the Pod when detecting a user-submitted Pod.

[0070] Filtering out the specified node according to the GPU number specified by the scheduling label in the Predicates stage of Kubernetes scheduling;

[0071] Scheduling the Pod to the specified node through the preset scheduling plug-in, and marking the Pod as scheduling.

[0072] The device of the application calls the specified GPU creation load implementation program stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0073] When detecting that the GPU device plug-in is started, registering the GUP device;

[0074] When the Kubelet detects the GPU resource requirement of the target Pod, sending a ListAndWatch request to the gRPC transparent proxy service;

[0075] Receiving the ListAndWatch request through the gRPC transparent proxy service, obtaining available GPU devices, forwarding the device information of the available GPU devices to the GPU device plug-in and caching the device information of the available GPU devices;

[0076] Selecting a GPU device from the available GPU devices through the Kubelet, and sending an Allocate gRPC request to the gRPC transparent proxy service.

[0077] The device of the application calls the specified GPU creation load implementation program stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0078] When detecting that the GPU device plug-in is started, registering the available GPU device information to the gRPC transparent proxy service through a UNIX socket;

[0079] Caching the available GPU device information through the gRPC transparent proxy service, forwarding the registration request to the Kubelet, and simulating the original device plug-in behavior.

[0080] The device of the application calls the specified GPU creation load implementation program stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0081] Finding the nearest Pod to be scheduled in the marked scheduling queue according to the FIFO principle through the gRPC transparent proxy service;

[0082] Extracting the specified GPU number from the metadata of the Pod to be scheduled, and converting the specified GPU number to the corresponding device ID through the gRPC transparent proxy service;

[0083] According to the device ID, the random device ID in the Allocate gRPC request is covered, a modified Allocate gRPC request is obtained, and the device ID is forwarded to the GPU device plugin.

[0084] The device of the application calls the specified GPU creation load implementation program stored in the memory 1005 through the processor 1001, and also performs the following operations:

[0085] Based on the gRPC transparent proxy service, the modified Allocate gRPC request is issued to the specified target GPU device plugin, and response information returned by the target GPU device plugin processing the modified Allocate gRPC request is received;

[0086] The response information is returned to the Kubelet through the gRPC transparent proxy service;

[0087] The specified GPU device of the specified node is bound to the target Pod through the Kubelet, so that the load runs on the specified GPU device.

[0088] Through the above scheme, the gRPC transparent proxy service is deployed between the Kubelet and the GPU device plugin, the target Pod is scheduled to the specified node through the Volcano scheduler; when the Kubelet detects the GPU resource requirement of the target Pod, the Allocate gRPC request is sent to the gRPC transparent proxy service; based on the gRPC transparent proxy service, the modified Allocate gRPC request is issued to the specified target GPU device plugin, and the specified GPU device is bound to the target Pod, so that the load runs on the specified GPU device, which can realize accurate control and allocation of GPU resources, support users to specify specific GPU devices to create loads in the containerization platform, and has compatibility, scalability and flexibility. Its non-invasive design ensures compatibility with the existing Kubernetes architecture, flexibility makes it suitable for a variety of GPU hardware environments, not only improves resource utilization efficiency and user experience, but also provides an efficient and flexible solution for GPU resource management, has significant technical advantages and application value, and improves the speed and efficiency of GPU resource management.

[0089] Based on the above hardware structure, the application proposes an embodiment of a method for creating a load on a specified GPU.

[0090] Reference Figure 2 , Figure 2 The flowchart of the first embodiment of the application is shown.

[0091] In the first embodiment, the method for creating a load on a specified GPU comprises the following steps:

[0092] Step S10, deploying a gRPC transparent proxy service between Kubelet and GPU device plugin, scheduling the target Pod to the specified node through Volcano scheduler.

[0093] It should be noted that in the containerization platform, Kubernetes is a widely used open source container orchestration system for automating deployment, expansion and management of containerized applications; Kubernetes supports management of hardware resources such as GPU, FPGA, etc. through its device plugin (DevicePlugin) mechanism; these hardware devices are particularly important in high-performance computing scenarios (such as deep learning training, scientific computing, etc.); the device plugin mechanism of Kubernetes allows developers to develop plugins for specific hardware devices, enabling these devices to be recognized, managed and allocated by the Kubernetes cluster; the device plugin communicates with Kubelet (a proxy component running on each node) through Remote Procedure Call (gRPC), a high-performance remote procedure call framework; deploying a gRPC transparent proxy service between Kubelet and GPU device plugin can support users to create a load on a specified GPU, and Volcano scheduler supports plugin-based scheduling, which can schedule the target Pod to the specified node through Volcano scheduler.

[0094] Step S20, when the Kubelet detects the GPU resource requirement of the target Pod, sending an Allocate gRPC request to the gRPC transparent proxy service.

[0095] It should be understood that when the Kubelet detects the GPU resource requirement of the target Pod, an Allocate gRPC request can be sent to the gRPC transparent proxy service, the Allocate gRPC request being an allocation request, and the Allocate gRPC request containing a randomly generated device ID.

[0096] Step S30, based on the gRPC transparent proxy service, issuing the modified Allocate gRPC request to the specified target GPU device plugin, binding the specified GPU device to the target Pod, so that the load runs on the specified GPU device.

[0097] It can be understood that, according to the gRPC transparent proxy service, the modified Allocate gRPC request can be sent to the specified target GPU device plugin, the specified GPU device can be bound to the target Pod, and the load can run on the specified GPU device.

[0098] The embodiment can realize accurate control and allocation of GPU resources, support users to specify a specific GPU device to create a load in a containerized platform, have compatibility, scalability and flexibility, and ensure compatibility with the existing Kubernetes architecture through the non-invasive design, and the flexibility makes it suitable for various GPU hardware environments, which not only improves resource utilization efficiency and user experience, but also provides an efficient and flexible solution for GPU resource management, has significant technical advantages and application value, and improves the speed and efficiency of GPU resource management.

[0099] Further, Figure 3 A flowchart of a method for creating a load by specifying a GPU according to the second embodiment of the application is shown in Figure 3 As shown in the first embodiment, the second embodiment of the method for creating a load by specifying a GPU according to the application is proposed, and in the embodiment, the step S10 specifically includes the following steps:

[0100] Step S11, a gRPC transparent proxy service is connected between Kubelet and GPU device plugin, the gRPC transparent proxy service is used to intercept and process the communication between the Kubelet and the GPU device plugin, and the gRPC transparent proxy service communicates with the Kubelet and the GPU device plugin through a UNIX socket.

[0101] It should be noted that a gRPC transparent proxy service is connected between Kubelet and GPU device plugin, and the gRPC communication between the two is intercepted and processed through the proxy gRPC service, and the proxy service is started through a UNIX socket, which can maintain compatibility with the existing k8s architecture, realize fine management and scheduling of GPU resources, and does not need to rely on the random allocation of Kubelet.

[0102] It can be understood that introducing a proxy service between the Kubelet and the device plugin, intercepting and modifying the gRPC communication, can realize custom scheduling logic.

[0103] In a specific implementation, by introducing a gRPC transparent proxy service between the Kubelet and the NVIDIA GPU DevicePlugin, support is provided for users to specify the creation of a load of a specific GPU on a certain node, compatibility with the existing K8sDevicePlugin framework is maintained, no invasive modification of the original code is required, flexibility and universality are provided, and adaptation to NVIDIA and other GPUs (such as domestic GPUs) is provided.

[0104] Step S12, a preset scheduling plugin is added in the Volcano scheduler, and the target Pod is scheduled to a specified node according to the preset scheduling plugin.

[0105] It can be understood that adding a custom preset scheduling plugin in the Volcano scheduler can schedule the target Pod to a specified node according to the preset scheduling plugin.

[0106] It should be noted that the GPU device plugin is a GPU manufacturer code, which is adapted to respective GPU devices, provides device information and allocation capabilities, and registers / communicates services with the Plugin proxy through a UNIX socket; from the perspective of the GPU device plugin Device Plugin, the Plugin proxy is the Kubelet; the Plugin proxy is a newly added gRPC transparent proxy service, which can intercept the gRPC communication between the Kubelet and the Device Plugin, and register / communicate services with the Kubelet through a UNIX socket; from the perspective of the Kubelet, the Plugin proxy is the Device Plugin.

[0107] Further, the step S12 specifically includes the following steps:

[0108] A preset scheduling plugin is added in the Volcano scheduler, and a scheduling label is added in the metadata of the Pod when a user-submitted Pod is detected;

[0109] The specified node is filtered out according to the GPU number specified by the scheduling label in the Predicates stage of Kubernetes scheduling;

[0110] The Pod is scheduled to the specified node through the preset scheduling plugin, and the Pod is marked as being scheduled.

[0111] It should be noted that all Pods that need to use GPU resources are uniformly scheduled through the Volcano scheduler in combination with the adapted k8sdeviceplugin; Volcano is a commonly used batch job scheduling framework on Kubernetes, which is good at scheduling complex resource-intensive tasks (such as big data analysis, deep learning, etc.); it has more flexible scheduling extension capabilities and supports plug-in scheduling decisions.

[0112] It should be understood that when the user submits the Pod, the scheduling label is added in the metadata, and the target GPU number is specified. Volcano schedules the Pod to the specified node according to the scheduling information in the metadata of the Pod through the extended custom scheduling plug-in, and marks the Pod as scheduling.

[0113] In a specific implementation, a custom plug-in can be added in the Volcano scheduler to filter the Pod in the Predicates stage of Kubernetes scheduling; the Predicates stage is an important step in the Kubernetes scheduling decision, which is responsible for filtering nodes that meet certain conditions.

[0114] It should be noted that the user can specify a specific GPU device (for example, a certain number of cards on a certain node) when creating a pod, instead of relying on the default random selection of Kubelet, and through the scheduling logic of the proxy layer, the precise demand of the specific workload for the GPU resource is met.

[0115] The above scheme is used in the embodiment. The gRPC transparent proxy service is connected between the Kubelet and the GPU device plug-in, the communication between the Kubelet and the GPU device plug-in is intercepted and processed by the gRPC transparent proxy service, the gRPC transparent proxy service communicates with the Kubelet and the GPU device plug-in through a UNIX socket, a preset scheduling plug-in is added in the Volcano scheduler, and the target Pod is scheduled to a specified node according to the preset scheduling plug-in. Precise control and allocation of GPU resources can be achieved, the user can specify a specific GPU device to create a load in a containerized platform, compatibility, scalability and flexibility are achieved, and the speed and efficiency of GPU resource management are improved.

[0116] Further, Figure 4 The flowchart of the method for creating a load by specifying a GPU according to the third embodiment of the application is shown in Figure 4 As shown in the first embodiment, the step S20 specifically includes the following steps:

[0117] Step S21, when detecting the GPU device plugin startup, proceed with GUP device registration.

[0118] It should be noted that when detecting the GPU device plugin startup, device registration can be performed, i.e. GUP device registration.

[0119] Further, the step S21 specifically includes the following steps:

[0120] When detecting the GPU device plugin startup, register the available GPU device information to the gRPC transparent proxy service through a UNIX socket;

[0121] Cache the available GPU device information through the gRPC transparent proxy service, forward the registration request to the Kubelet, and simulate the original device plugin behavior.

[0122] In a specific implementation, when the GPU Device Plugin starts, i.e. the GPU device plugin starts, the available GPU device information can be registered to the agent through a UNIX socket; the agent receives and caches these information, and forwards the registration request to the Kubelet, simulating the original Device Plugin device plugin behavior.

[0123] Step S22, when the Kubelet detects the GPU resource requirement of the target Pod, sends a ListAndWatch request to the gRPC transparent proxy service.

[0124] It can be understood that when the container requests resources, the kubelet initiates an allocation request to the device plugin through Allocate gRPC, and the device plugin is responsible for allocating specific devices to the container. The Kubelet detects the GPU resource requirement and sends a ListAndWatch request to the agent to obtain available devices. The Kubelet, i.e. in-tree code, is a Kubernetes node agent responsible for Pod life cycle management and resource allocation. When the Kubelet detects the GPU resource requirement of the target Pod, it can send a ListAndWatch request to the gRPC transparent proxy service. Through the ListAndWatch request, the available device list and its status on the current node can be viewed.

[0125] Step S23, receive the ListAndWatch request through the gRPC transparent proxy service, obtain the available GPU device, forward and cache the device information of the available GPU device to the GPU device plugin.

[0126] It should be understood that the proxy service receives the ListAndWatch request of the Kubelet, forwards and caches the device information to the GPU device plugin, receives the ListAndWatch request through the gRPC transparent proxy service, obtains available GPU devices, forwards and caches the device information of the available GPU devices to the GPU device plugin.

[0127] Step S24, randomly selecting a GPU device from the available GPU devices by the Kubelet, sending an Allocate gRPC request to the gRPC transparent proxy service.

[0128] It can be understood that the Kubelet randomly selects a GPU device, sends an Allocate gRPC request to the proxy, and contains a randomly generated Device ID, i.e., a device ID.

[0129] The above scheme, when the GPU device plugin is detected to start, the GUP device is registered; when the Kubelet detects the GPU resource demand of the target Pod, sends a ListAndWatch request to the gRPC transparent proxy service; through the gRPC transparent proxy service, the ListAndWatch request is received, the available GPU devices are obtained, and the device information of the available GPU devices is forwarded and cached to the GPU device plugin; through the Kubelet, a GPU device is randomly selected from the available GPU devices, and an Allocate gRPC request is sent to the gRPC transparent proxy service; can realize accurate control and allocation of GPU resources, support users to specify specific GPU devices to create loads in the containerization platform, have compatibility, scalability and flexibility, the non-invasive design ensures the compatibility with the existing Kubernetes architecture, the flexibility makes it suitable for a variety of GPU hardware environments, not only improves the resource utilization efficiency and user experience, but also provides an efficient and flexible solution for GPU resource management.

[0130] Further, Figure 5 The flowchart of the fourth embodiment of the method for creating a load by specifying a GPU is shown in FIG. 8. Figure 5 As shown in FIG. 8, the fourth embodiment of the method for creating a load by specifying a GPU is proposed based on the first embodiment, and before step S30, the method for creating a load by specifying a GPU further includes the following steps:

[0131] Step S301, finding the nearest Pod to be scheduled in the marked scheduling queue according to the FIFO principle through the gRPC transparent proxy service.

[0132] It should be noted that the agent finds the nearest to-be-scheduled Pod in the queue marked as scheduling according to the first-in first-out (FIFO) principle, that is, the nearest to-be-scheduled Pod in the marked scheduling queue can be found according to the FIFO principle through the gRPC transparent agent service.

[0133] Step S302, extracting the specified GPU number from the metadata of the to-be-scheduled Pod, and converting the specified GPU number into a corresponding device ID through the gRPC transparent agent service.

[0134] It can be understood that the agent extracts the specified GPU number from the metadata thereof and converts the GPU number into a corresponding device ID, that is, the specified GPU number is extracted from the metadata of the to-be-scheduled Pod, and the specified GPU number is converted into a corresponding device ID through the gRPC transparent agent service, the user-specified GPU number is extracted from the metadata of the Pod, and the device ID is converted, which can cover the original random allocation logic, and the function is enhanced through the agent layer without modifying the existing device plugin code.

[0135] Step S303, covering the random device ID in the Allocate gRPC request according to the device ID, obtaining a modified Allocate gRPC request, and forwarding the device ID to the GPU device plugin.

[0136] It should be understood that the agent service intercepts the Allocate request of Kubelet, extracts the GPU number in the metadata of the Pod, and forwards it to the GPU device plugin after covering the original Device ID, that is, after covering the random device ID in the Allocate gRPC request according to the device ID, a modified Allocate gRPC request can be obtained, and the device ID can be forwarded to the GPU device plugin.

[0137] In a specific implementation, the proxy gRPC service is responsible for registering GPU device information to the Kubelet, ensuring that the Kubelet can identify available GPU resources; when a pod applies for GPU resources, the Kubelet issues an allocate gRPC request, and the proxy receives the request and processes it according to preset logic: according to the pre-set scheduling label and FIFO (First In First Out) principle, the current pod to be scheduled is found; the user-specified GPU number is obtained from the metadata of the pod; the specified GPU number is converted into the corresponding device ID, which replaces the randomly selected device ID in the Allocate request; the modified request is sent to the downstream GPU device plugin to complete resource allocation.

[0138] According to the above scheme, the gRPC transparent proxy service finds the nearest pod to be scheduled in the marked scheduling queue according to the FIFO principle; the specified GPU number is extracted from the metadata of the pod to be scheduled, and the gRPC transparent proxy service converts the specified GPU number into the corresponding device ID; the device ID is used to replace the random device ID in the Allocate gRPC request, and the modified Allocate gRPC request is obtained, and the device ID is forwarded to the GPU device plugin; accurate control and allocation of GPU resources can be achieved, and the speed and efficiency of the specified GPU creation load implementation are improved.

[0139] Further, Figure 6 The flowchart of the fifth embodiment of the method for creating a load by specifying a GPU according to the present application is shown in Figure 6 Based on the first embodiment, the fifth embodiment of the method for creating a load by specifying a GPU according to the present application is proposed, and in this embodiment, the step S30 specifically includes the following steps:

[0140] Step S31, based on the gRPC transparent proxy service, the modified Allocate gRPC request is sent to the specified target GPU device plugin, and the response information returned by the target GPU device plugin processing the modified Allocate gRPC request is received.

[0141] It should be noted that the proxy sends the modified Allocate request to the GPU Device Plugin; the DevicePlugin processes the request and returns a response, that is, based on the gRPC transparent proxy service, the modified Allocate gRPC request is sent to the specified target GPU device plugin, and the response information returned by the target GPU device plugin processing the modified Allocate gRPC request is received.

[0142] Step S32, return the response information to the Kubelet through the gRPC transparent proxy service.

[0143] It can be understood that the gRPC transparent proxy service transparently returns the response to the Kubelet after receiving the response of the device plugin.

[0144] Step S33, bind the specified GPU device of the specified node to the target Pod through the Kubelet, so that the load runs on the specified GPU device.

[0145] It should be understood that the Kubelet binds the specified GPU to the Pod according to the response returned by the proxy, ensuring that the load runs on the target GPU, that is, the specified GPU device of the specified node is bound to the target Pod through the Kubelet, so that the load runs on the specified GPU device, realizing resource binding.

[0146] In a specific implementation, the present embodiment does not need to modify the existing k8s GPU device plugin code, but only extends the function through the proxy layer; this design maintains the modularity, maintainability and compatibility with the original architecture of the system; not only suitable for Nvidia GPU device plugin, but also suitable for other types of GPU (such as domestic GPU), through the abstract design of the proxy layer, the applicability of the scheme in different hardware environments is enhanced, which can efficiently manage and utilize high-performance GPU resources in a containerized environment.

[0147] It can be understood that the present embodiment realizes accurate control and allocation of GPU resources through the gRPC transparent proxy mechanism, supports users to specify specific GPU devices to create loads in a containerized platform; its non-invasive design ensures compatibility with the existing Kubernetes architecture, flexibility makes it suitable for various GPU hardware environments; not only improves resource utilization efficiency and user experience, but also provides an efficient and flexible solution for GPU resource management, with significant technical advantages and application value.

[0148] The embodiment implements precise control and allocation of GPU resources by the above scheme, receives response information returned by the target GPU device plugin processing the modified Allocate gRPC request through the gRPC transparent proxy service, returns the response information to the Kubelet through the gRPC transparent proxy service, and binds the specified GPU device of the specified node to the target Pod through the Kubelet, so that the load runs on the specified GPU device, can realize precise control and allocation of GPU resources, supports users to specify a specific GPU device to create a load in a containerized platform, has compatibility, scalability and flexibility, the non-invasive design ensures compatibility with the existing Kubernetes architecture, flexibility makes it suitable for a variety of GPU hardware environments, not only improves resource utilization efficiency and user experience, but also provides an efficient and flexible solution for GPU resource management, has significant technical advantages and application value, and improves the speed and efficiency of GPU resource management.

[0149] Correspondingly, the application further provides a device for implementing load creation by a specified GPU.

[0150] Reference Figure 7 , Figure 7 The function module diagram of the device for implementing load creation by a specified GPU according to the first embodiment of the application is shown in FIG. 1.

[0151] In the device for implementing load creation by a specified GPU according to the first embodiment of the application, the device comprises:

[0152] A deployment scheduling module 10 is configured to deploy a gRPC transparent proxy service between the Kubelet and the GPU device plugin, and schedule a target Pod to a specified node through a Volcano scheduler.

[0153] A request sending module 20 is configured to send an Allocate gRPC request to the gRPC transparent proxy service when the Kubelet detects the GPU resource requirement of the target Pod.

[0154] A device binding module 30 is configured to send a modified Allocate gRPC request to a specified target GPU device plugin based on the gRPC transparent proxy service, bind a specified GPU device to the target Pod, and make the load run on the specified GPU device.

[0155] The deployment scheduling module 10 is further configured to access a gRPC transparent proxy service between the Kubelet and the GPU device plugin, intercept and process communication between the Kubelet and the GPU device plugin through the gRPC transparent proxy service, and add a preset scheduling plugin in the Volcano scheduler, and schedule a target Pod to a specified node according to the preset scheduling plugin.

[0156] The deployment scheduling module 10 is further configured to add a preset scheduling plugin in the Volcano scheduler, add a scheduling label in metadata of a Pod when detecting that the Pod is submitted by a user, filter a specified node according to a GPU number specified by the scheduling label in a Predicates stage of Kubernetes scheduling, schedule the Pod to the specified node through the preset scheduling plugin, and mark the Pod as scheduling.

[0157] The request sending module 20 is further configured to register a GPU device when detecting that the GPU device plugin is started, send a ListAndWatch request to the gRPC transparent proxy service when the Kubelet detects that a GPU resource requirement of the target Pod, receive the ListAndWatch request through the gRPC transparent proxy service, obtain available GPU devices, forward and cache device information of the available GPU devices to the GPU device plugin, and select a GPU device from the available GPU devices randomly through the Kubelet, and send an Allocate gRPC request to the gRPC transparent proxy service.

[0158] The request sending module 20 is further configured to register available GPU device information to the gRPC transparent proxy service through a UNIX socket when detecting that the GPU device plugin is started, cache the available GPU device information through the gRPC transparent proxy service, forward a registration request to the Kubelet, and simulate an original device plugin behavior.

[0159] The device binding module 30 is further configured to find the nearest to-be-scheduled Pod in a marked scheduling queue according to a FIFO principle through the gRPC transparent proxy service, extract a specified GPU number from metadata of the to-be-scheduled Pod, convert the specified GPU number into a corresponding device ID through the gRPC transparent proxy service, cover a random device ID in the Allocate gRPC request according to the device ID, obtain a modified Allocate gRPC request, and forward the device ID to the GPU device plugin.

[0160] The device binding module 30 is further configured to send the modified Allocate gRPC request to a specified target GPU device plugin based on the gRPC transparent proxy service, receive response information returned by the target GPU device plugin in response to the modified Allocate gRPC request, return the response information to the Kubelet through the gRPC transparent proxy service, and bind a specified GPU device of the specified node to the target Pod through the Kubelet, so that a load runs on the specified GPU device.

[0161] The steps implemented by the various functional modules of the specified GPU creation load implementation apparatus can refer to the various embodiments of the specified GPU creation load implementation method, and will not be described here.

[0162] In addition, an embodiment of the present application further provides a storage medium, and the storage medium stores a specified GPU creation load implementation program. When the specified GPU creation load implementation program is executed by a processor, the following operations are implemented:

[0163] A gRPC transparent proxy service is deployed between the Kubelet and the GPU device plugin, and a target Pod is scheduled to a specified node through a Volcano scheduler;

[0164] When the Kubelet detects a GPU resource requirement of the target Pod, an Allocate gRPC request is sent to the gRPC transparent proxy service;

[0165] The gRPC transparent proxy service sends the modified Allocate gRPC request to a specified target GPU device plugin, binds a specified GPU device to the target Pod, and makes a load run on the specified GPU device.

[0166] Further, when the specified GPU creation load implementation program is executed by the processor, the following operations are further implemented:

[0167] accessing a gRPC transparent proxy service between the Kubelet and the GPU device plugin, intercepting and processing communications between the Kubelet and the GPU device plugin through the gRPC transparent proxy service, the gRPC transparent proxy service communicating with the Kubelet and the GPU device plugin through a UNIX socket;

[0168] adding a preset scheduling plugin in the Volcano scheduler, and scheduling a target Pod to a specified node according to the preset scheduling plugin.

[0169] Further, when the specified GPU creation load implementation program is executed by the processor, the following operations are further implemented:

[0170] adding a preset scheduling plugin in the Volcano scheduler, and adding a scheduling label in metadata of the Pod when detecting a user-submitted Pod;

[0171] filtering out a specified node according to a GPU number specified by the scheduling label in a Predicates stage of Kubernetes scheduling;

[0172] scheduling the Pod to the specified node through the preset scheduling plugin, and marking the Pod as being scheduled.

[0173] Further, when the specified GPU creation load implementation program is executed by the processor, the following operations are further implemented:

[0174] detecting a GPU device plugin startup, and performing GUP device registration;

[0175] when the Kubelet detects a GPU resource requirement of the target Pod, sending a ListAndWatch request to the gRPC transparent proxy service;

[0176] receiving the ListAndWatch request through the gRPC transparent proxy service, obtaining available GPU devices, forwarding the device information of the available GPU devices to the GPU device plugin, and caching the device information of the available GPU devices;

[0177] selecting a GPU device from the available GPU devices through the Kubelet, and sending an Allocate gRPC request to the gRPC transparent proxy service.

[0178] Further, when the specified GPU creation load implementation program is executed by the processor, the following operations are further implemented:

[0179] detecting a GPU device plugin startup, and registering available GPU device information to the gRPC transparent proxy service through a UNIX socket;

[0180] The gRPC transparent proxy service caches the available GPU device information, forwards the registration request to the Kubelet, and simulates the original device plugin behavior.

[0181] Further, the specified GPU creation load implementation program is executed by the processor to further implement the following operations:

[0182] The gRPC transparent proxy service finds the nearest to-be-scheduled Pod in the marked scheduling queue according to the FIFO principle.

[0183] The specified GPU number is extracted from the metadata of the to-be-scheduled Pod, and the gRPC transparent proxy service converts the specified GPU number into a corresponding device ID.

[0184] The device ID is used to overwrite the random device ID in the Allocate gRPC request, to obtain a modified Allocate gRPC request, and the device ID is forwarded to the GPU device plugin.

[0185] Further, the specified GPU creation load implementation program is executed by the processor to further implement the following operations:

[0186] The gRPC transparent proxy service sends the modified Allocate gRPC request to the specified target GPU device plugin, and receives response information returned by the target GPU device plugin in response to the modified Allocate gRPC request.

[0187] The gRPC transparent proxy service returns the response information to the Kubelet.

[0188] The Kubelet binds the specified GPU device of the specified node to the target Pod, so that the load runs on the specified GPU device.

[0189] Those skilled in the art can understand that all or part of the steps in the above-mentioned implementation methods can be completed by a program instructing related hardware. The program is stored in a storage medium and includes a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium is a computer-readable storage medium, including a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0190] It should be noted that the terms "comprising", "including", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0191] The above-mentioned embodiment numbers of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0192] The above is only the preferred embodiment of the application, and does not limit the patent scope of the application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the application.

Claims

1. A method for implementing a specified GPU-created workload, characterized in that, The specified GPU load creation implementation method includes: Deploy a transparent gRPC proxy service between Kubelet and GPU device plugins, and use the Volcano scheduler to schedule the target Pod to the specified node; When the Kubelet detects the GPU resource requirement of the target Pod, it sends an Allocate gRPC request to the gRPC transparent proxy service; Based on the gRPC transparent proxy service, the modified Allocate gRPC request is sent to the specified target GPU device plugin, and the specified GPU device is bound to the target Pod so that the workload runs on the specified GPU device; The method for creating a load on the specified GPU, which involves sending the modified Allocate gRPC request to the specified target GPU device plugin based on the gRPC transparent proxy service and binding the specified GPU device to the target Pod to ensure that the load runs on the specified GPU device, further includes: The gRPC transparent proxy service finds the nearest Pod to be scheduled in the marked scheduling queue according to the FIFO principle. Extract the specified GPU ID from the metadata of the Pod to be scheduled, and convert the specified GPU ID into the corresponding device ID through the gRPC transparent proxy service; The modified Allocate gRPC request is obtained by overwriting the random device ID in the Allocate gRPC request with the device ID, and the device ID is forwarded to the GPU device plugin.

2. The method for creating a load using a specified GPU as described in claim 1, characterized in that, The deployment of a gRPC transparent proxy service between the Kubelet and the GPU device plugin, and the scheduling of the target Pod to a specified node via the Volcano scheduler, includes: A gRPC transparent proxy service is connected between the Kubelet and the GPU device plugin. The gRPC transparent proxy service intercepts and processes the communication between the Kubelet and the GPU device plugin. The gRPC transparent proxy service communicates with the Kubelet and the GPU device plugin through UNIX sockets. Add a preset scheduling plugin to the Volcano scheduler, and schedule the target Pod to the specified node according to the preset scheduling plugin.

3. The method for creating a load using a specified GPU as described in claim 2, characterized in that, The step of adding a preset scheduling plugin to the Volcano scheduler and scheduling the target Pod to a specified node according to the preset scheduling plugin includes: Add a new preset scheduling plugin to the Volcano scheduler, which adds a scheduling tag to the metadata of a user-submitted Pod when a Pod is detected. During the Predicates phase of Kubernetes scheduling, the specified nodes are selected based on the GPU ID specified by the scheduling label; The Pod is scheduled to the specified node using the preset scheduling plugin, and the Pod is marked as being scheduled.

4. The method for creating a load using a specified GPU as described in claim 1, characterized in that, When the Kubelet detects the GPU resource requirements of the target Pod, it sends an Allocate gRPC request to the gRPC transparent proxy service, including: When the GPU device plugin is detected to be starting up, GUP device registration is performed; When the Kubelet detects the GPU resource requirement of the target Pod, it sends a ListAndWatch request to the gRPC transparent proxy service; The gRPC transparent proxy service receives the ListAndWatch request, obtains available GPU devices, forwards and caches the device information of the available GPU devices to the GPU device plugin; The Kubelet randomly selects a GPU device from the available GPU devices and sends an Allocate gRPC request to the gRPC transparent proxy service.

5. The method for creating a load using a specified GPU as described in claim 4, characterized in that, The step of registering the GPU device upon detecting the startup of the GPU device plugin includes: When a GPU device plugin is detected to be starting up, the available GPU device information is registered with the gRPC transparent proxy service via a UNIX socket; The available GPU device information is cached through the gRPC transparent proxy service, the registration request is forwarded to the Kubelet, and the behavior of the original device plugin is simulated.

6. The method for creating a load using a specified GPU as described in claim 1, characterized in that, The step of sending the modified Allocate gRPC request to the specified target GPU device plugin based on the gRPC transparent proxy service, and binding the specified GPU device to the target Pod so that the workload runs on the specified GPU device, includes: Based on the gRPC transparent proxy service, the modified Allocate gRPC request is sent to the specified target GPU device plugin, and the response information returned by the target GPU device plugin after processing the modified Allocate gRPC request is received; The response information is returned to the Kubelet via the gRPC transparent proxy service; The Kubelet binds the specified GPU device of the specified node to the target Pod, so that the load runs on the specified GPU device.

7. A device for implementing a specified GPU creation workload, characterized in that, The specified GPU-based workload creation implementation device includes: The deployment and scheduling module is used to deploy a transparent gRPC proxy service between Kubelet and GPU device plugins, and to schedule target Pods to specified nodes through the Volcano scheduler; The request sending module is used to send an Allocate gRPC request to the gRPC transparent proxy service when the Kubelet detects the GPU resource requirements of the target Pod. The device binding module is used to send the modified Allocate gRPC request to the specified target GPU device plugin based on the gRPC transparent proxy service, and bind the specified GPU device to the target Pod so that the load runs on the specified GPU device. The device binding module is further configured to find the nearest scheduled Pod in the marked scheduling queue according to the FIFO principle through the gRPC transparent proxy service; extract a specified GPU number from the metadata of the scheduled Pod, convert the specified GPU number into a corresponding device ID through the gRPC transparent proxy service; obtain a modified Allocate gRPC request by overwriting the random device ID in the Allocate gRPC request with the device ID, and forward the device ID to the GPU device plugin.

8. A device for implementing a specified GPU-created workload, characterized in that, The designated GPU creation workload implementation device includes: a memory, a processor, and a designated GPU creation workload implementation program stored on the memory and executable on the processor, wherein the designated GPU creation workload implementation program is configured to implement the steps of the designated GPU creation workload implementation method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a specified GPU creation load implementation program, which, when executed by a processor, implements the steps of the specified GPU creation load implementation method as described in any one of claims 1 to 6.

Images