Real load based gpu scheduling method, system, device and storage medium

Abstract

The application relates to the technical field of resource scheduling management, in particular to a GPU scheduling method and system based on real load, equipment and a storage medium, the method comprises the following steps: acquiring total equipment quantity information on a GPU server and registering the total equipment quantity information to a Kubernetes cluster; in response to a GPU resource scheduling request of a service container, a Kubernetes scheduler acquires current usage of the GPU server on each node, selects a node with the largest real-time margin, and binds the service container with all GPU equipment on the GPU server under the selected node; starting the service container, the service container sends a GPU equipment scheduling request, and GPU equipment is allocated to the service container based on the type of the GPU equipment scheduling request and the real-time margin of the bound GPU equipment. The application effectively solves the problem that the existing GPU scheduling mode lacks adaptability when facing dynamic changes in resource demand.

Description

Technical Field

[0001] This application relates to the field of resource scheduling and management technology, and in particular to a GPU scheduling method, system, device and storage medium based on real load. Background Technology

[0002] With the widespread deployment of computationally intensive tasks such as artificial intelligence, big data, and image processing, GPUs, as high-performance parallel computing resources, have become key computing units in data centers and cloud platforms. In containerized cluster management systems (such as Kubernetes), to support the elastic allocation and efficient scheduling of GPU resources, more and more cloud service providers and enterprise platforms are introducing GPU orchestration and scheduling mechanisms to achieve resource sharing, business isolation, and automated scheduling.

[0003] Existing GPU scheduling schemes primarily rely on resource requests submitted at task submission, statically allocating GPU resources to container tasks in a fixed manner. These scheduling methods typically depend on developers' experience in estimating task resource requirements, combined with strategies such as resource reservation, static partitioning, and quota limits. Specifically, fixed allocation allocates GPU resources all at once before task startup based on estimated resource needs; resource reservation mechanisms ensure resource fairness by setting upper and lower limits; and static partitioning methods isolate and allocate GPU resources by physically or logically dividing them into multiple parts. These methods are relatively simple to implement, easy to manage, and can guarantee basic task isolation and scheduling efficiency.

[0004] However, traditional static scheduling methods often lack adaptability when faced with fluctuations in resource usage during actual task execution. Because static allocation predetermines resource allocation before task initiation, even if the actual GPU requirements change during execution, the system cannot dynamically adjust resource configuration. This can lead to some resources being idle, while some tasks are delayed due to insufficient resources. Furthermore, fixed resource reservation and constraint policies can cause significant differences in GPU utilization between nodes, further exacerbating resource imbalance. Over time, this can easily lead to resource fragmentation, decreased overall utilization, and increased task waiting times. Therefore, existing GPU scheduling methods are significantly inadequate in handling dynamic resource demands and fail to meet the requirements for efficient GPU resource utilization and improved task scheduling performance. Summary of the Invention

[0005] This application provides a GPU scheduling method, system, device, and storage medium based on real-world load, effectively solving the problem of insufficient adaptability of existing GPU scheduling methods when facing dynamic changes in resource demands. This application provides the following technical solutions: Firstly, this application provides a GPU scheduling method based on real workload, the method comprising: Obtain the total number of devices on the GPU server and register them to the Kubernetes cluster; In response to the GPU resource scheduling request of the business container, the Kubernetes scheduler obtains the current usage on the GPU server of each node, selects the node with the largest real-time reserve, and binds the business container to all GPU devices on the GPU server under the selected node. The service container is started, and the service container issues a GPU device scheduling request. A GPU device is allocated to the service container based on the type of the GPU device scheduling request and the real-time availability of the bound GPU device.

[0006] In one specific implementation, obtaining the total number of devices on the GPU server and registering it to the Kubernetes cluster includes: Read each PCIe device one by one and parse each PCIe device to extract the basic hardware information of the GPU device; By parsing the registration files and affinity information of PCIe devices, the physical topology between different GPU devices and the hardware manufacturer information corresponding to each GPU device can be obtained. Obtain the corresponding GPU product information based on the PCIe device's registration information; All extracted hardware and product information of GPU devices are registered in a structured format in the Kubernetes cluster.

[0007] In one specific implementation, reading each PCIe device and resolving each PCIe device includes: Traverse the / sys / bus / pci / devices / path in the operating system and read all registered PCIe devices under the path one by one; The uevent and numa_node files in each PCIe device directory are parsed to extract basic hardware information of the GPU device.

[0008] In one specific implementation, in response to the GPU resource scheduling request from the business container, the Kubernetes scheduler obtains the current usage on the GPU server of each node, selects the node with the largest real-time reserve, and binds the business container to all GPU devices on the GPU server under the selected node, including: Call the Kubernetes scheduler to obtain the current usage of all GPU devices in the GPU server of each node; The Kubernetes scheduler uses the real-time availability of all GPU devices in the GPU server of each node as the basis for scheduling scores. The real-time availability is obtained by subtracting the current usage of all GPU devices from the total amount of GPU devices in the GPU server of each node, and the business containers are scheduled to the node with the most real-time availability. Once the scheduling is complete, the device plugin in Kubernetes will mount all GPU devices on the selected nodes into the application containers.

[0009] In one specific implementation, allocating GPU devices to the service container based on the type of the GPU device scheduling request and the real-time availability of the bound GPU devices includes: After the business container completes scheduling and starts, it issues a GPU device scheduling request during operation. The GPU interceptor preset in the business container is invoked to intercept the GPU device scheduling request, determine the type of GPU device scheduling request, and take appropriate action.

[0010] In one specific implementation, determining the type of GPU device scheduling request and processing it accordingly includes: If the GPU device scheduling request is a GPU initialization request, obtain the real-time availability of all GPU devices that are bound to the business container, and select the GPU device whose real-time availability is closest to the current expected usage of the business as the target device; If the GPU device scheduling request is a GPU counting request, obtain the real-time availability of all GPU devices bound to the business container, determine the number of GPU devices whose real-time availability meets the current business's expected usage, and return the result. If the GPU device scheduling request is a request to specify a GPU device, specifying to call a GPU device with a specific number, determine whether the real-time availability of the specified GPU device with the specific number meets the current expected usage of the business. If it does, select that GPU device as the target device; if it does not, exclude the specified GPU device with the specific number and re-index and specify the GPU device until the real-time availability of the specified GPU device meets the current expected usage of the business, and then select it as the target device. After selecting the target device, all GPU requests from the business container will be forwarded to the target device.

[0011] Secondly, this application provides a GPU scheduling system based on real workload, employing the following technical solution: A GPU scheduling system based on real-world workloads, comprising: The total number of devices acquisition module is used to obtain the total number of devices on the GPU server and register them to the Kubernetes cluster; The node scheduling module is used to respond to the GPU resource scheduling request of the business container. The Kubernetes scheduler obtains the current usage on the GPU server of each node, selects the node with the largest real-time reserve, and binds the business container to all GPU devices on the GPU server under the selected node. The GPU device allocation module is used to start the service container, the service container issues a GPU device scheduling request, and allocates a GPU device to the service container based on the type of the GPU device scheduling request and the real-time availability of the bound GPU device.

[0012] Thirdly, this application provides an electronic device, the device including a processor and a memory; the memory stores a program, the program being loaded and executed by the processor to implement a GPU scheduling method based on real load as described in the first aspect.

[0013] Fourthly, this application provides a computer-readable storage medium storing a program that, when executed by a processor, is used to implement a GPU scheduling method based on real load as described in the first aspect.

[0014] In summary, the beneficial effects of this application include at least the following: (1) By introducing a dual strategy of load awareness combined with centralized scheduling during GPU scheduling, the fine-grained management capability of GPU resources and the overall efficiency of task scheduling are significantly improved. On the one hand, real-time availability information of GPU devices on each node can be obtained during the business container scheduling phase, realizing GPU load awareness scheduling based on the node dimension; on the other hand, when the business container starts and initiates GPU calls, the GPU interceptor further perceives the current load of each GPU at the device level and dynamically selects the most suitable target device. This dual-level, dual-granularity perception mechanism enables the scheduling system to accurately grasp the resource supply and demand situation, thereby making more reasonable decisions on scheduling strategies. In addition, by guiding GPU services to be prioritized and centrally scheduled to GPU nodes that are closest to their expected load, and matching the most suitable GPU device at the device level, not only is the matching degree of resource allocation improved, but the distribution concentration of GPU tasks within the cluster is also effectively improved. Especially in the scenario of fragmented computing load, this mechanism can reduce the disordered dispersion of tasks among multiple nodes and devices, reduce the degree of resource fragmentation, help improve the overall utilization of nodes and the control efficiency of the scheduler, and enhance the system's ability to support multi-task mixed deployment and heterogeneous computing power.

[0015] (2) Compared with the traditional scheduling method that uses static binding based on the entire GPU load, this application can intercept and analyze each GPU call within the business container, and dynamically select the most suitable GPU device for processing based on the current real-time usage status of the GPU hardware and the physical topology of the GPU device in the server. This mechanism not only avoids the waste of resources caused by static pre-allocation, but also effectively solves the mismatch problem caused by the ever-changing resource requirements of tasks in actual operation. Through the refinement of scheduling granularity and real-time perception of load status, this application can achieve precise resource adaptation and maximize reuse in computing scenarios with multiple container concurrency and frequent load fluctuations, thereby significantly reducing the GPU resource vacancy rate and improving the overall system's resource scheduling capability and processing throughput.

[0016] During node initialization, a local state collection component is deployed to acquire hardware and product specifications of each GPU device on the GPU server, and this information is then structurally registered with the Kubernetes scheduling system. During scheduling, the current usage and total number of GPU devices are considered, and real-time GPU availability is used as a scoring criterion to make scheduling decisions for business containers and pre-bind devices. During runtime, business containers dynamically and precisely select and bind devices based on the actual GPU scheduling request type and the real-time load of each GPU device. This solution allows the scheduling system to dynamically perceive resource usage during actual business operation and match GPU devices as needed. It avoids the problems of resource reservation waste, scheduling imbalance, and rigid allocation inherent in traditional static allocation methods, significantly improving resource utilization and task scheduling flexibility. This effectively solves the core technical problem of existing GPU scheduling methods lacking adaptability to dynamically changing resource demands.

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating step S101 in an embodiment of this application.

[0019] Figure 2 This is a flowchart illustrating step S102 in an embodiment of this application.

[0020] Figure 3 This is a flowchart illustrating step S103 in an embodiment of this application.

[0021] Figure 4 This is a block diagram of an electronic device based on GPU scheduling of real load in an embodiment of this application. Detailed Implementation

[0022] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0023] First, let me introduce some of the terms used in this application.

[0024] A GPU (Graphics Processing Unit) is a processor specifically designed for handling graphics and image computation tasks. Although initially designed to accelerate graphics rendering, modern GPUs have evolved into powerful computing devices widely used for a wide range of computationally intensive tasks.

[0025] Actual workload: refers to the amount of tasks that the system or GPU is currently processing, including data processing volume, computational intensity, etc.

[0026] GPU scheduling refers to the process of deciding when and how to allocate computing tasks to the GPU for processing.

[0027] Dynamic load balancing: Automatically adjusts task allocation based on changes in GPU workload during runtime to ensure that the workload of each GPU core or computing unit is as balanced as possible, avoiding resource idleness or overload.

[0028] Optionally, this application uses the GPU scheduling method based on real load provided in various embodiments in an electronic device as an example for illustration. The electronic device is a terminal or a server. The terminal can be a computer, tablet computer, etc. This embodiment does not limit the type of electronic device.

[0029] One embodiment of this application provides a GPU scheduling method based on real workload, which includes at least the following steps: Step S101: Obtain the total number of devices on the GPU server and register them to the Kubernetes cluster.

[0030] Reference Figure 1 The diagram below illustrates step S101 of this embodiment. In step S101, the objective is to extract the total number of GPU devices from the GPU server on each node and register it with the Kubernetes cluster's scheduling system. The total number of devices includes the GPU device's hardware information and product specifications. A status collection component is deployed on each GPU server. This component starts during the node initialization phase and automatically identifies and collects information on all registered GPU devices within the server. This information is then registered with the Kubernetes cluster, enabling the Kubernetes scheduling system to recognize GPU resource capabilities.

[0031] It should be noted that the status collection component in this application, namely status-collector, is essentially a piece of code residing locally on the server. Its function is similar to a web crawler, extracting hardware information for each GPU device by traversing and reading standard device files in the Linux system. As a regular information collection program, status-collector does not perform control over the GPU devices; it is only responsible for reading registration data and organizing it into a format recognizable by the scheduling system. At this stage, status-collector is mainly used to obtain node GPU hardware information as a one-time initialization operation.

[0032] Specifically, the process begins by traversing the ` / sys / bus / pci / devices / ` path in the operating system, reading each registered PCIe device under that path. Next, the `uevent` and `numa_node` files in each PCIe device's directory are parsed to extract basic GPU device hardware information. Then, the registration files and affinity information of the PCIe devices are parsed to derive the physical topology between different GPU devices and the hardware manufacturer information corresponding to each GPU device. Following this, based on the PCIe device registration information, the corresponding GPU product information, such as memory and brand specifications, is obtained. Finally, all extracted GPU device hardware information and product specifications are registered in a structured format to the Kubernetes cluster and added to the scheduler.

[0033] The above operations enable the acquisition of the total number of GPU devices in the GPU server and the completion of synchronous registration with the Kubernetes cluster during the node initialization phase. This allows the scheduling system to accurately score and match resources based on real parameters such as the number of GPU devices, memory capacity, and physical topology of each node when performing GPU resource scheduling. This effectively improves the accuracy of resource identification and the rationality of scheduling allocation, avoiding resource waste or task failure due to missing information.

[0034] Step S102: In response to the GPU resource scheduling request of the business container, the Kubernetes scheduler obtains the current usage on the GPU server of each node, selects the node with the largest real-time reserve, and binds the business container to all GPU devices on the GPU server under the selected node.

[0035] Reference Figure 2The diagram below illustrates step S102 in this embodiment. In step S102, when a business container issues a GPU resource scheduling request, the Kubernetes scheduler combines the current usage of GPU servers on each node with real-time availability for scoring and sorting. It prioritizes the node with the most real-time GPU availability and mounts all GPU devices on that node into the business container. The goal of this step is to achieve more dynamic and accurate scheduling decisions by real-time sensing the GPU resource occupancy status of each node in the Kubernetes cluster, while simultaneously completing the pre-binding of GPU devices at the container level, providing a physical path for subsequent device selection.

[0036] In implementation, when a business container issues a GPU resource scheduling request, the Kubernetes scheduler first retrieves the current usage of all GPU devices on each node's GPU server. Specifically, the status-collector obtains the current usage of all GPU devices on each node's GPU server in real time and updates the cluster information. Subsequently, the metrics-server in the Kubernetes cluster periodically collects the current usage of all GPU devices on each GPU server in the cluster. The metrics-server is a standard component officially used by Kubernetes for collecting node and Pod-level resource usage. The Kubernetes scheduler obtains the periodically collected current usage of all GPU devices on each node's GPU server through the metrics-server. Then, the Kubernetes scheduler uses the real-time available resources of all GPU devices on each node's GPU server as the scheduling scoring criterion. The real-time available resources are obtained by subtracting the current usage of all GPU devices from the total available resources on each node's GPU server, and the business container is scheduled to the node with the most available resources.

[0037] After scheduling is complete, the Device Plugin in Kubernetes will mount all GPU devices on the selected node into the application container, so that the application container has the ability to access all GPU devices on the node after it starts, thus completing the physical binding between GPU devices and application containers.

[0038] Through the steps described above, this application can dynamically perceive the actual usage of each node's GPU server based on standard components and extension mechanisms within the native Kubernetes architecture. Using real-time GPU availability as the core scoring criterion, it guides the scheduler to allocate resources reasonably across the cluster. This scheduling method breaks through the traditional scoring logic based on static allocation, avoiding scheduling imbalances caused by inefficient resource usage or uneven distribution. Simultaneously, by uniformly mounting all GPU devices to the target business container, it lays the runtime access foundation for subsequent fine-grained selection of GPU devices based on real-time load, effectively improving the intelligence of the scheduling strategy and resource utilization, especially suitable for computing scenarios with multi-task concurrency and significant GPU load fluctuations.

[0039] Step S103: Start the business container. The business container sends a GPU device scheduling request. Based on the type of GPU device scheduling request and the real-time availability of the bound GPU device, the business container is allocated a GPU device.

[0040] Reference Figure 3 The diagram below illustrates step S103 in this embodiment. In step S103, to achieve dynamic and fine-grained allocation of GPU devices, a GPU interceptor is injected into the business container to intercept all scheduling requests related to GPU devices within the business container. For different types of GPU device scheduling requests, the interceptor will dynamically select and allocate GPU devices based on the real-time availability of GPU devices already bound to the node, thereby achieving an on-demand allocation and precise matching device usage strategy.

[0041] Specifically, after the business container completes scheduling and starts, it issues a GPU device scheduling request during operation. At this time, the GPU interceptor preset in the business container is invoked to intercept the GPU device scheduling request, determine the type of GPU device scheduling request, and take appropriate action.

[0042] If the GPU device scheduling request is a GPU initialization request, then the real-time availability of all GPU devices bound to the business container is obtained, calculated by subtracting the current usage from the total number of GPU devices. The GPU device with the real-time availability closest to the expected current business usage is selected as the target device. If the GPU device scheduling request is a GPU counting request, then obtain the real-time availability of all GPU devices bound to the business container, calculated by subtracting the current usage from the total number of GPU devices. Determine the number of GPU devices whose real-time availability meets the current business's expected usage and return that number.

[0043] If the GPU device scheduling request is a request to specify a particular GPU device, i.e., to call a GPU device with a specific ID, the system first checks whether the real-time availability of the specified GPU device meets the current expected usage. If it does, that GPU device is selected as the target device. If not, the specified GPU device is excluded, and the system re-indexes and specifies a GPU device until the real-time availability of the specified GPU device meets the current expected usage, at which point it is selected as the target device.

[0044] Once the target device is selected, the business container will stably use the GPU device to complete GPU-related computing tasks during runtime, and forward all subsequent GPU requests to that GPU device.

[0045] Through the above operations, this application further implements a dynamic allocation mechanism based on scheduling request type and real-time GPU resource status, assuming the business container has already been bound to the GPU node. Compared to the traditional method of statically determining GPU devices during container startup, this solution can adapt resources according to the actual time when the business initiates GPU calls, avoiding the waste of reserved GPU resources and allocation errors. Especially in computing environments with significant GPU resource fluctuations, uneven call intensity, or uncertain task startup order, this dynamic selection mechanism can significantly improve GPU resource utilization efficiency and the rationality of device allocation, providing a higher-granularity resource scheduling capability for the Kubernetes scheduling system.

[0046] In summary, during the node initialization phase, hardware and product specifications of each GPU device on the GPU server are acquired by deploying a local state collection component and registered in a structured manner to the Kubernetes scheduling system. During the scheduling phase, the scheduling decisions for business containers and device pre-binding are completed based on the current usage and total number of GPU devices, using real-time GPU availability as a scoring criterion. During the runtime phase, business containers dynamically and precisely select and bind devices based on the actual GPU scheduling request type and the real-time load of each GPU device. This solution allows the scheduling system to dynamically perceive resource usage status during actual business operation and match GPU devices as needed, avoiding the resource reservation waste, scheduling imbalance, and rigid allocation problems of traditional static allocation methods. It significantly improves resource utilization and task scheduling flexibility, effectively solving the core technical problem of existing GPU scheduling methods lacking adaptability to dynamic changes in resource demand.

[0047] Compared to traditional static scheduling methods that bind resources based on the entire GPU load, this application intercepts and analyzes each GPU call within the application container. By combining the real-time usage status of the GPU hardware with the physical topology of the GPU devices within the server, it dynamically selects the most suitable GPU for processing. This mechanism not only avoids resource waste caused by static pre-allocation but also effectively solves the mismatch problem caused by constantly changing resource requirements of tasks during actual operation. Through refined scheduling granularity and real-time load awareness, this application can achieve precise resource adaptation and maximized reuse in computing scenarios with multiple container concurrency and frequent load fluctuations, thereby significantly reducing GPU resource vacancy rates and improving the overall system's resource scheduling capabilities and processing throughput.

[0048] One embodiment of this application also provides a GPU scheduling system based on real workload, which includes at least the following modules: The total number of devices acquisition module is used to obtain the total number of devices on the GPU server and register them to the Kubernetes cluster; The node scheduling module is used to respond to GPU resource scheduling requests from business containers. The Kubernetes scheduler obtains the current usage on the GPU server of each node, selects the node with the largest real-time reserve, and binds the business container to all GPU devices on the GPU server under the selected node. The GPU device allocation module is used to start the business container. The business container issues a GPU device scheduling request, and the module allocates GPU devices to the business container based on the type of GPU device scheduling request and the real-time availability of the bound GPU devices.

[0049] For relevant details, please refer to the above method implementation examples.

[0050] Figure 4 This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 401 and a memory 402.

[0051] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0052] Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash storage devices. In some embodiments, the non-transitory computer-readable storage media in memory 402 is used to store at least one instruction, which is executed by processor 401 to implement the GPU scheduling method based on real-load provided in the method embodiments of this application.

[0053] In some embodiments, the electronic device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 401, memory 402, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to: radio frequency circuits, touch displays, audio circuits, and power supplies.

[0054] Of course, electronic devices may also include fewer or more components, and this embodiment does not limit this.

[0055] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the GPU scheduling method based on real load described in the above method embodiments.

[0056] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the GPU scheduling method based on real load described in the above method embodiments.

[0057] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A GPU scheduling method based on real workload, characterized in that, The method includes: Obtain the total number of devices on the GPU server and register them to the Kubernetes cluster; The step of obtaining the total number of devices on the GPU server and registering them to the Kubernetes cluster includes: A state collection component is deployed on each GPU server. The state collection component is started during the node initialization phase and is used to automatically identify and collect information on all registered GPU devices inside the server and register the collected GPU device information to the Kubernetes cluster. Read each PCIe device one by one and parse each PCIe device to extract the basic hardware information of the GPU device; parse the registration file and affinity information of the PCIe device to obtain the physical topology between different GPU devices and the hardware manufacturer information of each GPU device; obtain the corresponding GPU product information based on the device registration information of the PCIe device; register all the extracted hardware information and product information of the GPU devices in a structured format to the Kubernetes cluster. The step of reading and parsing each PCIe device includes: traversing the / sys / bus / pci / devices / path in the operating system and reading each registered PCIe device under the path; parsing the uevent file and numa_node file in the directory of each PCIe device to extract the basic hardware information of the GPU device; In response to the GPU resource scheduling request of the business container, the Kubernetes scheduler obtains the current usage on the GPU server of each node, selects the node with the largest real-time reserve, and binds the business container to all GPU devices on the GPU server under the selected node. The service container is started, and the service container issues a GPU device scheduling request. A GPU device is allocated to the service container based on the type of the GPU device scheduling request and the real-time availability of the bound GPU device.

2. The GPU scheduling method based on real load according to claim 1, characterized in that, In response to the GPU resource scheduling request from the business container, the Kubernetes scheduler obtains the current usage on the GPU server of each node, selects the node with the largest real-time reserve, and binds the business container to all GPU devices on the GPU server under the selected node, including: Call the Kubernetes scheduler to obtain the current usage of all GPU devices in the GPU server of each node; The Kubernetes scheduler uses the real-time availability of all GPU devices in the GPU server of each node as the basis for scheduling scores. The real-time availability is obtained by subtracting the current usage of all GPU devices from the total amount of GPU devices in the GPU server of each node, and the business containers are scheduled to the node with the most real-time availability. Once the scheduling is complete, the device plugin in Kubernetes will mount all GPU devices on the selected nodes into the application containers.

3. The GPU scheduling method based on real load according to claim 1, characterized in that, The process of allocating GPU devices to the service container based on the type of the GPU device scheduling request and the real-time availability of the bound GPU device includes: After the business container completes scheduling and starts, it issues a GPU device scheduling request during operation. The GPU interceptor preset in the business container is invoked to intercept the GPU device scheduling request, determine the type of GPU device scheduling request, and take appropriate action.

4. The GPU scheduling method based on real load according to claim 3, characterized in that, The process of determining the type of GPU device scheduling request and taking corresponding action includes: If the GPU device scheduling request is a GPU initialization request, obtain the real-time availability of all GPU devices that are bound to the business container, and select the GPU device whose real-time availability is closest to the current expected usage of the business as the target device; If the GPU device scheduling request is a GPU counting request, obtain the real-time availability of all GPU devices bound to the business container, determine the number of GPU devices whose real-time availability meets the current business's expected usage, and return the result. If the GPU device scheduling request is a request to specify a GPU device, specifying to call a GPU device with a specific number, determine whether the real-time availability of the specified GPU device with the specific number meets the current expected usage of the business. If it does, select that GPU device as the target device; if it does not, exclude the specified GPU device with the specific number and re-index and specify the GPU device until the real-time availability of the specified GPU device meets the current expected usage of the business, and then select it as the target device. After selecting the target device, all GPU requests from the business container will be forwarded to the target device.

5. A GPU scheduling system based on real-world load, characterized in that, include: The total number of devices acquisition module is used to obtain the total number of devices on the GPU server and register them to the Kubernetes cluster; The step of obtaining the total number of devices on the GPU server and registering them to the Kubernetes cluster includes: A state collection component is deployed on each GPU server. The state collection component is started during the node initialization phase and is used to automatically identify and collect information on all registered GPU devices inside the server and register the collected GPU device information to the Kubernetes cluster. Read each PCIe device one by one and parse each PCIe device to extract the basic hardware information of the GPU device; parse the registration file and affinity information of the PCIe device to obtain the physical topology between different GPU devices and the hardware manufacturer information of each GPU device; obtain the corresponding GPU product information based on the device registration information of the PCIe device; register all the extracted hardware information and product information of the GPU devices in a structured format to the Kubernetes cluster. The step of reading and parsing each PCIe device includes: traversing the / sys / bus / pci / devices / path in the operating system and reading each registered PCIe device under the path; parsing the uevent file and numa_node file in the directory of each PCIe device to extract the basic hardware information of the GPU device; The node scheduling module is used to respond to the GPU resource scheduling request of the business container. The Kubernetes scheduler obtains the current usage on the GPU server of each node, selects the node with the largest real-time reserve, and binds the business container to all GPU devices on the GPU server under the selected node. The GPU device allocation module is used to start the service container, the service container issues a GPU device scheduling request, and allocates a GPU device to the service container based on the type of the GPU device scheduling request and the real-time availability of the bound GPU device.

6. An electronic device, characterized in that, The device includes a processor and a memory; the memory stores a program that is loaded and executed by the processor to implement a GPU scheduling method based on real load as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement a GPU scheduling method based on real load as described in any one of claims 1 to 4.

Images