GPU Pass-Through and Resource Hybrid Scheduling Method, System and Chip

Abstract

The present application provides a GPU direct pass-through and resource hybrid scheduling method, system, and chip. The method includes: S1: Obtain the resource information of the GPUs of each node and save it to the scheduling buffer; S2: Based on hybrid quantization calculation, obtain the computing power data and total video memory demand information of the GPUs of each node; S3: Based on the total video memory demand information of the GPUs of each node, perform a first screening on the nodes to obtain a first screening result; S4: Based on the full permutation and combination function and the pruning function, obtain a reduced two-dimensional full permutation array; S5: Based on the first screening result, perform a second screening operation on the nodes to obtain an optimal node result. This method can expand the application scenarios of GPU scheduling by no longer distinguishing between GPU direct pass-through and virtualization scheduling scenarios, using a new hybrid scheduling method to obtain optimal nodes, further improving the utilization rate and usage rate of GPU resources, saving labor and material costs, and having advantages such as better scheduling performance and higher scheduling accuracy.

Description

Technical Field

[0001] This application relates to the technical field of cloud-native containers, and particularly to GPU passthrough and resource hybrid scheduling methods, systems, and chips. Background Art

[0002] A GPU (Graphics Processing Unit) has a large number of cores and high-speed memory, is good at parallel computing, and is suitable for scenarios such as deep learning training, inference, and high-performance scientific computing. Currently, many AI applications need to utilize a large amount of GPU resources for acceleration training. When there are more and more GPU resources and training tasks, the utilization and management of GPU resources become difficult. For example, during daytime working hours, the demand for GPU resources is very tense, and it is often "hard to get a single card", while at night, a considerable number of GPU cards are idle, resulting in unnecessary resource waste. Therefore, most enterprises and universities have adopted K8s (Kubernetes) as a unified orchestration and scheduling management platform to automate and uniformly manage training resources, thereby improving the utilization rate of GPU resources and work efficiency.

[0003] In the prior art, the two scenarios of GPU passthrough whole-card scheduling and virtualization scheduling are separated, and the two scheduling scenarios cannot coexist. When GPU passthrough is enabled, the most efficient computing power resources can be obtained, but virtualization scheduling cannot be enabled for the GPU. Often, the resources of a cluster are occupied early, turning away some demands that only require a small amount of computing power and video memory. When virtualization scheduling is enabled, whole-card passthrough cannot be taken into account, resulting in only a part of the computing power being obtained when there are passthrough demands that require a large amount of computing resources, leading to low training efficiency. This limits the application scenarios of the GPU and causes unnecessary trouble. The prior art cannot be efficient in the case of GPU passthrough. Therefore, how to perform GPU passthrough and resource hybrid scheduling has become an urgent problem to be solved.

[0004] Patent CN114816746A discloses a method for implementing hybrid type virtualization of a GPU with multiple virtualization types, including: monitoring, scheduling, and allocation of vGPU resources; monitoring of vGPU resources is used to monitor the health status of the GPU and set monitoring metrics; scheduling is based on the monitoring metrics to obtain useful data therefrom for scheduling to vGPU resources; scheduling includes filtering and evaluation: filtering selects corresponding GPUs according to requirements, which is implemented through multiple filters; evaluation is to score according to evaluation metrics among multiple resources that still meet the requirements after filtering, and finally select the one with the highest score for scheduling; the said allocation is achieved by adding multiple mdevs of different vGPU types under the same PCI device directory to allocate hybrid type vGPUs. The efficiency of this method for scheduling is low and it cannot take into account whole-card passthrough.

[0005] Based on this, the present application provides a GPU direct pass and resource hybrid scheduling method, system and chip to improve the existing technology. Summary of the Invention

[0006] The purpose of the present application is to provide a GPU direct pass and resource hybrid scheduling method, system and chip, which can expand the application scenarios of GPU scheduling by no longer distinguishing between GPU direct pass and virtualization scheduling scenarios, utilize the new hybrid scheduling method to obtain the optimal nodes, further improve the utilization rate and usage rate of GPU resources, save manpower and material costs, and has advantages such as better scheduling performance and higher scheduling accuracy.

[0007] The purpose of the present application is achieved by the following technical solutions:

[0008] In the first aspect, the present application provides a GPU direct pass and resource hybrid scheduling method, and the method includes:

[0009] S1: Obtain the resource information of the GPUs of each node and save it in the scheduling buffer area, and the resource information includes the total available resource information and the remaining resource information;

[0010] S2: Based on hybrid quantization calculation, obtain the computing power data and the total video memory demand information of the GPUs of each node;

[0011] S3: Based on the total video memory demand information of the GPUs of each node, perform a first screening on the nodes to obtain a first screening result;

[0012] S4: Based on the full permutation and combination function and the pruning function, obtain a refined two-dimensional full permutation array;

[0013] S5: Based on the first screening result, perform a second screening operation on the nodes to obtain a preferred node result.

[0014] The beneficial effects of this technical solution are as follows: First, obtain the resource information of the GPUs of each node and save it in the scheduling buffer area; based on hybrid quantization calculation, obtain the computing power data and the total video memory demand information of the GPUs of each node; secondly, based on the total video memory demand information of the GPUs of each node, perform a first screening on the nodes to obtain a first screening result; then, based on the full permutation and combination function and the pruning function, obtain a refined two-dimensional full permutation array; finally, based on the first screening result, perform a second screening operation on the nodes to obtain a preferred node result. This method can expand the application scenarios of GPU scheduling by no longer distinguishing between GPU direct pass and virtualization scheduling scenarios, utilize the new hybrid scheduling method to obtain the optimal nodes, further improve the utilization rate and usage rate of GPU resources, save manpower and material costs, and has advantages such as better scheduling performance and higher scheduling accuracy.

[0015] In some alternative embodiments, obtaining and saving the resource information of the GPUs of each node, where the resource information includes total available resource information and remaining resource information, includes:

[0016] Using a device-plugin component to obtain the attribute information of the GPUs corresponding to each node, where the attribute information of the GPUs includes the GPU information reporting time and the GPU status information;

[0017] When the periodic scheduler starts scheduling, determine whether the GPU information reporting time is greater than a preset reporting time; if the GPU information reporting time is greater than the preset reporting time, set the GPU to an unavailable state;

[0018] If the GPU information reporting time is not greater than the preset reporting time, determine whether the GPU status information is in a ready state; if the GPU status information is not in a ready state, set the GPU to an unavailable state;

[0019] If the GPU status information is in a ready state, traverse the attribute information of the GPU to obtain the total available resource information and save it to the scheduling buffer;

[0020] Based on the total available resource information, obtain the remaining resource information and save it to the scheduling buffer.

[0021] The beneficial effects of this technical solution are as follows: Through the device-plugin component, the attribute information of the GPUs corresponding to each node can be obtained in real time, including the GPU information reporting time and the GPU status information. This helps to understand the real-time usage of GPU resources in the cluster and provides accurate data support for task scheduling. When the periodic scheduler starts scheduling, it can determine whether the GPU resources are available based on the preset reporting time. If the GPU information reporting time is greater than the preset reporting time, or the GPU status information is not in the ready state, the GPU is set to the unavailable state. In this way, it can be ensured that the scheduler will not schedule tasks to unavailable GPU resources, improving the task success rate. When the GPU status information is in the ready state, the attribute information of the GPU can be traversed to obtain the total available resource information and saved to the scheduling buffer. Then, based on the total available resource information, the remaining resource information can be obtained and saved to the scheduling buffer. This helps to better understand the remaining GPU resources in the cluster, so as to achieve the optimal allocation of resources during task scheduling and improve the utilization rate of cluster resources. Through the device-plugin component, the GPU resource management policy can be customized according to actual needs, such as setting different reporting times and preset states. This makes the technical solution highly flexible and able to adapt to the GPU resource management requirements in different scenarios. In summary, the technical solution for implementing GPU resource management using the device-plugin component has beneficial effects such as real-time monitoring, periodic scheduling, optimized resource utilization, and flexible adaptation to different scenarios, which helps to improve the utilization rate of GPU resources and the task scheduling efficiency in the Kubernetes cluster.

[0022] In some alternative embodiments, the step of obtaining the remaining resource information based on the total available resource information and saving it to the scheduling buffer includes:

[0023] Obtain the GPU index and GPU usage corresponding to the applications on each node;

[0024] Based on the GPU index and GPU usage corresponding to the applications on each node, obtain the used resource information;

[0025] Based on the total available resource information, remove the used resource information from the scheduling buffer to obtain the remaining resource information and save it to the scheduling buffer.

[0026] The beneficial effects of this technical solution are as follows: It can accurately master the GPU usage of each application on each node, including the used GPU index and usage amount, which helps to more precisely understand the usage status of GPU resources in the cluster. Based on the GPU index and GPU usage amount corresponding to the applications on each node, the used resource information can be obtained. This helps to more accurately calculate the used resources during resource scheduling and avoid duplicate resource allocation. When obtaining the remaining resource information, the used resource information can be removed from the scheduling buffer area to obtain the remaining resource information and save it to the scheduling buffer area. This helps to better understand the remaining resources in the cluster, so as to achieve optimal resource allocation during task scheduling and improve the utilization rate of cluster resources. Through refined resource monitoring and optimized resource scheduling, it can ensure that tasks can be efficiently executed in the cluster and improve task execution efficiency. In summary, the technical solution for obtaining the GPU index and GPU usage amount corresponding to the applications on each node has beneficial effects such as refined management, accurate calculation of used resources, optimized resource scheduling, and improved task execution efficiency, which helps to improve the resource management efficiency and task execution efficiency of the cluster.

[0027] In some optional embodiments, obtaining the computing power data and total video memory demand information of the GPUs of each node based on hybrid quantization calculation includes:

[0028] Respectively determine whether the applications on each node have GPU requirements;

[0029] If an application on a certain node has GPU requirements, calculate and save the GPU index corresponding to the application on that node; if an application on a certain node has no GPU requirements, do not perform any operation on the application on that node;

[0030] Determine whether the GPU requirement is in the whole-card direct-through mode. If the GPU requirement is in the whole-card direct-through mode, obtain the computing power data and total video memory demand information of the GPU of the node through a multiplication operation based on the GPU index;

[0031] If the GPU requirement is not in the whole-card direct-through mode, obtain the computing power data and total video memory demand information of the GPU of the node through an addition operation based on the GPU index.

[0032] The beneficial effects of this technical solution are as follows: It determines whether the applications on each node require GPU resources. For applications with GPU requirements, it calculates their corresponding GPU indices and saves them, while for applications without GPU requirements, no operation is performed. In this way, resource management can be carried out according to the personalized requirements of applications, improving resource utilization efficiency. For applications with GPU requirements, it further determines whether they require the whole-card direct pass-through mode. If the requirement is the whole-card direct pass-through mode, it obtains the computing power data and total video memory requirement information of the node's GPU through a multiplication operation based on the GPU index; if the requirement is not the whole-card direct pass-through mode, it obtains the corresponding information through an addition operation. In this way, the requirements of applications on each node for GPU resources can be accurately calculated, providing accurate data support for resource scheduling. The technical solution can adapt to the GPU resource requirements in different scenarios. Whether it is the whole-card direct pass-through mode or other modes, the required resource information can be obtained through corresponding operations, with strong flexibility.

[0033] In some alternative embodiments, the first screening of the nodes based on the total video memory requirement information of the GPUs of the nodes to obtain a first screening result includes:

[0034] Comparing the total video memory requirement information of the node's GPU with the remaining resource information;

[0035] If the total video memory requirement information of the node's GPU is less than the remaining resource information, it is considered that the resources of this node are insufficient and this node is filtered out;

[0036] If the total video memory requirement information of the node's GPU is not less than the remaining resource information, this node is saved to obtain the first screening result.

[0037] The beneficial effects of this technical solution are as follows: If the total video memory requirement information of a node's GPU is less than the remaining resource information, it means that the resources of this node are insufficient to meet the requirements of new tasks. In this case, this node can be filtered out to avoid scheduling tasks to nodes with insufficient resources, thereby improving the success rate of tasks. If the total video memory requirement information of a node's GPU is not less than the remaining resource information, it indicates that the resources of this node can meet the requirements of new tasks. In this case, this node will be saved to become the first screening result. In this way, it can ensure that nodes with sufficient resources are fully utilized, improving the resource utilization rate of the cluster. By comparing the total video memory requirement information of nodes and the remaining resource information, it can quickly determine which nodes have sufficient resources to execute new tasks. This helps to improve task scheduling efficiency and reduce task failures caused by insufficient resources. It can adapt to the GPU resource requirements in different scenarios. Whether it is video memory requirements or other requirements, it can determine whether the resources of a node are sufficient through comparison operations, with strong flexibility.

[0038] In some alternative embodiments, obtaining the refined two-dimensional permutation array based on the full permutation combination function and the pruning function includes:

[0039] Inputting preset application information to obtain an index array with a GPU;

[0040] Inputting the index array into a preset full permutation function to obtain a two-dimensional full permutation array;

[0041] Determining whether the computing power data of the GPU is equal to the video memory data of the GPU;

[0042] If the computing power data of the GPU is equal to the video memory data of the GPU, marking the index array of the GPU to obtain an equivalent array;

[0043] Inputting the two-dimensional full permutation array and the equivalent array into a preset pruning function to obtain the refined two-dimensional permutation array.

[0044] The beneficial effects of this technical solution are as follows: By inputting preset application information, an index array with a GPU can be obtained according to the specific requirements of the application, thereby realizing personalized resource management for different applications. Inputting the index array into a preset full permutation function can obtain a two-dimensional full permutation array, so that all possible GPU resource allocation schemes can be comprehensively considered during task scheduling, improving resource utilization and task scheduling efficiency. After determining whether the computing power data of the GPU is equal to the video memory data of the GPU, if they are equal, the index array of the GPU is marked to obtain an equivalent array. Then, the two-dimensional full permutation array and the equivalent array are input into a preset pruning function to obtain the refined two-dimensional permutation array. In this way, precise pruning can be performed to remove GPU resource allocation schemes that do not meet the conditions, thereby improving task scheduling efficiency and resource utilization. The technical solution of inputting preset application information and obtaining an index array with a GPU, and then performing full permutation and pruning operations has beneficial effects such as personalized resource management, full permutation optimization, precise pruning, and adaptation to different scenarios, which helps to improve the resource management efficiency and task execution efficiency of the cluster.

[0045] In some alternative embodiments, the second screening operation on the nodes based on the first screening result to obtain the preferred node result includes:

[0046] Performing comparison screening on the refined two-dimensional permutation array based on the first screening result to obtain a comparison screening result;

[0047] Performing score-based optimization on the nodes respectively based on the comparison screening result to obtain the preferred node result.

[0048] The beneficial effects of this technical solution are as follows: Based on the first screening result, the refined two-dimensional full permutation array is compared and screened to obtain the comparison and screening result. In this way, nodes that meet the conditions can be accurately screened out, improving task scheduling efficiency and resource utilization rate. Based on the comparison and screening result, the nodes are scored and optimized to obtain the optimized node result. In this way, according to the resource status and task requirements of the nodes, the best execution node can be selected for the task, improving task execution efficiency and cluster resource utilization rate. By accurately screening and optimizing node selection, it can be ensured that tasks can be executed efficiently and smoothly in the cluster, improving task execution effect.

[0049] In a second aspect, the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor is configured to perform the following steps when executing the computer program:

[0050] S1: Obtain the resource information of the GPUs of each node and save it in the scheduling buffer area. The resource information includes total available resource information and remaining resource information;

[0051] S2: Based on hybrid quantization calculation, obtain the computing power data and total video memory demand information of the GPUs of each node;

[0052] S3: Based on the total video memory demand information of the GPUs of each node, perform a first screening on the nodes to obtain a first screening result;

[0053] S4: Based on the full permutation combination function and pruning function, obtain a refined two-dimensional full permutation array;

[0054] S5: Based on the first screening result, perform a second screening operation on the nodes to obtain an optimized node result.

[0055] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is used to obtain and save the resource information of the GPUs of each node. The resource information includes total available resource information and remaining resource information:

[0056] Use the device-plugin component to obtain the attribute information of the GPUs corresponding to each node. The attribute information of the GPUs includes GPU information reporting time and GPU status information;

[0057] When the periodic scheduler starts scheduling, determine whether the GPU information reporting time is greater than the preset reporting time; if the GPU information reporting time is greater than the preset reporting time, set the GPU to an unavailable state;

[0058] If the GPU information reporting time is not greater than the preset reporting time, determine whether the GPU status information is in a ready state; if the GPU status information is not in a ready state, set the GPU to an unavailable state;

[0059] If the GPU status information is in a ready state, traverse the attribute information of the GPU to obtain the total available resource information and save it to the scheduling buffer;

[0060] Based on the total available resource information, obtain the remaining resource information and save it to the scheduling buffer.

[0061] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to obtain the remaining resource information based on the total available resource information and save it to the scheduling buffer:

[0062] Obtain the GPU index and GPU usage corresponding to the applications on each node;

[0063] Based on the GPU index and GPU usage corresponding to the applications on each node, obtain the used resource information;

[0064] Based on the total available resource information, remove the used resource information from the scheduling buffer to obtain the remaining resource information and save it to the scheduling buffer.

[0065] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to obtain the computing power data and total video memory demand information of the GPUs on each node based on mixed quantization calculation:

[0066] Respectively determine whether the applications on each node have GPU requirements;

[0067] If an application on a certain node has GPU requirements, calculate and save the GPU index corresponding to the application on that node; if an application on a certain node has no GPU requirements, do not perform any operation on the application on that node;

[0068] Determine whether the GPU requirement is in the whole-card direct-through mode. If the GPU requirement is in the whole-card direct-through mode, obtain the computing power data and total video memory demand information of the GPU of the node through a multiplication operation based on the GPU index;

[0069] If the GPU requirement is not in the whole-card direct-through mode, obtain the computing power data and total video memory demand information of the GPU of the node through an addition operation based on the GPU index.

[0070] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to perform a first screening on the nodes based on the total video memory requirement information of the GPUs of the respective nodes to obtain a first screening result:

[0071] Compare the total video memory requirement information of the GPUs of the nodes with the remaining resource information;

[0072] If the total video memory requirement information of the GPUs of the node is less than the remaining resource information, it is considered that the resources of the node are insufficient and the node is filtered;

[0073] If the total video memory requirement information of the GPUs of the node is not less than the remaining resource information, save the node to obtain the first screening result.

[0074] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to obtain a reduced two-dimensional full permutation array based on the full permutation combination function and the pruning function:

[0075] Input preset application information to obtain an index array with GPUs;

[0076] Input the index array into a preset full permutation function to obtain a two-dimensional full permutation array;

[0077] Determine whether the computing power data of the GPUs is equal to the video memory data of the GPUs;

[0078] If the computing power data of the GPUs is equal to the video memory data of the GPUs, mark the index array of the GPUs to obtain an equivalent array;

[0079] Input the two-dimensional full permutation array and the equivalent array into a preset pruning function to obtain the reduced two-dimensional full permutation array.

[0080] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to perform a second screening operation on the nodes based on the first screening result to obtain a preferred node result:

[0081] Based on the first screening result, perform a comparison and screening on the reduced two-dimensional full permutation array to obtain a comparison and screening result;

[0082] Based on the comparison and screening result, perform a scoring and preference selection on the nodes respectively to obtain a preferred node result.

[0083] In a third aspect, the present application provides a scheduling system, and the electronic device includes:

[0084] The above-mentioned electronic device.

[0085] Fourthly, the present application provides a chip, which stores a computer program. When the computer program is executed by a processor, the steps of any one of the above methods are implemented. Description of the Drawings

[0086] The present application will be further described below in conjunction with the drawings and embodiments.

[0087] Figure 1 Fig. shows a schematic flow chart of a GPU direct pass and resource hybrid scheduling method provided by an embodiment of the present application;

[0088] Figure 2 Fig. shows a schematic flow chart of a method for obtaining resource information provided by an embodiment of the present application;

[0089] Figure 3 Fig. shows a schematic flow chart of a method for obtaining remaining resource information provided by an embodiment of the present application;

[0090] Figure 4 Fig. shows a schematic flow chart of a method for obtaining a reduced two-dimensional full permutation array provided by an embodiment of the present application;

[0091] Figure 5 Fig. shows a structural framework diagram of an electronic device provided by an embodiment of the present application;

[0092] Figure 6 Fig. shows a schematic structural diagram of a scheduling system provided by an embodiment of the present application;

[0093] Figure 7 Fig. shows a schematic structural diagram of a program product provided by an embodiment of the present application. Detailed Embodiments

[0094] Hereinafter, in conjunction with the drawings and specific embodiments, embodiments of the present application will be further described. It should be noted that, on the premise of no conflict, the following-described embodiments or technical features may be arbitrarily combined to form new embodiments.

[0095] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B may be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item or plural items. For example, at least one (item) of a, b, or c may represent: a, b, c, a and b, a and c, b and c, a and b and c, where a, b, and c may be single or multiple. It should be noted that "at least one (item)" can also be interpreted as "one item or multiple items".

[0096] It should also be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to give examples, illustrations, or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0097] Method embodiments

[0098] See Figure 1 , Figure 1 which shows a schematic flowchart of a GPU direct pass and resource hybrid scheduling method provided by the embodiments of the present application.

[0099] The present application provides a GPU direct pass and resource hybrid scheduling method, and the method includes:

[0100] S1: Obtain the resource information of the GPUs of each node and save it in the scheduling buffer area, where the resource information includes total available resource information and remaining resource information;

[0101] S2: Based on hybrid quantization calculation, obtain the computing power data and total video memory demand information of the GPUs of each node;

[0102] S3: Based on the total video memory demand information of the GPUs of each node, perform a first screening on the nodes to obtain a first screening result;

[0103] S4: Based on the full permutation combination function and pruning function, obtain a refined two-dimensional full permutation array;

[0104] S5: Based on the first screening result, perform a second screening operation on the nodes to obtain a preferred node result.

[0105] Thus, first, obtain the resource information of the GPUs of each node and save it to the scheduling buffer; based on hybrid quantization calculation, obtain the computing power data and total video memory demand information of the GPUs of each node; second, based on the total video memory demand information of the GPUs of each node, perform a first screening on the nodes to obtain a first screening result; then, based on the full permutation combination function and the pruning function, obtain a refined two-dimensional full permutation array; finally, based on the first screening result, perform a second screening operation on the nodes to obtain an optimal node result. This method can expand the application scenarios of GPU scheduling by no longer distinguishing between GPU passthrough and virtualization scheduling scenarios, using a new hybrid scheduling method to obtain the optimal nodes, further improving the utilization rate and usage rate of GPU resources, saving labor and material costs, and having the advantages of better scheduling performance and higher scheduling accuracy.

[0106] See Figure 2 , Figure 2 which shows a schematic flowchart of a method for obtaining resource information provided by an embodiment of the present application.

[0107] In some optional implementation manners, for the obtaining and saving of the resource information of the GPUs of each node, the resource information includes available total resource information and remaining resource information, and it includes:

[0108] Step S201: Use the device-plugin component to obtain the attribute information of the GPUs corresponding to each node, where the attribute information of the GPUs includes the GPU information reporting time and the GPU status information;

[0109] Step S202: When the periodic scheduler starts scheduling, determine whether the GPU information reporting time is greater than a preset reporting time; if the GPU information reporting time is greater than the preset reporting time, set the GPU to an unavailable state;

[0110] Step S203: If the GPU information reporting time is not greater than the preset reporting time, determine whether the GPU status information is in a ready state; if the GPU status information is not in a ready state, set the GPU to an unavailable state;

[0111] Step S204: If the GPU status information is in a ready state, traverse the attribute information of the GPU to obtain the available total resource information and save it to the scheduling buffer;

[0112] Step S205: Based on the available total resource information, obtain the remaining resource information and save it to the scheduling buffer.

[0113] Thus, through the device-plugin component, the attribute information of the GPUs corresponding to each node can be obtained in real time, including the GPU information reporting time and the GPU status information. This helps to understand the real-time usage of GPU resources in the cluster and provides accurate data support for task scheduling. When the periodic scheduler starts scheduling, it can determine whether the GPU resources are available based on the preset reporting time. If the GPU information reporting time is greater than the preset reporting time, or the GPU status information is not in the ready state, the GPU is set to the unavailable state. In this way, it can be ensured that the scheduler will not schedule tasks to unavailable GPU resources, improving the task success rate. When the GPU status information is in the ready state, the attribute information of the GPU can be traversed to obtain the total available resource information and saved to the scheduling buffer. Then, based on the total available resource information, the remaining resource information can be obtained and saved to the scheduling buffer. This helps to better understand the remaining GPU resources in the cluster, so as to achieve the optimal allocation of resources during task scheduling and improve the cluster resource utilization rate. Through the device-plugin component, the GPU resource management policy can be customized according to actual needs, such as setting different reporting times and preset states. This makes the technical solution highly flexible and able to adapt to the GPU resource management requirements in different scenarios. In summary, the technical solution for implementing GPU resource management using the device-plugin component has beneficial effects such as real-time monitoring, periodic scheduling, optimized resource utilization, and flexible adaptation to different scenarios, which helps to improve the utilization rate of GPU resources and the task scheduling efficiency in the Kubernetes cluster.

[0114] In a specific embodiment, the total GPU resources and the remaining resources of each node are counted and saved. Counting and saving the total GPU resources and the remaining resources of each node specifically includes selecting a node and obtaining the GPU information reported by the device-plugin on the node, including the model of each GPU card, the maximum number of virtualized vGPUs that support splitting, the video card memory, the video card computing power, the video card health status, and the video card information reporting time. When the scheduler starts a new round of scheduling cycle every 3s, it is judged that if the difference between the device-plugin reporting time and the current time is greater than 60s, then all the GPU cards on the node are considered to be in the unavailable state, that is, there is no video card resource on the node. When the video card reporting time is normal, it is further judged whether each GPU card on the node is in the Ready or Healthy ready state. If it is in the unready NotReady or Unhealthy state, the GPU card is considered to be in the unavailable state. Traverse and count all the GPU card information, and store the total available GPU information in the scheduling cache.

[0115] See Figure 3 , Figure 3The flowchart shows a method for obtaining remaining resource information provided by an embodiment of the present application.

[0116] In some optional embodiments, obtaining the remaining resource information based on the total available resource information and saving it to the scheduling buffer includes:

[0117] Step S301: Obtain the GPU index and GPU usage amount corresponding to the applications on each node;

[0118] Step S302: Based on the GPU index and GPU usage amount corresponding to the applications on each node, obtain the used resource information;

[0119] Step S303: Based on the total available resource information, remove the used resource information from the scheduling buffer to obtain the remaining resource information and save it to the scheduling buffer.

[0120] Thus, it is possible to detail the GPU usage of applications on each node, including the used GPU index and usage amount, which helps to more accurately understand the usage of GPU resources in the cluster. Based on the GPU index and GPU usage amount corresponding to the applications on each node, the used resource information can be obtained. This helps to more accurately calculate the used resources during resource scheduling and avoid duplicate resource allocation. When obtaining the remaining resource information, the used resource information can be removed from the scheduling buffer to obtain the remaining resource information and save it to the scheduling buffer. This helps to better understand the remaining resources in the cluster, so as to achieve optimal resource allocation during task scheduling and improve the utilization rate of cluster resources. Through refined resource monitoring and optimized resource scheduling, it can be ensured that tasks can be efficiently executed in the cluster and the task execution efficiency can be improved. In summary, the technical solution of obtaining the GPU index and GPU usage amount corresponding to the applications on each node has beneficial effects such as refined management, accurate calculation of used resources, optimized resource scheduling, and improved task execution efficiency, which helps to improve the resource management efficiency and task execution efficiency of the cluster.

[0121] In a specific embodiment, the applications for which the GPU usage amount is counted include the applications that have been allocated GPU resources on the node and are not in the Succeed and Failed states. According to the index and usage amount of the GPU cards declared in the applications, the corresponding used resources are subtracted in the scheduling cache to obtain the remaining GPU resources of the node.

[0122] In some optional embodiments, obtaining the computing power data and total video memory demand information of the GPUs of each node based on hybrid quantization calculation includes:

[0123] Respectively determine whether the applications on each node have GPU requirements;

[0124] If the application on a certain node has GPU requirements, calculate and save the GPU index corresponding to the application on this node; if the application on a certain node has no GPU requirements, do nothing to the application on this node.

[0125] Judge whether the GPU requirement is in the whole-card direct-passing mode. If the GPU requirement is in the whole-card direct-passing mode, obtain the computing power data and total video memory requirement information of the GPU of the node through multiplication operation based on the GPU index.

[0126] If the GPU requirement is not in the whole-card direct-passing mode, obtain the computing power data and total video memory requirement information of the GPU of the node through addition operation based on the GPU index.

[0127] Thus, it is judged whether the applications on each node need GPU resources. For applications with GPU requirements, their corresponding GPU indexes will be calculated and saved, while for applications without GPU requirements, nothing will be done. In this way, resource management can be carried out according to the personalized requirements of the applications, improving the resource utilization efficiency. For applications with GPU requirements, it will be further judged whether they need the whole-card direct-passing mode. If the requirement is in the whole-card direct-passing mode, the computing power data and total video memory requirement information of the GPU of the node will be obtained through multiplication operation based on the GPU index; if the requirement is not in the whole-card direct-passing mode, the corresponding information will be obtained through addition operation. In this way, the requirements of the applications on each node for GPU resources can be accurately calculated, providing accurate data support for resource scheduling. The technical solution can adapt to the GPU resource requirements in different scenarios. Whether it is the whole-card direct-passing mode or other modes, the required resource information can be obtained through corresponding operations, with strong flexibility.

[0128] In a specific embodiment, hybrid quantization calculation and simple preliminary screening are performed. The hybrid quantization calculation method specifically includes determining whether there is GPU resources declared in the application and whether it is in a non-Succeed and Failed state, and no GPU resources are allocated on the Annotation of the application; if the above conditions are met, the index of the container with GPU requirements in the application is counted and saved; if there is no GPU requirement, all subsequent calculations are skipped; according to the above GPU container index, each container in the application is traversed; the container requirement is in the whole-card direct-through mode, and the total GPU computing power requirement of the container is the result of multiplying the total computing power of a whole card (the total computing power of one card can be regarded as 100) by the number N of applied GPU cards, obtaining the total GPU requirement 100*N required by the container; if the container is in the virtualization mode, the total GPU computing power requirement of the container is equal to the product of the computing power V applied by the container (the range of V is (0, 100]) and the number N of applied GPU cards, and the total video memory requirement of the container is the product of the video memory applied by the container and the number of applied GPU cards, and the result is V*N. Save the total GPU computing power and total video memory requirements of the container; save the requirements of each container into the Map structure, where the key value of the Map structure is the container index, and the value value is the serialized GPU computing power requirement and video memory requirement required by the container. At the same time, add the requirements of all containers (including GPU direct-through and virtualization requirements) in the application to obtain the total GPU computing power and total video memory of the application;

[0129] In some alternative embodiments, the first screening of the nodes based on the total video memory requirement information of the GPUs of the nodes to obtain a first screening result includes:

[0130] Compare the total video memory requirement information of the GPUs of the node with the remaining resource information;

[0131] If the total video memory requirement information of the GPUs of the node is less than the remaining resource information, it is considered that the resources of the node are insufficient and the node is filtered;

[0132] If the total video memory requirement information of the GPUs of the node is not less than the remaining resource information, save the node to obtain the first screening result.

[0133] Therefore, if the total GPU video memory requirement information of a node is less than the remaining resource information, it means that the resources of this node are insufficient to meet the new task requirements. In this case, this node can be filtered out to avoid scheduling tasks to nodes with insufficient resources, thereby improving the success rate of tasks. If the total GPU video memory requirement information of a node is not less than the remaining resource information, it indicates that the resources of this node can meet the new task requirements. In this case, this node will be saved to become the first screening result. In this way, it can ensure that nodes with sufficient resources are fully utilized, improving the resource utilization rate of the cluster. By comparing the total GPU video memory requirement information of a node with the remaining resource information, it is possible to quickly determine which nodes have sufficient resources to execute new tasks. This helps to improve the task scheduling efficiency and reduce task failures caused by insufficient resources. It can adapt to GPU resource requirements in different scenarios. Whether it is video memory requirements or other requirements, it is possible to determine whether the resources of a node are sufficient through comparison operations, with strong flexibility.

[0134] In a specific embodiment, the total GPU computing power and total video memory required by the application are compared with the total remaining resources of each node in the scheduling cache. If the remaining GPU resources of this node are smaller than the resources required by the application, it can be directly considered that the resources of this node are insufficient, and this node is filtered out; if the remaining resources of the node are sufficient, the screening continues.

[0135] See Figure 4 , Figure 4 which shows a schematic flowchart of a process for obtaining a reduced two-dimensional permutation array provided by an embodiment of the present application.

[0136] In some optional implementation manners, obtaining the reduced two-dimensional permutation array based on the permutation combination function and the pruning function includes:

[0137] Step S401: Input preset application information to obtain an index array with GPUs;

[0138] Step S402: Input the index array into a preset permutation function to obtain a permutation two-dimensional array;

[0139] Step S403: Determine whether the computing power data of the GPU and the video memory data of the GPU are equal;

[0140] Step S404: If the computing power data of the GPU and the video memory data of the GPU are equal, mark the index array of the GPU to obtain an equivalent array;

[0141] Step S405: Input the permutation two-dimensional array and the equivalent array into a preset pruning function together to obtain the reduced two-dimensional permutation array.

[0142] Thus, by inputting preset application information, an index array with GPUs can be obtained according to the specific requirements of the application, thereby realizing personalized resource management for different applications. By inputting the index array into a preset permutation function, a two-dimensional permutation array can be obtained, so that all possible GPU resource allocation schemes can be comprehensively considered during task scheduling, improving resource utilization and task scheduling efficiency. After determining whether the computing power data of the GPU is equal to the video memory data of the GPU, if they are equal, the index array of the GPU is marked to obtain an equivalent array. Then, the two-dimensional permutation array and the equivalent array are input into a preset pruning function together to obtain a refined two-dimensional permutation array. In this way, precise pruning can be performed to remove GPU resource allocation schemes that do not meet the conditions, thereby improving task scheduling efficiency and resource utilization. The technical solution of inputting preset application information, obtaining an index array with GPUs, and then performing permutation and pruning operations has beneficial effects such as personalized resource management, permutation optimization, precise pruning, and adaptation to different scenarios, which helps to improve the resource management efficiency and task execution efficiency of the cluster.

[0143] In a specific embodiment, the permutation combination and pruning, and the permutation combination and pruning method specifically include: inputting preset application information to obtain an index array of the container where the GPU requirement is located; using the index array as an input parameter to the permutation function to obtain a two-dimensional permutation array; judging the computing power and video memory applied for by each container. If the computing power and video memory applied for by one container are the same as those of other containers, it can be regarded as an "equivalent container", and each "equivalent container" in the container index array is marked to obtain an "equivalent" array. The two-dimensional permutation array and the "equivalent" array are input into the pruning function together to remove the duplicate permutations of the two-dimensional permutation array and obtain a pruned refined two-dimensional permutation array.

[0144] In some optional embodiments, the second screening operation on the node based on the first screening result to obtain a preferred node result includes:

[0145] Based on the first screening result, performing a comparison and screening on the refined two-dimensional permutation array to obtain a comparison and screening result;

[0146] Based on the comparison and screening result, respectively performing a scoring and preference selection on the node to obtain a preferred node result.

[0147] Thus, based on the first screening result, the refined two-dimensional full permutation array is compared and screened to obtain the comparison and screening result. In this way, nodes that meet the conditions can be accurately screened out, improving the task scheduling efficiency and resource utilization rate. Based on the comparison and screening result, the nodes are scored and optimized to obtain the optimized node result. In this way, according to the resource status and task requirements of the nodes, the best execution node can be selected for the task, improving the task execution efficiency and cluster resource utilization rate. By accurately screening and optimizing node selection, it can be ensured that the task can be executed efficiently and smoothly in the cluster, improving the task execution effect.

[0148] In a specific embodiment, the comparison and screening specifically includes: taking out the i-th container index array A in the pruned refined two-dimensional full permutation array, initializing the GPU usage flag bit M to 0, the structure BindMap for binding the container to the GPU card (the key value is the container index, and the value is the index of the node matching the GPU card), and recording the required number of GPUs for the application as X. Then, making a copy of the GPU margin R of the node resources, taking out the data in the container index array A, which is the container index I; comparing the margin of the n-th GPU in the GPU margin resources of the node with the computing power and video memory resources declared by the container. If the GPU margin resources are insufficient, skip this card and go to the next card; if the margin is sufficient after comparison, subtract the computing power and video memory applied by the container from the copied node resources R, increment the value of the flag bit M by 1, and record the key-value pair of the container index I and the GPU index n into BindMap. Once the value of the flag bit M is equal to the required number of GPUs X, it is considered that the scheduling resources match successfully, return BindMap, and exit the loop; traverse all GPU cards. If M is equal to X, directly return BindMap; otherwise, clear BindMap and enter the (i + 1)-th container index array B until the last group of container index arrays is traversed; since the result is returned within the loop when the scheduling resources match successfully, the final result is a match failure result, that is, directly return the emptied BindMap.

[0149] The steps of scoring and prioritizing specifically include: using the obtained BindMap above to read the stored data; if the length of the BindMap data is exactly 1, it proves that only one node meets the conditions, so directly select this node to bind with the application; if the length of the BindMap data is greater than or equal to 2, there are multiple nodes that meet the conditions. Considering different characteristics of the nodes, including indicators such as the resources already used by the node, the remaining resources, and the number of bound applications as the scoring criteria, list the node with the highest score (if multiple nodes have the same highest score, randomly select one of them) as the optimized node. Perform the binding operation between the application and the prioritized node, and the scheduling process for this application ends.

[0150] Device Embodiment

[0151] An embodiment of the present application further provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above methods are implemented. Its specific implementation manners are the same as those described in the above method embodiments and achieve the same technical effects, and some contents will not be repeated.

[0152] An embodiment of the present application further provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor is configured to implement the following steps when executing the computer program:

[0153] S1: Obtain the resource information of the GPUs of each node and save it to the scheduling buffer. The resource information includes total available resource information and remaining resource information;

[0154] S2: Based on hybrid quantization calculation, obtain the computing power data and total video memory requirement information of the GPUs of each node;

[0155] S3: Based on the total video memory requirement information of the GPUs of each node, perform a first screening on the nodes to obtain a first screening result;

[0156] S4: Based on the full permutation combination function and pruning function, obtain a reduced two-dimensional full permutation array;

[0157] S5: Based on the first screening result, perform a second screening operation on the nodes to obtain a preferred node result.

[0158] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is used to obtain and save the resource information of the GPUs of each node. The resource information includes total available resource information and remaining resource information:

[0159] Use the device-plugin component to obtain the attribute information of the GPUs corresponding to each node. The attribute information of the GPUs includes GPU information reporting time and GPU status information;

[0160] When the periodic scheduler starts scheduling, determine whether the GPU information reporting time is greater than the preset reporting time; if the GPU information reporting time is greater than the preset reporting time, set the GPU to an unavailable state;

[0161] If the GPU information reporting time is not greater than the preset reporting time, determine whether the GPU status information is in a ready state; if the GPU status information is not in a ready state, set the GPU to an unavailable state;

[0162] If the GPU status information is in a ready state, traverse the attribute information of the GPU to obtain the total available resource information and save it to the scheduling buffer;

[0163] Based on the total available resource information, obtain the remaining resource information and save it to the scheduling buffer.

[0164] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to obtain the remaining resource information based on the total available resource information and save it to the scheduling buffer:

[0165] Obtain the GPU index and GPU usage corresponding to the applications on each node;

[0166] Based on the GPU index and GPU usage corresponding to the applications on each node, obtain the used resource information;

[0167] Based on the total available resource information, remove the used resource information from the scheduling buffer to obtain the remaining resource information and save it to the scheduling buffer.

[0168] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to obtain the computing power data and total video memory requirement information of the GPUs on each node based on hybrid quantization calculation:

[0169] Respectively determine whether the applications on each node have GPU requirements;

[0170] If an application on a certain node has GPU requirements, calculate and save the GPU index corresponding to the application on that node; if an application on a certain node has no GPU requirements, do nothing to the application on that node;

[0171] Determine whether the GPU requirement is in the whole-card direct pass-through mode. If the GPU requirement is in the whole-card direct pass-through mode, multiply based on the GPU index to obtain the computing power data and total video memory requirement information of the GPU of the node;

[0172] If the GPU requirement is not in the whole-card direct pass-through mode, add based on the GPU index to obtain the computing power data and total video memory requirement information of the GPU of the node.

[0173] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to perform a first screening on the nodes based on the total video memory requirement information of the GPUs on each node to obtain a first screening result:

[0174] Compare the total video memory requirement information of the GPU of the node with the remaining resource information;

[0175] If the total video memory requirement information of the GPU of the node is less than the remaining resource information, it is considered that the resources of the node are insufficient and the node is filtered;

[0176] If the total video memory requirement information of the GPU of the node is not less than the remaining resource information, the node is saved to obtain the first screening result.

[0177] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to obtain a reduced two-dimensional permutation array based on a full permutation combination function and a pruning function:

[0178] Input preset application information to obtain an index array with GPUs;

[0179] Input the index array into a preset full permutation function to obtain a two-dimensional full permutation array;

[0180] Determine whether the computing power data of the GPU is equal to the video memory data of the GPU;

[0181] If the computing power data of the GPU is equal to the video memory data of the GPU, mark the index array of the GPU to obtain an equivalent array;

[0182] Input the two-dimensional full permutation array and the equivalent array into a preset pruning function to obtain the reduced two-dimensional full permutation array.

[0183] In some alternative embodiments, when the processor is configured to execute the computer program, the following method is adopted to perform a second screening operation on the node based on the first screening result to obtain a preferred node result:

[0184] Based on the first screening result, perform a comparison screening on the reduced two-dimensional full permutation array to obtain a comparison screening result;

[0185] Based on the comparison screening result, perform a scoring and preference selection on the nodes respectively to obtain a preferred node result.

[0186] See Figure 5 , Figure 5 shows a structural framework diagram of an electronic device provided by an embodiment of the present application.

[0187] The electronic device includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0188] The memory 210 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 211 and / or cache memory 212, and may further include read-only memory (ROM) 213.

[0189] Among them, the memory 210 also stores a computer program, which can be executed by the processor 220, so that the processor 220 implements the steps of any of the above methods.

[0190] The memory 210 may further include utilities 214 having at least one program module 215. Such program modules 215 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0191] Correspondingly, the processor 220 may execute the above computer program and may also execute the utilities 214.

[0192] The processor 220 may employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0193] The bus 230 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure of multiple bus structures.

[0194] The electronic device can also communicate with one or more external devices 240 such as a keyboard, a pointing device, a Bluetooth device, etc., and can also communicate with one or more devices capable of interacting with the electronic device, and / or communicate with any device (such as a router, a modem, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication can be carried out through the input / output interface 250. Moreover, the electronic device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 260. The network adapter 260 can communicate with other modules of the electronic device through the bus 230. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0195] System Embodiment

[0196] See Figure 6 , Figure 6 which shows a schematic structural diagram of a scheduling system provided by an embodiment of the present application.

[0197] An embodiment of the present application also provides a scheduling system, which includes:

[0198] The above-mentioned electronic device.

[0199] Medium Embodiment

[0200] An embodiment of the present application also provides a chip, which stores a computer program. When the computer program is executed by a processor, it implements the steps of any one of the above methods. Its specific implementation manners are consistent with the implementation manners and the achieved technical effects recorded in the above method embodiments, and some contents will not be repeated.

[0201] See Figure 7 , Figure 7 which shows a schematic structural diagram of a program product provided by an embodiment of the present application.

[0202] The program product is used to implement any of the above methods. The program product can be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In the embodiments of the present application, the readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0203] The chip can include data signals propagated in the baseband or as part of a carrier wave, which carry the readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium that can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above. The program code for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as C language, Python language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, executed as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0204] This application is described from the perspectives of purpose of use, efficacy, progress, and novelty, and has met the functional enhancement and use requirements emphasized by the patent law. The above description and accompanying drawings of this application are only preferred embodiments of this application and do not limit this application thereby. Therefore, all those that are similar or identical to the structure, device, features, etc. of this application, that is, all equivalent substitutions or modifications made according to the scope of the patent application of this application, shall fall within the scope of protection of the patent application of this application.

Claims

1. A GPU direct pass-through and resource hybrid scheduling method, characterized in that The method includes: S1: Obtain the resource information of the GPUs of each node and save it to the scheduling buffer. The resource information includes total available resource information and remaining resource information; S2: Based on hybrid quantization calculation, obtain the computing power data and total video memory requirement information of the GPUs of each node, including: Determine whether each application on each node has a GPU requirement; If an application on a certain node has a GPU requirement, calculate the GPU index corresponding to the application on that node and save it; if an application on a certain node has no GPU requirement, do nothing to the application on that node; Determine whether the GPU requirement is in the whole-card direct-passing mode. If the GPU requirement is in the whole-card direct-passing mode, obtain the computing power data and total video memory requirement information of the GPU of that node through a multiplication operation based on the GPU index; If the GPU requirement is not in the whole-card direct-passing mode, obtain the computing power data and total video memory requirement information of the GPU of that node through an addition operation based on the GPU index; S3: Based on the total video memory requirement information of the GPUs of each node, perform a first screening on the nodes to obtain a first screening result; S4: Based on the full permutation combination function and pruning function, obtain a refined two-dimensional full permutation array, including: Input the preset application information to obtain an index array of the containers where the GPU requirements are located; use the index array as an input parameter to the full permutation function to obtain a two-dimensional full permutation array; judge the computing power and video memory applied for by each container. If the computing power and video memory applied for by one container are the same as those of other containers, it is regarded as an equivalent container, and mark each equivalent container in the container index array to obtain an equivalent array. Input the two-dimensional full permutation array and the equivalent array into the pruning function to remove the duplicate permutations in the two-dimensional full permutation array to obtain a pruned refined two-dimensional full permutation array; S5: Based on the first screening result, perform a second screening operation on the nodes to obtain a preferred node result, including based on the first screening result, performing a comparison screening on the refined two-dimensional full permutation array to obtain a comparison screening result; Based on the comparison screening result, score and select the nodes respectively to obtain a preferred node result.

2. The GPU direct pass-through and resource hybrid scheduling method according to claim 1, wherein The obtaining and saving of the resource information of the GPUs of each node, where the resource information includes total available resource information and remaining resource information, includes: Use the device-plugin component to obtain the attribute information of the GPUs corresponding to each node. The attribute information of the GPUs includes GPU information reporting time and GPU status information; When the periodic scheduler starts scheduling, determine whether the GPU information reporting time is greater than the preset reporting time; if the GPU information reporting time is greater than the preset reporting time, set the GPU to an unavailable state; If the GPU information reporting time is not greater than the preset reporting time, determine whether the GPU status information is in a ready state; if the GPU status information is not in a ready state, set the GPU to an unavailable state; If the GPU status information is in a ready state, traverse the attribute information of the GPU to obtain the total available resource information and save it to the scheduling buffer; Based on the total available resource information, obtain the remaining resource information and save it to the scheduling buffer.

3. The GPU direct pass-through and resource hybrid scheduling method according to claim 2, wherein The obtaining the remaining resource information based on the total available resource information and saving it to the scheduling buffer includes: Obtain the GPU indices and GPU usage amounts corresponding to the applications on each node; Based on the GPU indices and GPU usage amounts corresponding to the applications on each node, obtain the used resource information; Based on the total available resource information, remove the used resource information from the scheduling buffer to obtain the remaining resource information and save it to the scheduling buffer.

4. The GPU direct pass-through and resource hybrid scheduling method according to claim 1, wherein The first screening of the nodes based on the total video memory demand information of the GPUs of the nodes to obtain a first screening result includes: Compare the total video memory demand information of the GPUs of the nodes with the remaining resource information; If the total video memory demand information of the GPUs of the node is less than the remaining resource information, save the node to obtain the first screening result; If the total video memory demand information of the GPUs of the node is not less than the remaining resource information, consider that the resources of the node are insufficient and filter the node.

5. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program. When the processor is configured to execute the computer program, the following steps are implemented: S1: Obtain the resource information of the GPUs of each node and save it to the scheduling buffer. The resource information includes total available resource information and remaining resource information; S2: Based on hybrid quantization calculation, obtain the computing power data and total video memory demand information of the GPUs of each node, including: Respectively determine whether the applications on each node have GPU requirements; If the application on a certain node has GPU requirements, calculate the GPU index corresponding to the application on the node and save it; if the application on a certain node has no GPU requirements, do not perform any operation on the application on the node; Determine whether the GPU requirement is in the whole-card direct-through mode. If the GPU requirement is in the whole-card direct-through mode, obtain the computing power data and total video memory demand information of the GPU of the node through a multiplication operation based on the GPU index; If the GPU requirement is not in the whole-card direct-through mode, obtain the computing power data and total video memory demand information of the GPU of the node through an addition operation based on the GPU index; S3: Based on the total video memory demand information of the GPUs of each node, perform a first screening on the nodes to obtain a first screening result; S4: Based on the full permutation combination function and the pruning function, obtain a refined two-dimensional full permutation array, including: Input the preset application information to obtain an index array of the containers where the GPU requirements are located; use the index array as an input parameter to the permutation function to obtain a two-dimensional permutation array; judge the computing power and video memory applied for by each container. If the computing power and video memory applied for by one container are the same as those of other containers, they are regarded as equivalent containers, and the equivalent containers in the container index array are marked to obtain an equivalent array. Input the two-dimensional permutation array and the equivalent array into the pruning function to remove the duplicate permutations in the two-dimensional permutation array and obtain a pruned and refined two-dimensional permutation array. S5: Based on the first screening result, perform a second screening operation on the node to obtain a preferred node result, including performing a comparison and screening on the pruned two-dimensional permutation array based on the first screening result to obtain a comparison and screening result. Based on the comparison and screening result, score and select the nodes respectively to obtain a preferred node result.

6. A scheduling system, characterized in that, The scheduling system includes: The electronic device according to claim 5.

7. A chip, characterized in that, The chip stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images