Resource scheduling method and device, electronic equipment and storage medium
By virtualizing and partitioning the GPU card on the GPU host and employing a two-level scoring strategy, the problem of wasted GPU computing power in existing technologies is solved, enabling multiple tasks to be executed in parallel on the same GPU card, thereby improving the utilization rate of the GPU card.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE INFORMATION TECHNOLOGY CO LTD
- Filing Date
- 2022-03-15
- Publication Date
- 2026-04-24
AI Technical Summary
Existing GPU virtualization methods are complex to design, suffer from significant hardware performance loss, and have poor versatility. As a result, many AI services cannot fully utilize the computing power of the entire GPU card, leading to a waste of GPU card computing power.
After virtualizing and partitioning the GPU cards installed on the GPU host, sub-GPU resources are obtained. A two-level scoring strategy is adopted. First, the GPU cards in the GPU host are scored to determine the GPU host to be scheduled. Then, the sub-GPU resources on the GPU host to be scheduled are scored. Combined with resource request requests, the sub-GPU resources to be scheduled are determined. Finally, the tasks to be executed are assigned to the most suitable GPU cards.
This enables multiple tasks to be executed in parallel on the same GPU card, reducing costs, improving GPU card utilization, and avoiding waste of GPU card computing power.
Smart Images
Figure CN116795520B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of virtualization technology, specifically to a resource scheduling method, apparatus, electronic device, and storage medium. Background Technology
[0002] With the rise of cloud computing, machine learning, and artificial intelligence, graphics processing units (GPUs) have become essential acceleration hardware.
[0003] GPUs on the market generally come with hardware virtualization functionality, but GPU server applications with built-in hardware virtualization are very expensive. They require the purchase of computing cards that support this virtualization technology, which are very expensive, and the upgrade cost is too high for users who already own older computing cards.
[0004] Furthermore, current GPU virtualization methods are complex to design, suffer significant hardware performance loss, and have poor versatility, resulting in many AI services not being able to fully utilize the computing power of the entire GPU card, thus wasting GPU computing power. Summary of the Invention
[0005] This application provides a resource scheduling method, apparatus, electronic device, and storage medium to address the technical problem that many artificial intelligence services cannot fully utilize the computing power of the entire GPU card, resulting in wasted GPU card computing power.
[0006] In a first aspect, embodiments of this application provide a resource scheduling method, including:
[0007] Identify the resource request for the task to be executed;
[0008] The GPU host to be scheduled is determined based on the first resource score of the GPU card in the GPU host, where the first resource score is determined based on the resource usage of the GPU card.
[0009] Based on the resource request and the second resource score of the sub-GPU resources on the GPU host to be scheduled, the sub-GPU resources to be scheduled are determined. The sub-GPU resources are obtained by virtualizing and dividing the GPU cards installed on the GPU host. The second resource score is determined based on the resource usage of the sub-GPU resources.
[0010] The GPU card corresponding to the sub-GPU resource to be scheduled is used as the scheduling node to execute the task to be executed.
[0011] According to the resource scheduling method provided in the embodiments of this application, the second resource score is determined based on the following steps:
[0012] Obtain the second used resources and the second total resources from the resource usage status of the sub-GPU resources;
[0013] The second resource score is determined based on the second used resources and the second total resources.
[0014] According to the resource scheduling method provided in the embodiments of this application, the first resource score is determined based on the following steps:
[0015] Obtain the first used resources and the first total resources from the resource usage status of the GPU card;
[0016] The first resource score is determined based on the first used resources and the first total resources.
[0017] According to the resource scheduling method provided in the embodiments of this application, determining the sub-GPU resource to be scheduled based on the resource request and the second resource score of the sub-GPU resource on the GPU host to be scheduled includes:
[0018] The sub-GPU resources are sorted in descending order of the second resource score;
[0019] The sub-GPU resources to be scheduled are selected sequentially from the sub-GPU resources until the amount of idle resources of the sub-GPU resources to be scheduled is greater than or equal to the amount of resources requested in the resource request request.
[0020] According to the resource scheduling method provided in the embodiments of this application, the sub-GPU resources are obtained through the following steps:
[0021] The GPU host machine is installed into the containerized application deployment cluster to obtain the GPU node in the containerized application deployment cluster;
[0022] Based on the extended resource plugin, the resources of the GPU node are divided and registered into the containerized application deployment cluster to obtain the sub-GPU resources.
[0023] According to the resource scheduling method provided in the embodiments of this application, the step of using the GPU card corresponding to the sub-GPU resource to be scheduled as a scheduling node to execute the task to be executed further includes:
[0024] Based on the resource monitoring component, monitor the sub-GPU resource usage of the task to be executed;
[0025] If the usage of sub-GPU resources exceeds the requested amount, an alarm will be issued based on the resource monitoring component.
[0026] According to the resource scheduling method provided in the embodiments of this application, the step of monitoring the sub-GPU resource usage of the task to be executed based on the resource monitoring component includes:
[0027] Obtain the process ID of the task to be executed;
[0028] Based on the process ID, query the sub-GPU resource usage of the task to be executed process.
[0029] Secondly, embodiments of this application provide a resource scheduling apparatus, including: a request determination unit, a host determination unit, a resource determination unit, and a GPU card scheduling unit. The request determination unit is used to determine resource request needs for tasks to be executed.
[0030] The host determination unit is used to determine the GPU host to be scheduled based on the first resource score of the GPU card in the GPU host, wherein the first resource score is determined based on the resource usage of the GPU card;
[0031] The resource determination unit is used to determine the sub-GPU resources to be scheduled based on the resource request and the second resource score of the sub-GPU resources on the GPU host to be scheduled. The sub-GPU resources are obtained by virtualizing and dividing the GPU cards installed on the GPU host. The second resource score is determined based on the resource usage of the sub-GPU resources.
[0032] The GPU card scheduling unit is used to use the GPU card corresponding to the sub-GPU resource to be scheduled as a scheduling node to execute the task to be executed.
[0033] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the resource scheduling method described in the first aspect.
[0034] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the resource scheduling method described in the first aspect.
[0035] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the resource scheduling method described in the first aspect.
[0036] The resource scheduling method, apparatus, electronic device, and storage medium provided in this application virtualize and divide the GPU card installed on the GPU host to obtain sub-GPU resources, thereby enabling multiple tasks to be executed in parallel on the same GPU card and reducing costs.
[0037] In addition, a two-level scoring strategy is adopted for resource scheduling. First, the GPU cards in the GPU host are scored to determine the GPU host to be scheduled. Then, the sub-GPU resources on the GPU host to be scheduled are scored, and combined with the resource request, the sub-GPU resources to be scheduled are determined. Then, the GPU card to execute the task is determined based on the sub-GPU resources to be scheduled. This can allocate the task to the most suitable GPU card, thereby further improving the utilization of GPU cards and avoiding the waste of GPU card computing power. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 This is one of the flowcharts illustrating the resource scheduling method provided in the embodiments of this application;
[0040] Figure 2 This is a flowchart illustrating the second resource score determination method provided in an embodiment of this application;
[0041] Figure 3 This is a flowchart illustrating the first resource score determination method provided in an embodiment of this application;
[0042] Figure 4 This is a flowchart illustrating step 130 in the resource scheduling method provided in this application embodiment;
[0043] Figure 5 This is a flowchart illustrating the sub-GPU resource determination method provided in an embodiment of this application;
[0044] Figure 6 This is one of the flowcharts illustrating the sub-GPU resource monitoring method provided in the embodiments of this application;
[0045] Figure 7 This is a second flowchart illustrating the sub-GPU resource monitoring method provided in this application embodiment;
[0046] Figure 8 This is a schematic diagram of the structure of the resource scheduling device provided in the embodiments of this application;
[0047] Figure 9 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0049] Currently, GPU virtualization methods are complex to design, suffer from significant hardware performance loss, and have poor versatility. As a result, many AI services cannot fully utilize the computing power of the entire GPU card, leading to a waste of GPU computing power.
[0050] Based on this, embodiments of this application provide a resource scheduling method to improve GPU resource utilization and avoid wasting GPU computing power.
[0051] Figure 1 This is one of the flowcharts illustrating the resource scheduling method provided in this application. This method can be applied to distributed computing clusters, such as the Kubernetes platform. Further, the Kubernetes platform can be a laptop computer, a virtual machine from a cloud service provider, a bare-metal server on a rack, etc., and this application does not limit the scope of the application. (Refer to...) Figure 1 This application provides a resource scheduling method, which may include:
[0052] Step 110: Determine the resource request for the task to be executed.
[0053] Specifically, the task to be executed is the task that requires resource scheduling. For example, the task to be executed may be the training and inference prediction task of a deep learning model. Such tasks often require a large number of GPUs to support larger models and datasets.
[0054] The resource request for a task to be executed can carry various information, such as the task number, business type, and the requested GPU, memory, or CPU resources. It should be noted that the resource scheduling method provided in this embodiment primarily targets GPU resources.
[0055] The requested GPU resources may include GPU capacity, GPU level, or GPU type, etc., without specific limitations, and can be flexibly selected according to actual needs. For example, the task to be executed requires a GPU card of model A and a capacity of 1GB to run normally.
[0056] Understandably, in order to improve GPU resource utilization and avoid wasting GPU computing power, it is necessary to schedule the most suitable GPU resources for each resource request of a task to be executed.
[0057] Step 120: Determine the GPU host to be scheduled based on the first resource score of the GPU card in the GPU host. The first resource score is determined based on the resource usage of the GPU card.
[0058] Specifically, a GPU host machine refers to a machine equipped with GPU cards. Each host machine can contain at least one GPU card. Each GPU card used on the GPU host machine can be scored based on its resource usage, thus obtaining a first resource score.
[0059] The first resource score represents the probability that the GPU host machine hosting the GPU card will be the machine executing the task. The higher the first resource score, the greater the probability that the GPU host machine hosting the GPU card will be the machine executing the task, and the more suitable it is as a GPU host machine to be scheduled; the lower the first resource score, the lower the probability that the GPU host machine hosting the GPU card will be the machine executing the task, and the less suitable it is as a GPU host machine to be scheduled.
[0060] Furthermore, the GPU host machine containing the GPU card with the highest resource score can be used as the GPU host machine to be scheduled.
[0061] Specifically, the resource usage of the GPU card can include the used capacity and remaining capacity of the GPU card.
[0062] Step 130: Based on the resource request and the second resource score of the sub-GPU resources on the GPU host to be scheduled, determine the sub-GPU resources to be scheduled. The sub-GPU resources are obtained by virtualizing and dividing the GPU cards installed on the GPU host. The second resource score is determined based on the resource usage of the sub-GPU resources.
[0063] Specifically, considering that in most cases the GPU demand for the task to be executed cannot fully utilize the computing power of the entire card, and that the cost of a single GPU card is high, the computing power of the GPU card is wasted, resulting in increased costs.
[0064] Based on this, this application virtualizes and partitions the GPU cards installed on the GPU host machine. Virtualization partitioning involves splitting the single-card-dimensional GPU into a memory-dimensional GPU. Specifically, it means dividing the GPU resources of each GPU card into different sub-regions, each sub-region containing a sub-GPU resource. Each sub-GPU resource can be used to execute a single task, thus enabling the parallel execution of multiple tasks on the same GPU. Here, the sub-GPU resource refers to the memory-dimensional GPU.
[0065] Understandably, after virtualizing and partitioning the GPU card installed on the GPU host, multiple sub-GPU resources are obtained. Then, each sub-GPU resource can be scored based on its resource usage, resulting in a second resource score. Specifically, the resource usage of each sub-GPU resource can include used resources and remaining resources.
[0066] The resulting second resource score characterizes the probability that the sub-GPU resource will be used as a scheduled sub-GPU resource, where the scheduled sub-GPU resource is used to execute the task. A higher second resource score indicates a higher probability of using the sub-GPU resource to execute the task; conversely, a lower second resource score indicates a lower probability of using the sub-GPU resource to execute the task.
[0067] Then, by combining the requested GPU capacity and the second resource score, the sub-GPU resources to be scheduled can be determined. Provided the requested GPU capacity is met, the sub-GPU resources with the higher second resource score are selected as the sub-GPU resources to be scheduled.
[0068] Step 140: The GPU card corresponding to the sub-GPU resource to be scheduled is used as the scheduling node to execute the task to be executed.
[0069] Specifically, once the sub-GPU resources to be scheduled are determined, the GPU card corresponding to the sub-GPU resource can be identified, and the GPU card can be used as a scheduling node to execute the task to be executed.
[0070] It should be noted that there may be one or more sub-GPU resources to be scheduled, and the corresponding GPU cards may also be one or more. This application does not specifically limit this.
[0071] The resource scheduling method provided in this application virtualizes and divides the GPU card installed on the GPU host to obtain sub-GPU resources, thereby enabling multiple tasks to be executed in parallel on the same GPU card and reducing costs.
[0072] In addition, a two-level scoring strategy is adopted for resource scheduling. First, the GPU cards in the GPU host are scored to determine the GPU host to be scheduled. Then, the sub-GPU resources on the GPU host to be scheduled are scored, and combined with the resource request, the sub-GPU resources to be scheduled are determined. Then, the GPU card to execute the task is determined based on the sub-GPU resources to be scheduled. This can allocate the task to the most suitable GPU card, thereby further improving the utilization of GPU cards and avoiding the waste of GPU card computing power.
[0073] Based on the above embodiments, Figure 2This is a flowchart illustrating the second resource score determination method provided in an embodiment of this application. (Refer to...) Figure 2 The second resource score is determined based on the following steps:
[0074] Step 210: Obtain the second used resources and the second total resources from the resource usage status of the sub-GPU resources;
[0075] Step 220: Determine the second resource score based on the second used resources and the second total resources.
[0076] Specifically, the second resource score can characterize the probability that a sub-GPU resource is to be scheduled. Further, resource usage, idle resource quantity, resource utilization rate, or resource remaining rate can be used as scoring criteria. For example, the more idle resources a sub-GPU resource has, the greater the probability that it is to be scheduled; conversely, the fewer idle resources a sub-GPU resource has, the lower the probability.
[0077] Therefore, the resource usage of sub-GPU resources can be monitored and updated to obtain the second used resources and the second total resources. This monitoring can be real-time or scheduled.
[0078] Based on the second used resources and the second total resources, a preset algorithm is used to calculate the second resource score.
[0079] In one embodiment, the formula for calculating the second resource score is as follows:
[0080] Score2=mGPU(10*sum(requested) / capacity)
[0081] In the formula, mGPU() represents the second resource score function, sum(requested) represents the sum of the resources used by the sub-GPU resource, and capacity represents the total resources of the sub-GPU resource.
[0082] The method provided in this application determines a second resource score by using the second used resources and the second total resources in the resource usage of sub-GPU resources, thereby quickly and accurately finding GPU cards with relatively sufficient sub-GPU resources.
[0083] Based on any of the above embodiments Figure 3 This is a flowchart illustrating the first resource score determination method provided in an embodiment of this application. (Refer to...) Figure 3 The first resource score is determined based on the following steps:
[0084] Step 310: Obtain the first used resources and the first total resources from the resource usage status of the GPU card;
[0085] Step 320: Determine the first resource score based on the first used resources and the first total resources.
[0086] Specifically, the first resource score can characterize the probability that the GPU host machine hosting the GPU card will be a GPU host machine awaiting scheduling. Further, the scoring can be based on factors such as the GPU card's resource usage, idle resource amount, resource utilization rate, or resource remaining rate. For example, the more idle resources a GPU card has, the greater the probability that its host machine will be a GPU host machine awaiting scheduling; conversely, the fewer idle resources a GPU card has, the lower the probability that its host machine will be a GPU host machine awaiting scheduling.
[0087] Therefore, the resource usage of the GPU card can be monitored and updated to obtain the first used resources and the first total resources. This monitoring can be real-time or scheduled.
[0088] Based on the first used resource and the first total resource, the first resource score is calculated using a preset algorithm.
[0089] In one embodiment, the formula for calculating the first resource score is as follows:
[0090] Score1=GPU(10*sum(requested) / capacity)
[0091] In the formula, GPU() represents the first resource score function, sum(requested) represents the sum of resources used by the GPU card, and capacity represents the total resources of the GPU card.
[0092] It should be noted that both the second resource score and the first resource score can be implemented using an extended scheduler exposed by Kubernetes, such as mGPU SchedulerExtender.
[0093] The method provided in this application determines a first resource score by using the first used resources and the first total resources in the resource usage of the GPU card, thereby quickly and accurately finding a GPU host machine with relatively sufficient GPU card resources.
[0094] Based on any of the above embodiments Figure 4 This is a flowchart illustrating step 130 of the resource scheduling method provided in this application embodiment. (Refer to...) Figure 4 Step 130 specifically includes:
[0095] Step 131: Sort the sub-GPU resources according to the second resource score from high to low.
[0096] Step 132: Select the sub-GPU resources to be scheduled from the sub-GPU resources in sequence until the amount of idle resources of the sub-GPU resources to be scheduled is greater than or equal to the amount of resources requested in the resource request request.
[0097] Specifically, after obtaining the second resource score, the sub-GPU resources can be sorted from highest to lowest score to form a sub-GPU resource list. It's understandable that the higher the sub-GPU resource is ranked in the sub-GPU resource list, the higher its second resource score. Therefore, when determining the sub-GPU resources to be scheduled, they can be selected sequentially from the sub-GPU resource list.
[0098] If the available resources of the first sub-GPU resource in the current list can satisfy the request amount carried in the resource request, then the selection stops, and the first sub-GPU resource in the current list is determined as the sub-GPU resource to be scheduled. At this time, the number of sub-GPU resources to be scheduled is one.
[0099] If the available resources of the first sub-GPU resource in the current list are insufficient to meet the requested amount in the resource request, then the first sub-GPU resource in the current list will be designated as the sub-GPU resource to be scheduled, and the second sub-GPU resource in the current list will be selected.
[0100] If the sum of the free resources of the first and second sub-GPU resources can satisfy the request amount carried in the resource request, then the first and second sub-GPU resources in the current list are determined as sub-GPU resources to be scheduled. At this time, the number of sub-GPU resources to be scheduled is two.
[0101] If the sum of the idle resources of the first and second sub-GPU resources is insufficient to meet the request amount carried in the resource request, then the third sub-GPU resource in the current list is selected, until the idle resources of the sub-GPU resources to be scheduled are greater than or equal to the request amount carried in the resource request.
[0102] Based on any of the above embodiments Figure 5 This is a flowchart illustrating the sub-GPU resource determination method provided in an embodiment of this application. (Refer to...) Figure 5 The sub-GPU resources are obtained through the following steps:
[0103] Step 510: Install the GPU host machine into the containerized application deployment cluster to obtain the GPU node in the containerized application deployment cluster.
[0104] Step 520: Based on the extended resource plugin, the resources of the GPU node are divided and registered to the containerized application deployment cluster to obtain the sub-GPU resources.
[0105] Specifically, containerized application deployment clusters can be Kubernetes, or K8S for short, which is an open-source version of large-scale container management technology. It is a container cluster management system, an open-source platform that enables automated deployment, automatic scaling, rolling updates, and hot maintenance of container clusters. K8S features good portability, strong scalability, high automation, and effective resource saving.
[0106] The extended resource plugins here can be virtual GPU device plugins, such as the custom plugin mGPU Device Plugin implemented based on the device plugin mechanism provided by Kubernetes. The purpose of this plugin is to support fine-grained partitioning and scheduling of GPU resources.
[0107] First, all GPU host machines are installed into the Kubernetes cluster, and each GPU host machine becomes a GPU node in Kubernetes.
[0108] Install the extended resource plugin mGPU DevicePlugin on each Kubernetes GPU node.
[0109] The mGPU Device Plugin obtains the video memory information of all GPU cards on the host GPU machine, for example, in units of hundreds of megabytes. It converts the GPU card units to GPU video memory units, divides them into several GPU resources, and registers the divided GPU resources to each GPU node of the Kubernetes machine, i.e., sub-GPU resources.
[0110] Furthermore, the extended resource plugin mGPU Device Plugin obtains the usage status of the sub-GPU resources through the Kubernetes interface and updates the sub-GPU resource status, resources, and other information to the Kubernetes cluster. At this point, the addition of sub-GPU resources to each Kubernetes GPU node is successful.
[0111] Based on any of the above embodiments Figure 6 This is one of the flowcharts illustrating the sub-GPU resource monitoring method provided in this application embodiment. (Refer to...) Figure 6 Step 140 and subsequent steps include:
[0112] Step 610: Based on the resource monitoring component, monitor the sub-GPU resource usage of the task to be executed;
[0113] Step 620: If the sub-GPU resource usage exceeds the requested amount, issue an alarm based on the resource monitoring component.
[0114] Specifically, considering the visualization of sub-GPU resource monitoring within a Kubernetes cluster, the resource monitoring component can monitor the sub-GPU resource consumption of each task within the Kubernetes cluster periodically or in real time.
[0115] The resource monitoring component here can be a Kubernetes-based resource monitoring plugin that can collect the usage of sub-GPU resources within the Kubernetes cluster, process this data, and present it through a visualization interface, such as displaying the data intuitively in the form of images.
[0116] If the sub-GPU resource usage of a monitored task exceeds the GPU resource request limit, the resource monitoring component will issue an alert to prompt operations personnel to adjust GPU resources in a timely manner to prevent task execution failure. It should be noted that the resource request for the task to be executed includes the requested limit, which the resource monitoring component can obtain through the Kubernetes interface.
[0117] The method provided in this application embodiment monitors the usage of sub-GPU resources through a monitoring component and issues an alarm when the usage of sub-GPU resources exceeds the GPU resource request limit, thereby improving GPU resource utilization and task execution efficiency.
[0118] Based on any of the above embodiments Figure 7 This is the second flowchart illustrating the sub-GPU resource monitoring method provided in this application embodiment. (Refer to...) Figure 7 Step 610 specifically includes:
[0119] Step 611: Obtain the process ID of the task to be executed;
[0120] Step 612: Based on the process ID, query the sub-GPU resource usage of the task to be executed process.
[0121] Specifically, the resource monitoring component can obtain the process ID of the task to be executed through a data exchange interface, such as the Docker interface, and then query the sub-GPU resource usage of the process based on the obtained process ID through the Nvidia interface. Specifically, the query can be performed based on keyword matching.
[0122] Based on any of the above embodiments, this application provides a resource scheduling method, the method comprising:
[0123] S1 integrates all GPU host machines into the Kubernetes cluster, making them Kubernetes GPU nodes.
[0124] S2 registers sub-GPU resources to the Kubernetes cluster through its self-developed mGPU DevicePlugin, an extension resource plugin for Kubernetes. This means that the original GPU card resources of each Kubernetes GPU node are divided into several sub-GPU resources, thereby greatly expanding the resources of the original GPU card.
[0125] S3 uses its self-developed mGPU SchedulerExtender, an extended scheduler exposed to the public by Kubernetes, to achieve the first resource scoring of GPU cards in the GPU host and the second resource scoring of sub-GPU resources on the GPU host. Then, it registers the extended scheduler mGPUScheduler Extender to the Kubernetes cluster.
[0126] S4: When an AI service requests GPU resources from a Kubernetes cluster, the AI service will be scheduled on the best GPU card of the best GPU host in the Kubernetes cluster.
[0127] S5, considering the visualization of mGPU resource monitoring within the Kubernetes cluster, uses a self-developed resource monitoring component to periodically monitor the GPU resource consumption of each AI service sub-service within the Kubernetes cluster.
[0128] The resource scheduling apparatus provided in the embodiments of this application is described below. The resource scheduling apparatus described below can be referred to in correspondence with the resource scheduling method described above.
[0129] Based on any of the above embodiments Figure 8 This is a schematic diagram of the structure of the resource scheduling device provided in an embodiment of this application. (Refer to...) Figure 8 The resource scheduling device includes: a request determination unit 810, a host determination unit 820, a resource determination unit 830, and a GPU card scheduling unit 840. Among them,
[0130] The request determination unit 810 is used to determine the resource request of the task to be executed;
[0131] The host determination unit 820 is used to determine the GPU host to be scheduled based on the first resource score of the GPU card in the GPU host, wherein the first resource score is determined based on the resource usage of the GPU card;
[0132] The resource determination unit 830 is used to determine the sub-GPU resources to be scheduled based on the resource request and the second resource score of the sub-GPU resources on the GPU host to be scheduled. The sub-GPU resources are obtained by virtualizing and dividing the GPU cards installed on the GPU host. The second resource score is determined based on the resource usage of the sub-GPU resources.
[0133] The GPU card scheduling unit 840 is used to use the GPU card corresponding to the sub-GPU resource to be scheduled as a scheduling node to execute the task to be executed.
[0134] The resource scheduling device provided in this application embodiment virtualizes and divides the GPU card installed on the GPU host to obtain sub-GPU resources, thereby enabling multiple tasks to be executed in parallel on the same GPU card and reducing costs.
[0135] In addition, a two-level scoring strategy is adopted for resource scheduling. First, the GPU cards in the GPU host are scored to determine the GPU host to be scheduled. Then, the sub-GPU resources on the GPU host to be scheduled are scored, and combined with the resource request, the sub-GPU resources to be scheduled are determined. Then, the GPU card to execute the task is determined based on the sub-GPU resources to be scheduled. This can allocate the task to the most suitable GPU card, thereby further improving the utilization of GPU cards and avoiding the waste of GPU card computing power.
[0136] Based on any of the above embodiments, the resource scheduling apparatus provided in this application further includes a second resource score determination unit, used for:
[0137] Obtain the second used resources and the second total resources from the resource usage status of the sub-GPU resources;
[0138] The second resource score is determined based on the second used resources and the second total resources.
[0139] Based on any of the above embodiments, the resource scheduling apparatus provided in this application further includes a first resource score determination unit, used for:
[0140] Obtain the first used resources and the first total resources from the resource usage status of the GPU card;
[0141] The first resource score is determined based on the first used resources and the first total resources.
[0142] Based on any of the above embodiments, the resource determination unit 830 is further configured to:
[0143] The sub-GPU resources are sorted in descending order of the second resource score;
[0144] The sub-GPU resources to be scheduled are selected sequentially from the sub-GPU resources until the amount of idle resources of the sub-GPU resources to be scheduled is greater than or equal to the amount of resources requested in the resource request request.
[0145] Based on any of the above embodiments, the resource scheduling apparatus provided in this application further includes a sub-GPU resource creation unit, used for:
[0146] The GPU host machine is installed into the containerized application deployment cluster to obtain the GPU node in the containerized application deployment cluster;
[0147] Based on the extended resource plugin, the resources of the GPU node are divided and registered into the containerized application deployment cluster to obtain the sub-GPU resources.
[0148] Based on any of the above embodiments, the resource scheduling device provided in this application further includes a sub-GPU resource monitoring unit, used for:
[0149] Based on the resource monitoring component, monitor the sub-GPU resource usage of the task to be executed;
[0150] If the usage of sub-GPU resources exceeds the requested amount, an alarm will be issued based on the resource monitoring component.
[0151] Based on any of the above embodiments, the sub-GPU resource monitoring unit is further configured to:
[0152] Obtain the process ID of the task to be executed;
[0153] Based on the process ID, query the sub-GPU resource usage of the task to be executed process.
[0154] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 can call a computer program in the memory 930 to execute a resource scheduling method, which includes, for example: determining a resource request for a task to be executed; determining a GPU host to be scheduled based on a first resource score of a GPU card in the GPU host, wherein the first resource score is determined based on the resource usage of the GPU card; determining a sub-GPU resource to be scheduled based on the resource request and a second resource score of a sub-GPU resource on the GPU host to be scheduled, wherein the sub-GPU resource is obtained by virtualizing and partitioning the GPU card installed on the GPU host, and the second resource score is determined based on the resource usage of the sub-GPU resource; and using the GPU card corresponding to the sub-GPU resource to be scheduled as a scheduling node to execute the task to be executed.
[0155] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0156] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the resource scheduling methods provided in the above embodiments, such as: determining a resource request for a task to be executed; determining a GPU host to be scheduled based on a first resource score of a GPU card in a GPU host, wherein the first resource score is determined based on the resource usage of the GPU card; determining a sub-GPU resource to be scheduled based on the resource request and a second resource score of a sub-GPU resource on the GPU host to be scheduled, wherein the sub-GPU resource is obtained by virtualizing and dividing the GPU card installed on the GPU host, and the second resource score is determined based on the resource usage of the sub-GPU resource; and using the GPU card corresponding to the sub-GPU resource to be scheduled as a scheduling node to execute the task to be executed.
[0157] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing a processor to execute the resource scheduling methods provided in the above embodiments. For example, the program includes: determining a resource request for a task to be executed; determining a GPU host to be scheduled based on a first resource score of a GPU card in a GPU host, wherein the first resource score is determined based on the resource usage of the GPU card; determining a sub-GPU resource to be scheduled based on the resource request and a second resource score of a sub-GPU resource on the GPU host to be scheduled, wherein the sub-GPU resource is obtained by virtualizing and partitioning the GPU card installed on the GPU host, and the second resource score is determined based on the resource usage of the sub-GPU resource; and using the GPU card corresponding to the sub-GPU resource to be scheduled as a scheduling node to execute the task to be executed.
[0158] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).
[0159] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A resource scheduling method, characterized in that, include: Identify the resource request for the task to be executed; The GPU host to be scheduled is determined based on the first resource score of the GPU card in the GPU host, where the first resource score is determined based on the resource usage of the GPU card. Based on the resource request and the second resource score of the sub-GPU resources on the GPU host to be scheduled, the sub-GPU resources to be scheduled are determined. The sub-GPU resources are obtained by virtualizing and dividing the GPU cards installed on the GPU host. The second resource score is determined based on the resource usage of the sub-GPU resources. The GPU card corresponding to the sub-GPU resource to be scheduled is used as the scheduling node to execute the task to be executed.
2. The resource scheduling method according to claim 1, characterized in that, The second resource score is determined based on the following steps: Obtain the second used resources and the second total resources from the resource usage status of the sub-GPU resources; The second resource score is determined based on the second used resources and the second total resources.
3. The resource scheduling method according to claim 1, characterized in that, The first resource score is determined based on the following steps: Obtain the first used resources and the first total resources from the resource usage status of the GPU card; The first resource score is determined based on the first used resources and the first total resources.
4. The resource scheduling method according to claim 1, characterized in that, The process of determining the sub-GPU resources to be scheduled based on the resource request and the second resource score of the sub-GPU resources on the GPU host to be scheduled includes: The sub-GPU resources are sorted in descending order of the second resource score; The sub-GPU resources to be scheduled are selected sequentially from the sub-GPU resources until the amount of idle resources of the sub-GPU resources to be scheduled is greater than or equal to the amount of resources requested in the resource request request.
5. The resource scheduling method according to claim 1, characterized in that, The sub-GPU resources are obtained through the following steps: The GPU host machine is installed into the containerized application deployment cluster to obtain the GPU node in the containerized application deployment cluster; Based on the extended resource plugin, the resources of the GPU node are divided and registered into the containerized application deployment cluster to obtain the sub-GPU resources.
6. The resource scheduling method according to any one of claims 1-5, characterized in that, The step of using the GPU card corresponding to the sub-GPU resource to be scheduled as a scheduling node to execute the task to be executed further includes: Based on the resource monitoring component, monitor the sub-GPU resource usage of the task to be executed; If the usage of sub-GPU resources exceeds the requested amount, an alarm will be issued based on the resource monitoring component.
7. The resource scheduling method according to claim 6, characterized in that, The monitoring of sub-GPU resource usage for the task to be executed, based on the resource monitoring component, includes: Obtain the process ID of the task to be executed; Based on the process ID, query the sub-GPU resource usage of the task to be executed process.
8. A resource scheduling device, characterized in that, include: The request determination unit is used to determine the resource request for the task to be executed; The host determination unit is used to determine the GPU host to be scheduled based on the first resource score of the GPU card in the GPU host, wherein the first resource score is determined based on the resource usage of the GPU card; The resource determination unit is used to determine the sub-GPU resources to be scheduled based on the resource request and the second resource score of the sub-GPU resources on the GPU host to be scheduled. The sub-GPU resources are obtained by virtualizing and dividing the GPU cards installed on the GPU host. The second resource score is determined based on the resource usage of the sub-GPU resources. The GPU card scheduling unit is used to use the GPU card corresponding to the sub-GPU resource to be scheduled as a scheduling node to execute the task to be executed.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the resource scheduling method as described in any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the resource scheduling method as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Cloud platform and virtual machine scheduling method and device based on cloud platform
CN109960565A
CPU resource allocation method, device, and equipment and computer readable storage medium
CN111143035A