Scheduling method and device, electronic equipment and storage medium

By monitoring the congestion between the GPU and the system memory communication link, adjusting the scheduling priority and frequency according to the workload of the virtual machine, the response delay problem caused by link blockage in the GPU virtualization system is solved, and the number of response delays between virtual machines is improved. Isolation and GPU utilization.

CN119088508APending Publication Date: 2024-12-06MOORE THREADS TECHNOLOGY (SHANGHAI) CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411215940.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In GPU virtualization systems, when multiple virtual machines share PCIe links, heavy workload virtual machines may cause link blockage, interfere with light workload virtual machines, resulting in command response latency and GPU utilization decreases.

Method used

By monitoring the congestion of communication links between the target GPU and system memory, determine the scheduling priority of each virtual machine, adjust the scheduling order and frequency of access commands according to their workload conditions, so as to prioritize virtual machines with lighter workloads.

Benefits of technology

It improves the response speed of access commands to system memory for virtual machines with lighter workloads, reduces the interference of virtual machines with heavy workloads to lighter virtual machines, and improves isolation and GPU utilization between virtual machines.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088508A_ABST
    Figure CN119088508A_ABST
Patent Text Reader

Abstract

The invention relates to a scheduling method and device, electronic equipment and a storage medium, and is applied to a distributed virtualization system, the distributed virtualization system comprises a plurality of virtual machines, and the plurality of virtual machines are in one-to-one correspondence with a plurality of virtual graphics processors (vGPUs) formed by virtualizing a target GPU. The method can comprise the following steps: in response to congestion of a communication link between a target GPU (Graphic Processing Unit) and a system memory, determining a scheduling priority of each target virtual machine in a plurality of virtual machines; wherein the scheduling priority is in negative correlation with the workload of the virtual machine; the target virtual machine is a virtual machine currently accessing the system memory; and based on the scheduling priority of each target virtual machine, scheduling an access command of each target virtual machine to a system memory. According to the method and the device, the access command of the target virtual machine with light workload to the system memory is preferentially scheduled, so that the access command of the target virtual machine with light workload to the system memory can be timely responded.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a scheduling method, device, electronic device and storage medium. Background Art

[0002] Graphics processing unit (GPU) is very important in the fields of graphics rendering, parallel computing, artificial intelligence, etc. In GPU virtualization technology, in order to support multiple virtual machines (VM) using a GPU at the same time, the GPU hardware resources are divided into multiple parts to provide independent GPU hardware resources for each virtual machine.

[0003] When a virtual machine is allocated less video memory resources and has a heavy workload, the virtual machine can use the system memory through the Peripheral Component Interconnect Express (PCIe) link. However, since all virtual machines share a PCIe link, all virtual machine commands to access system memory are submitted in sequence; when multiple virtual machines access system memory through the PCIe link at the same time, the virtual machine with a heavier workload may cause the PCIe link to be blocked, thereby seriously interfering with the virtual machine with a lighter workload, resulting in the virtual machine's commands not being responded to in a timely manner, causing the GPU utilization of the virtual machine with a lighter workload to increase significantly. Summary of the invention

[0004] In view of this, the present disclosure proposes a scheduling method, device, electronic device, storage medium and computer program product.

[0005] According to one aspect of the present disclosure, a scheduling method is provided, which is applied to a distributed virtualization system, wherein the distributed virtualization system includes a plurality of virtual machines, wherein the plurality of virtual machines correspond one-to-one to a plurality of virtual graphics processors vGPU formed by virtualizing a target graphics processor GPU;

[0006] The method comprises:

[0007] In response to congestion in the communication link between the target GPU and the system memory, determining a scheduling priority of each target virtual machine among the multiple virtual machines; wherein the scheduling priority is negatively correlated with the workload of the virtual machine; and the target virtual machine is a virtual machine currently accessing the system memory;

[0008] Based on the scheduling priority of each target virtual machine, the access commands of each target virtual machine to the system memory are scheduled.

[0009] In a possible implementation, the method further includes:

[0010] Obtaining throughput statistics information of the communication link and / or congestion statistics information of the communication link;

[0011] In response to the throughput statistics information of the communication link and / or the congestion statistics information of the communication link exceeding a preset threshold, it is determined that congestion occurs on the communication link.

[0012] In a possible implementation manner, determining the scheduling priority of each target virtual machine among the multiple virtual machines includes:

[0013] Obtaining the operation information of each target virtual machine; wherein the operation information of each target virtual machine includes at least one of the following: the utilization rate of the vGPU corresponding to the target virtual machine, the size of the system memory used by the vGPU corresponding to the target virtual machine, and the average time taken by the target GPU to execute the access command of the target virtual machine to the system memory;

[0014] Based on the operation information of each target virtual machine, a scheduling priority of each target virtual machine is determined.

[0015] In a possible implementation, the throughput statistics information in the communication link includes: at least one of the amount of data transmitted on the communication link, the number of data packets transmitted on the communication link, and the average size of each data packet transmitted on the communication link; and / or, the congestion statistics information of the communication link includes: back pressure information of the target GPU and / or the quota of the system memory; wherein the back pressure information includes the value of the write stall register and / or the value of the read stall register in the target GPU, and the quota is used to indicate the amount of data that can be received by the system memory.

[0016] In a possible implementation, scheduling the access commands of the target virtual machines to the system memory based on the scheduling priorities of the target virtual machines includes:

[0017] Determine the scheduling order corresponding to each target virtual machine in order of scheduling priority;

[0018] The access commands of the target virtual machines to the system memory are scheduled according to the scheduling order corresponding to the target virtual machines.

[0019] In a possible implementation, scheduling the access commands of the target virtual machines to the system memory based on the scheduling priorities of the target virtual machines includes:

[0020] Based on the scheduling priority of each target virtual machine, adjusting the scheduling frequency corresponding to each target virtual machine;

[0021] The access commands of the target virtual machines to the system memory are scheduled according to the scheduling frequencies corresponding to the target virtual machines.

[0022] In a possible implementation, adjusting the scheduling frequency corresponding to each target virtual machine based on the scheduling priority of each target virtual machine includes:

[0023] For any target virtual machine among the target virtual machines, in response to a scheduling priority of the target virtual machine being lower than a preset priority, reducing a scheduling frequency of the target virtual machine.

[0024] According to another aspect of the present disclosure, a scheduling device is provided, which is applied to a distributed virtualization system, wherein the distributed virtualization system includes a plurality of virtual machines, wherein the plurality of virtual machines correspond one-to-one to a plurality of virtual graphics processors vGPU formed by virtualizing a target graphics processor GPU;

[0025] The device includes: a priority determination module, which is used to determine the scheduling priority of each target virtual machine among the multiple virtual machines in response to congestion in the communication link between the target GPU and the system memory; wherein the scheduling priority is negatively correlated with the workload of the virtual machine; the target virtual machine is the virtual machine currently accessing the system memory; and a scheduling module, which is used to schedule the access commands of each target virtual machine to the system memory based on the scheduling priority of each target virtual machine.

[0026] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0027] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0028] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0029] Through various aspects of the present disclosure, in response to congestion in the communication link between the target GPU and the system memory, the scheduling priority of each target virtual machine among the multiple virtual machines is determined; wherein the scheduling priority is negatively correlated with the workload of the virtual machine; the target virtual machine is the virtual machine currently accessing the system memory; based on the scheduling priority of each target virtual machine, the access commands of each target virtual machine to the system memory are scheduled. In this way, when the current workloads of multiple target virtual machines are uneven and the communication link between the target GPU and the system memory is blocked, a higher scheduling priority is configured for the target virtual machine with a lighter workload, so that it has more priority when using system memory resources, and the access commands of the target virtual machine with a lighter workload to the system memory are preferentially scheduled, that is, the access commands of the target virtual machine with a lighter workload to the system memory can be responded to in a timely manner; at the same time, the impact of the operation of the target virtual machine with a heavier workload on the target virtual machine with a lighter workload is reduced, and the isolation between the target virtual machines is improved.

[0030] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0032] Figure 1 A schematic diagram showing a distributed virtualization system according to an embodiment of the present disclosure is shown;

[0033] Figure 2 A flowchart of a scheduling method according to an embodiment of the present disclosure is shown;

[0034] Figure 3 A flowchart of a scheduling method according to an embodiment of the present disclosure is shown;

[0035] Figure 4 A flowchart showing an example of a scheduling method according to an embodiment of the present disclosure is shown;

[0036] Figure 5 A flowchart of a scheduling method according to an embodiment of the present disclosure is shown;

[0037] Figure 6 A structural diagram of a scheduling device according to an embodiment of the present disclosure is shown;

[0038] Figure 7 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0039] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0040] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present disclosure include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the phrases "exemplary", "in one embodiment", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0041] In the present disclosure, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or plural.

[0042] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0043] Figure 1 A schematic diagram of a distributed virtualization system according to an embodiment of the present disclosure is shown as follows: Figure 1As shown in the figure, the distributed virtualization system includes a client (GUEST) and a host (HOST); wherein the GPU of the host is virtualized into multiple vGPUs, namely vGPU1, vGPU2, vGPU3...vGPUn in the figure; certain video memory resources are allocated to each vGPU, and the video memory resources allocated to each vGPU are relatively independent, and the video memory resources allocated to different vGPUs can be the same or different. The client includes multiple virtual machines, namely virtual machine 1, virtual machine 2, virtual machine 3...virtual machine n in the figure, a total of n virtual machines; wherein these n virtual machines correspond one-to-one to the n vGPUs formed by virtualizing the GPU of the host.

[0044] Exemplarily, the host side is also configured with a graphics card driver, a GPU scheduler, etc. (not shown in the figure); wherein, the graphics card driver is configured with a scheduling module for generating a scheduling policy and sending the scheduling policy to the GPU scheduler; the GPU scheduler is responsible for scheduling the work of different hardware units on the GPU based on the scheduling policy. When the distributed virtualization system is working, the client can create tasks for multiple virtual machines, and each virtual machine can use the corresponding vGPU to independently process its own tasks. In this way, multiple virtual machines can use one GPU to process their own tasks at the same time, that is, multiple virtual machines can send commands to the GPU concurrently, and the GPU scheduler can identify commands from different virtual machines and schedule these commands based on the scheduling policy so that the GPU can execute them, thereby realizing the GPU concurrently processing the tasks of each virtual machine.

[0045] When the GPU is working, it can use local memory and system memory at the same time. Local memory and system memory play different roles in the memory hierarchy: local memory is a dedicated high-speed memory located on the GPU chip. Local memory provides low-latency and high-bandwidth access and is a key part of GPU computing. It can share data between different GPU cores and accelerate parallel computing operations; system memory is a storage space shared by the CPU and the GPU on the host side, which is used to store application data, code and other resources. Figure 1As shown in , the GPU can communicate with the system memory through the Peripheral Component Interconnect Express (PCIe) link to access the data in the system memory; compared with accessing the local video memory, the access speed of the system memory is slower because it needs to go through the PCIe link. Accordingly, when the distributed virtualization system is working, the virtual machine needs to access and store the relevant data of the current task. For example, when the virtual machine runs an application, it needs to access the data or code related to the application, and also needs to save the processing results and temporary data generated during the processing. For each virtual machine, the latency of accessing the local video memory is smaller and the bandwidth is larger, so each virtual machine usually gives priority to using the local video memory to store the relevant data of the current task. However, when the workload of the virtual machine does not match the size of the video memory resources allocated to the corresponding vGPU, that is, the vGPU corresponding to the virtual machine is allocated less video memory resources, and the workload of the virtual machine is heavier; the virtual machine can use the system memory through the PCIe link. For example, the virtual machine can store part of the relevant data of the current task in the system memory, so the virtual machine needs to frequently access the data and code stored in the system memory through the PCIe link. Since all virtual machines share a PCIe link, when multiple virtual machines access the system memory simultaneously through the PCIe link, the GPU scheduler schedules the access commands of multiple virtual machines to the system memory in the order in which the commands of the multiple virtual machines are submitted. However, when there are multiple virtual machines with heavy workloads accessing the system memory through the PCIe link at the same time, the PCIe link will be blocked. At this time, the virtual machines with heavier workloads among the multiple virtual machines seriously interfere with the virtual machines with lighter workloads, resulting in the virtual machines with lighter workloads not being able to respond in time, causing the GPU utilization of the virtual machines with lighter workloads to increase significantly.

[0046] In order to solve the above technical problems, the present disclosure provides a scheduling method (described in detail below), which can be applied to Figure 1In the distributed virtualization system shown, in response to congestion in the communication link between the target GPU and the system memory, the scheduling priority of each target virtual machine among the multiple virtual machines is determined; wherein the scheduling priority is negatively correlated with the workload of the virtual machine; the target virtual machine is the virtual machine currently accessing the system memory; and based on the scheduling priority of each target virtual machine, the access commands of each target virtual machine to the system memory are scheduled. In this way, when the current workloads of multiple target virtual machines are uneven and the communication link between the target GPU and the system memory is blocked, a higher scheduling priority is configured for the target virtual machine with a lighter workload, so that it has more priority when using system memory resources, and the access commands of the target virtual machine with a lighter workload to the system memory are preferentially scheduled, so that the access commands of the target virtual machine with a lighter workload to the system memory can be responded to in a timely manner; at the same time, the impact of the operation of the target virtual machine with a heavier workload on the target virtual machine with a lighter workload is reduced, and the isolation between the target virtual machines is improved.

[0047] The scheduling method provided in the embodiments of the present disclosure may be executed by an electronic device or a functional module in the electronic device, wherein the functional module may be a hardware entity or a software module, which is not limited; illustratively, the electronic device may be a terminal or a server, wherein the server may be an independent physical server, or a server cluster composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms, etc. The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc.

[0048] Figure 2 A flowchart of a scheduling method according to an embodiment of the present disclosure is shown. For example, it can be executed by a host end or a component in the host end in a distributed virtualization system. For example, it can be executed by Figure 1 The graphics card driver and GPU scheduler configured on the host side in the distributed virtualization system are executed as shown in FIG. Figure 2 As shown, the following steps are included: Step 201, in response to congestion in the communication link between the target GPU and the system memory, determining the scheduling priority of each target virtual machine in the multiple virtual machines.

[0049] Among them, the target GPU and system memory are the GPU and system memory configured on the host side in the distributed virtualization system, and the distributed virtualization system includes multiple virtual machines configured on the client side, wherein the multiple virtual machines correspond one-to-one to the multiple virtual graphics processors vGPU formed by virtualizing the target GPU.

[0050] The communication link between the target GPU and the system memory can be used for each virtual machine to access the system memory. For example, when the local video memory resources allocated to the vGPU corresponding to the virtual machine are insufficient to process the current work of the virtual machine, the virtual machine can use the system memory; illustratively, the virtual machine currently accessing the system memory can exchange data with the system memory through the communication link between the target GPU and the system memory. As an example, the communication link between the target GPU and the system memory can be a PCIE link. When the target GPU executes the virtual machine's access command to the system memory, it exchanges data with the system memory through the PCIE link, thereby realizing that when the local video memory resources allocated to the vGPU corresponding to the virtual machine are insufficient to process the current work of the virtual machine, the system memory is used to process the current work of the virtual machine.

[0051] Exemplarily, the graphics card driver configured in the host side may periodically detect the congestion of the communication link between the target GPU and the system memory. Exemplarily, the congestion of the communication link may include: the communication link is congested, the communication link is unobstructed, etc. Then, in response to the congestion of the communication link between the target GPU and the system memory, the scheduling priority of each target virtual machine among the multiple virtual machines is determined.

[0052] Exemplarily, when the graphics card driver detects that the communication link between the current target GPU and the system memory is congested, the scheduling priority of each target virtual machine in the distributed virtualization system is determined.

[0053] The target virtual machine is a virtual machine currently accessing the system memory in the distributed virtualization system. That is, the workload of the target virtual machine does not match the size of the local video memory resources allocated to the corresponding vGPU, and the target virtual machine uses the system memory through the PCIe link. For example, the target GPU can execute the target virtual machine's access command to the system memory, and exchange data with the system memory through the PCIe link, so that the target virtual machine can use the system memory through the PCIe link.

[0054] Among them, the scheduling priority is negatively correlated with the workload of the virtual machine; that is, for a target virtual machine, the heavier the workload of the target virtual machine, the lower the scheduling priority of the target virtual machine; correspondingly, the lighter the workload of the target virtual machine, the higher the scheduling priority of the target virtual machine.

[0055] In one possible implementation, determining the scheduling priority of each target virtual machine among the multiple virtual machines includes: obtaining operating information of each target virtual machine; wherein the operating information of each target virtual machine includes at least one of the following: utilization rate of the vGPU corresponding to the target virtual machine, size of the system memory used by the vGPU corresponding to the target virtual machine, and average time taken by the target GPU to execute access commands of the target virtual machine to the system memory; and determining the scheduling priority of each target virtual machine based on the operating information of each target virtual machine. Considering that there is currently no hardware-based PCIe Quality of Service (QoS) and PCIe traffic statistics function, it is impossible to use dedicated hardware to accurately know the data volume of each target virtual machine in the PCIe link in the congested communication link; therefore, based on the running information of each target virtual machine, the target virtual machine that is currently accessed by a large number of PCie and the target virtual machine with less access can be identified, among which the target virtual machine that is accessed by a large number of PCie is the target virtual machine with a heavier workload, and the target virtual machine with less access is the target virtual machine with a lighter workload; further, based on the running information of each target virtual machine, the scheduling priority of each target virtual machine is determined to schedule the access command of each target virtual machine to the system memory.

[0056] It is understandable that the workload of each target virtual machine can be reflected by the operation information of each target virtual machine, and the scheduling priority of each target virtual machine can be determined by using the operation information of the target virtual machine. Exemplarily, the graphics card driver can obtain parameters representing the operation information from each target virtual machine.

[0057] Exemplarily, the utilization rate of the vGPU corresponding to the target virtual machine indicates the ratio of the time that the vGPU takes to execute the target virtual machine's access command to the system memory within a preset processing cycle to the length of the preset processing cycle. This indicator can reflect the efficiency of the use of the vGPU. The higher the utilization rate of the vGPU corresponding to the target virtual machine, the lower the efficiency of the vGPU in executing the target virtual machine's access command to the system memory; conversely, the lower the utilization rate, the higher the efficiency of the vGPU in executing the target virtual machine's access command to the system memory. For example, taking 10ms as a processing cycle, if the vGPU takes 2ms to complete a target virtual machine's access command to the system memory within the processing cycle, the utilization rate of the vGPU is 20%; if the communication link between the target GPU and the system memory is congested, the data transmission between the target GPU and the system memory will be delayed, and the vGPU may take 8ms to complete the execution of the command, then the utilization rate of the vGPU is 80%. The utilization of vGPU can reflect the workload size of the target virtual machine to a certain extent. Usually, the workload of the target virtual machine is heavier, and more data will be transmitted through the communication link between the target GPU and the system memory. If the communication link is congested, the corresponding vGPU may take a long time to complete the task of the target virtual machine, and the corresponding vGPU utilization rate is higher; on the contrary, when the workload of the target virtual machine is lighter, the corresponding vGPU utilization rate is lower.

[0058] Exemplarily, the size of the system memory used by the target virtual machine's corresponding vGPU is the amount of data in the target virtual machine's outbound / inbound Read / Write on the PCie link, which reflects the target virtual machine's occupation of the system memory. It is understandable that the size of the system memory used by the target virtual machine's corresponding vGPU will change with the size of the target virtual machine's workload. For example, when the target virtual machine's workload is light, it can complete the processing task by using less system memory based on the local video memory resources allocated by the corresponding vGPU; and when the target virtual machine's workload is heavy, it needs to use relatively more system memory based on the local video memory resources allocated by the corresponding vGPU to complete the processing task.

[0059] Exemplarily, the average time taken by the target GPU to execute the target virtual machine's access command to the system memory reflects the length of time the target virtual machine occupies the GPU hardware resources; for example, the average time may be the average time taken by the GPU to execute all commands of the target virtual machine within a certain period of time. The average time taken by the target GPU to execute the target virtual machine's access command to the system memory may reflect the workload size of the target virtual machine to a certain extent; when the workload of the target virtual machine is heavy, the target virtual machine has more access commands to the system memory, and the target GPU needs to execute more commands, and accordingly, the time taken to execute these commands is also longer.

[0060] In a possible implementation, determining the scheduling priority of each target virtual machine based on the running information of each target virtual machine includes: determining the scheduling priority of each target virtual machine according to a relative size of the running information of each target virtual machine.

[0061] As an example, the operation information of each target virtual machine may include: the utilization rate of the vGPU corresponding to the target virtual machine. After obtaining the utilization rates of the vGPUs corresponding to all target virtual machines, the graphics card driver compares them and determines the priority of each corresponding target virtual machine in order of the utilization rates of the vGPUs from low to high, that is, the lower the utilization rate of the vGPU, the higher the priority of the corresponding target virtual machine, and correspondingly, the higher the utilization rate of the vGPU, the lower the priority of the corresponding target virtual machine.

[0062] As an example, the operation information of each target virtual machine may include: the size of the system memory used by the vGPU corresponding to the target virtual machine. After obtaining the size of the system memory used by the vGPU corresponding to each target virtual machine, the graphics card driver compares them and determines the priority of each corresponding target virtual machine in the order of the vGPU using the system memory from small to large, that is, the less the vGPU uses the system memory, the higher the priority of the corresponding target virtual machine, and correspondingly, the more the vGPU uses the system memory, the lower the priority of the corresponding target virtual machine.

[0063] As another example, the operation information of each target virtual machine may include: the average time taken by the target GPU to execute the access command of the target virtual machine to the system memory. After obtaining the average time taken by the target GPU to execute the access command of each target virtual machine to the system memory, the graphics card driver compares the average time taken by the target GPU to execute the access command of each target virtual machine to the system memory, and determines the priority of each corresponding target virtual machine in the order of the average time from short to long, that is, the shorter the average time taken by the target GPU to execute the access command of the target virtual machine to the system memory, the higher the priority of the corresponding target virtual machine, and correspondingly, the longer the average time taken by the target GPU to execute the access command of the target virtual machine to the system memory, the lower the priority of the corresponding target virtual machine.

[0064] As another example, the running information of each target virtual machine may include: the utilization rate of the vGPU corresponding to the target virtual machine, the size of the system memory used by the vGPU corresponding to the target virtual machine, and the average time taken by the target GPU to execute the target virtual machine's access command to the system memory; after obtaining the running information of all target virtual machines, the graphics card driver compares them. Exemplarily, the three pieces of information of all target virtual machines can be compared separately, and the scheduling priority of each target virtual machine can be determined by comprehensively comparing the results of the three pieces of information; thereby, the target virtual machines that have used a large amount of system memory and whose corresponding vGPU utilization rate is too high and whose average time taken by the target GPU to execute the target virtual machine's access command to the system memory is long can be determined, and the scheduling priority of these target virtual machines can be set to a lower priority.

[0065] Step 202: Schedule access commands of each target virtual machine to the system memory based on the scheduling priority of each target virtual machine.

[0066] Exemplarily, the graphics card driver can determine the scheduling priority of each target virtual machine as described above, and send a scheduling policy to the GPU scheduler configured on the host side to instruct the GPU scheduler to schedule the access commands of each target virtual machine to the system memory according to the scheduling policy, and execute them by the target GPU.

[0067] Exemplarily, the access commands of each target virtual machine to the system memory may be cached in the form of a command list, and the scheduler schedules the access commands of each target virtual machine in the command list to the system memory based on the scheduling priority of each target virtual machine.

[0068] When congestion is detected in the communication link between the target GPU and the system memory, it indicates that the current access volume of each target virtual machine exceeds the threshold that the communication link can bear, causing some target virtual machines to fail to respond to the access command of the system memory in a timely manner, thereby affecting the normal operation of each target virtual machine; based on the scheduling priority of each target virtual machine determined above, the access command of each target virtual machine to the system memory is scheduled.

[0069] In a possible implementation, the access commands of each target virtual machine to the system memory are scheduled based on the scheduling priority of each target virtual machine, including: determining the scheduling order corresponding to each target virtual machine in order of scheduling priority; and scheduling the access commands of each target virtual machine to the system memory according to the scheduling order corresponding to each target virtual machine. In this way, the access commands of the target virtual machine with a lighter workload to the system memory are scheduled preferentially, and the access commands of the target virtual machine with a heavier workload to the system memory are delayed, thereby limiting the throughput of the virtual machine with a heavier workload on the PCIe link (i.e., the amount of data transmitted through the communication link per unit time), improving the congestion of the communication link, and enabling the target virtual machine with a lighter workload to access the system memory in a timely manner, thereby ensuring the GPU computing power support of the target virtual machine with a lighter workload.

[0070] In a possible implementation, the access commands of each target virtual machine to the system memory are scheduled based on the scheduling priority of each target virtual machine, including: adjusting the scheduling frequency corresponding to each target virtual machine based on the scheduling priority of each target virtual machine; and scheduling the access commands of each target virtual machine to the system memory according to the scheduling frequency corresponding to each target virtual machine.

[0071] The scheduling frequency indicates the ratio of the access commands executed by the target virtual machine to the system memory to all the commands executed in the period within a preset period.

[0072] Exemplarily, adjusting the scheduling frequency corresponding to each target virtual machine based on the scheduling priority of each target virtual machine includes: for any target virtual machine among the target virtual machines, in response to the scheduling priority of the target virtual machine being less than a preset priority, reducing the scheduling frequency of the target virtual machine. Exemplarily, it may also include: in response to the scheduling priority of the target virtual machine being not less than a preset priority, increasing the scheduling frequency of the target virtual machine. Among them, the preset priority can be set according to demand and is not limited to this. In this way, the scheduling frequency of the target virtual machine with a lighter workload is increased and / or the scheduling frequency of the target virtual machine with a heavier workload is reduced, thereby limiting the throughput of the virtual machine with a heavier workload on the PCIe link, improving the congestion of the communication link, and enabling the target virtual machine with a lighter workload to obtain a timely response to the access command to the system memory.

[0073] Exemplarily, scheduling the access commands of each target virtual machine to the system memory according to the scheduling frequency corresponding to each target virtual machine may include: scheduling the access commands of each target virtual machine to the system memory according to the scheduling frequency corresponding to each target virtual machine and at the same time in the scheduling order corresponding to each target virtual machine; for example, the scheduling frequency of the target virtual machine with a heavier workload at present may be limited to limit the throughput of the virtual machine with a heavier workload on the PCIe link, thereby saving part of the throughput; at the same time, scheduling the access commands of each target virtual machine to the system memory according to the scheduling order corresponding to each target virtual machine, so that the target virtual machine with a lighter workload can give priority to using the saved throughput.

[0074] Through the above steps 201-202, in response to the congestion of the communication link between the target GPU and the system memory, the GPU scheduler schedules the access commands of each target virtual machine to the system memory based on the scheduling priority of each target virtual machine; since the scheduling priority is negatively correlated with the workload of the virtual machine, the access commands of the target virtual machine with a lighter workload to the system memory are scheduled first, so that the target GPU can process them first; in this way, the access commands of the target virtual machine with a lighter workload to the system memory can be responded to in a timely manner and can access the system memory in a timely manner. Exemplarily, the graphics card driver can determine the scheduling priority of each target virtual machine in combination with the operation information of all the target virtual machines currently running, and the GPU scheduler schedules the access commands of each target virtual machine to the system memory based on the determined scheduling priority of each target virtual machine until the communication link between the target GPU and the system memory is no longer congested, then restore the original scheduling strategy and schedule the access commands of each target virtual machine to the system memory. Exemplarily, the original scheduling strategy can be that the access commands of each target virtual machine to the system memory are scheduled in sequence according to the order of submission.

[0075] In the embodiment of the present disclosure, in response to the congestion of the communication link between the target GPU and the system memory, the scheduling priority of each target virtual machine among the multiple virtual machines is determined; wherein the scheduling priority is negatively correlated with the workload of the virtual machine; the target virtual machine is the virtual machine currently accessing the system memory; based on the scheduling priority of each target virtual machine, the access command of each target virtual machine to the system memory is scheduled. In this way, when the current workloads of multiple target virtual machines are uneven and the communication link between the target GPU and the system memory is blocked, a higher scheduling priority is configured for the target virtual machine with a lighter workload, so that it has more priority when using system memory resources, and the access command of the target virtual machine with a lighter workload to the system memory is preferentially scheduled, that is, the access command of the target virtual machine with a lighter workload to the system memory can be responded to in time; at the same time, the impact of the operation of the target virtual machine with a heavier workload on the target virtual machine with a lighter workload is reduced, and the isolation between the target virtual machines is improved. In some scenarios, the scheduling priority of each target virtual machine is determined in combination with the operation information of each target virtual machine, so as to schedule the access command of each target virtual machine to the system memory. In this way, through fair and effective scheduling, the access commands of virtual machines with lighter workloads to the system memory can be responded to in a timely manner, thereby effectively reducing the interference of virtual machines with heavier workloads on virtual machines with lighter workloads caused by communication link congestion, and realizing distributed scheduling optimization based on communication link information statistics in a distributed virtualization environment.

[0076] Figure 3 A flowchart of a scheduling method according to an embodiment of the present disclosure is shown. For example, it can be executed by a host end or a component in the host end in a distributed virtualization system. For example, it can be executed by Figure 1 The graphics card driver and GPU scheduler configured on the host side in the distributed virtualization system are executed as shown in FIG. Figure 3 As shown, the following steps are included:

[0077] Step 300: Obtain throughput statistics information of the communication link and / or congestion statistics information of the communication link; in response to the throughput statistics information of the communication link and / or the congestion statistics information of the communication link exceeding a preset threshold, determine that congestion occurs in the communication link.

[0078] It can be understood that the throughput statistics information of the communication link between the target GPU and the system memory and the congestion statistics information of the communication link are dynamically changing, that is, as the distributed virtualization system runs, the throughput statistics information of the communication link and the congestion statistics information of the communication link will continue to change. Exemplarily, the graphics card driver can periodically obtain the throughput statistics information of the communication link and / or the congestion statistics information of the communication link, and then determine whether the throughput statistics information of the communication link and / or the congestion statistics of the communication link currently obtained exceeds a preset threshold, thereby determining the congestion situation of the communication link.

[0079] As an example, the graphics card driver can periodically obtain the throughput statistics of the communication link between the target GPU and the system memory, and after each time obtaining the throughput statistics of the communication link, determine whether the throughput statistics of the communication link exceeds a first preset threshold, and if it exceeds the first preset threshold, determine that the communication link between the target GPU and the system memory is congested; otherwise, determine that the communication link between the target GPU and the system memory is unblocked. The first preset threshold can be set according to demand and is not limited to this.

[0080] As another example, the graphics card driver can periodically obtain the congestion statistics of the communication link between the target GPU and the system memory, and after each acquisition of the congestion statistics of the communication link, determine whether the congestion statistics of the communication link exceeds the second preset threshold, if it exceeds the second preset threshold, it is determined that the communication link between the target GPU and the system memory is congested; otherwise, it is determined that the communication link between the target GPU and the system memory is unblocked. The second preset threshold can be set according to demand, and the second preset threshold can be the same as the first preset threshold, or it can be different.

[0081] As another example, the graphics card driver may periodically obtain throughput statistics information and congestion statistics information of the communication link between the target GPU and the system memory, and after each time obtaining the throughput statistics information of the communication link, determine whether the throughput statistics information of the communication link exceeds a first preset threshold and whether the congestion statistics information of the communication link exceeds a second preset threshold; if the throughput statistics information of the communication link exceeds the first preset threshold and the congestion statistics information of the communication link exceeds the second preset threshold, it is determined that the communication link between the target GPU and the system memory is congested; otherwise, it is determined that the communication link between the target GPU and the system memory is unblocked.

[0082] Exemplarily, the throughput statistics information in the communication link includes at least one of: the amount of data transmitted on the communication link, the number of data packets transmitted on the communication link, and the average size of each data packet transmitted on the communication link.

[0083] Among them, the amount of data transmitted on the communication link between the target GPU and the system memory is the total amount of data actually transmitted on the communication link per unit time, the number of data packets transmitted on the communication link between the target GPU and the system memory is the number of data packets actually transmitted on the communication link per unit time, and the average size of each data packet transmitted on the communication link between the target GPU and the system memory is the ratio of the total amount of data actually transmitted on the communication link per unit time to the number of data packets actually transmitted on the communication link. It can be understood that at different moments, the amount of data transmitted on the communication link between the target GPU and the system memory, the number of data packets transmitted on the communication link between the target GPU and the system memory, and the average size of each data packet transmitted on the communication link will continue to change; for a certain moment, the amount of data transmitted on the communication link between the target GPU and the system memory currently obtained by the graphics card driver is: the total amount of data actually transmitted on the communication link per unit time forward, with that moment as the end time.

[0084] Exemplarily, taking the communication link as a PCIe link as an example, the throughput statistics in the PCIe link include: the amount of data transmitted on the PCIe link, the number of data packets transmitted on the PCIe link, and the average size of each data packet transmitted on the PCIe link. Among them, the amount of data transmitted on the PCIe link can be the amount of data of PCie outbound (outbound) / inbound (inbound) read (read) / write (write) in the PCie link per unit time (that is, the amount of data of PCie Outbound read and write operations and the amount of data of PCie inbound read and write operations); the number of data packets transmitted on the PCIe link can be the number of data packets of PCie Outbound / inbound read / write in the PCie link per unit time; the average size of each data packet transmitted on the PCIe link can be the average size of each data packet of PCie Outbound / inbound read / write in the PCie link. In the PCIe link, "Outbound" refers to the process of the memory domain accessing the PCI domain. When the system memory needs to send data to the GPU or perform a write operation, these data are transmitted through PCie Outbound; correspondingly, "inbound" refers to the process of the PCI domain accessing the memory domain.

[0085] Exemplarily, the congestion statistics may include parameters in the target GPU and / or system memory that reflect the degree of congestion of the communication link between the target GPU and the system memory. For example, the congestion statistics of the communication link include: back pressure information of the target GPU and / or the quota of the system memory.

[0086] Wherein, the back pressure information includes the value of the write stall register and / or the value of the read stall register in the target GPU. Taking the communication link as a PCIe link as an example, the back pressure information of the target GPU includes: the value of the GPU read_stall (read stall) register and / or the value of the GPU write_stall (write stall) register. It should be noted that when the current PCie link is congested, the data generated by the GPU to the system memory will be back-pressed to the GPU side, and there will be corresponding registers in the GPU to reflect the congestion level of the current PCie link, namely the GPU read_stall register and the GPU write_stall register; illustratively, the values ​​of the GPU read_stall register and the GPU write_stall register can be configured in advance according to the different congestion levels of the PCie link. In this way, when the current PCie link is congested, the current congestion level of the PCie link can be determined according to the values ​​of the GPU read_stall register and / or the GPU write_stall register.

[0087] Among them, the quota of the system memory is used to indicate the amount of data that the system memory can receive. Taking the communication link as a PCIe link as an example, the quota of the system memory is the credit value obtained by the read and write operations in the current PCie link. It should be noted that in the Flow control mechanism on the PCie link, when the sender sends data to the receiver, the sender needs to obtain a credit. The value of the credit is the abstract quantified value of the system memory receivable cache resources. Only when the value of the credit is greater than 0, it means that the receiver is still able to receive data. If the value of the credit is equal to 0, it means that the current receiver has no extra space to store data, then all sent commands will wait on the sender side. Therefore, the credit value obtained through the read and write operations in the current PCie link can reflect the congestion of the PCie link to a certain extent. Exemplarily, the credit value obtained for each read and write operation in the PCie link within a period of time can be obtained, so as to count the 0 rate of the credit values ​​obtained for the read and write operations in the PCie link, that is, the ratio of the number of read and write operations corresponding to the credit value of 0 in the period of time to the total number of read and write operations in the period of time. If the 0 rate of the credit value exceeds a certain value, it indicates that the receive buffer of the flow control on the PCie link in the system memory is full and can no longer receive new data packets sent by the GPU, causing the GPU data packets to be in a waiting state.

[0088] Step 301: In response to congestion in a communication link between the target GPU and a system memory, determine a scheduling priority of each target virtual machine among the multiple virtual machines.

[0089] This step is similar to the above Figure 2 The step 201 is the same as that in the above step and will not be described again.

[0090] Step 302: Schedule access commands of each target virtual machine to the system memory based on the scheduling priority of each target virtual machine.

[0091] This step is similar to the above Figure 2 The same as step 202 in FIG. 1 and will not be repeated here.

[0092] Through the above steps 300-302, the graphics card driver can detect the congestion of the communication link between the target GPU and the system memory. When the communication link is congested, the GPU scheduler can schedule the access commands of each target virtual machine to the system memory based on the scheduling priority of each target virtual machine. Since the scheduling priority is negatively correlated with the workload of the virtual machine, the access commands of the target virtual machine with a lighter workload to the system memory are preferentially scheduled so that the target GPU can process them preferentially. In this way, the access commands of the target virtual machine with a lighter workload to the system memory can be responded to in a timely manner, and the system memory can be accessed in a timely manner. Exemplarily, the graphics card driver can detect the congestion of the communication link between the target GPU and the system memory once at a fixed period. The graphics card driver comprehensively considers the throughput statistics of the communication link and the congestion statistics of the communication link to determine whether the communication link is congested. When the communication link is currently detected to be congested, the graphics card driver combines the operation information of all currently running target virtual machines to determine the scheduling priority of each target virtual machine. The GPU scheduler schedules the access commands of each target virtual machine to the system memory based on the determined scheduling priority of each target virtual machine until it is detected that the communication link between the target GPU and the system memory is no longer congested, then restores the original scheduling strategy and schedules the access commands of each target virtual machine to the system memory. Exemplarily, the original scheduling strategy can be for each target virtual machine to schedule the access commands to the system memory in the order of submission.

[0093] In the disclosed embodiment, the congestion of the communication link between the target GPU and the system memory is detected; when the communication link is congested, the scheduling priority of each target virtual machine among the multiple virtual machines is determined; wherein the scheduling priority is negatively correlated with the workload of the virtual machine; the target virtual machine is the virtual machine currently accessing the system memory; based on the scheduling priority of each target virtual machine, the access commands of each target virtual machine to the system memory are scheduled. In this way, when the current workloads of multiple target virtual machines are uneven and the communication link between the target GPU and the system memory is blocked, a higher scheduling priority is configured for the target virtual machine with a lighter workload, so that it has more priority when using the system memory resources, and the access commands of the target virtual machine with a lighter workload to the system memory are preferentially scheduled, that is, the access commands of the target virtual machine with a lighter workload to the system memory can be responded to in a timely manner; at the same time, the impact of the operation of the target virtual machine with a heavier workload on the target virtual machine with a lighter workload is reduced, and the isolation between the target virtual machines is improved. In some scenarios, the throughput statistics of the communication link and the congestion statistics of the communication link are used to determine whether the communication link is congested. In the case of congestion, the scheduling priority of each target virtual machine is determined in combination with the operation information of each target virtual machine, so as to schedule the access commands of each target virtual machine to the system memory. In this way, through fair and effective scheduling, the access commands of virtual machines with lighter workloads to the system memory can be responded to in a timely manner, thereby effectively reducing the interference of virtual machines with heavier workloads on virtual machines with lighter workloads caused by communication link congestion, and realizing distributed scheduling optimization based on communication link information statistics in a distributed virtualization environment.

[0094] Figure 4 A flowchart of a scheduling method according to an embodiment of the present disclosure is shown, which can be Figure 1 The graphics card driver and GPU scheduler configured on the host side in the distributed virtualization system are executed as shown in FIG. Figure 4As shown, the client in the distributed virtualization system is configured with n target virtual machines, namely VM1, VM2, VM3…VMn; wherein, the workload of each target virtual machine decreases in sequence, that is, the workload of VM1 is very heavy, the workload of VM2 is relatively heavy, the workload of VM3 is generally heavy…the workload of VMn is light. First, the graphics card driver obtains the throughput statistics of the Pcie link, the congestion statistics of the Pcie link, and the operation information of each target virtual machine; this information is summarized into the scheduling module (schedule) in the graphics card driver; then, the scheduling module processes the summarized information, and determines whether the current PCie link is congested through the throughput statistics of the Pcie link and the congestion statistics of the Pcie link. If congestion has occurred, the operation information of all target virtual machines is combined to determine the scheduling priority of each target virtual machine, and the scheduling priority of each target virtual machine is used from high to low as the scheduling order of each target virtual machine's access command to the system memory, as shown in Figure 4 As shown, the scheduling order of the access commands of each target virtual machine to the system memory is VMn...VM3, VM2, VM1. Finally, the graphics card driver sends the scheduling order of the access commands of each target virtual machine to the system memory to the GPU scheduler, and the GPU schedules the access commands of each target virtual machine to the system memory according to the scheduling order of the access commands of each target virtual machine to the system memory, so that the GPU gives priority to responding to the access commands to the system memory sent by the target virtual machine with light workload, that is, giving priority to responding to the access commands to the system memory sent by VMn; reducing the interference of the target virtual machine with heavy workload on the target virtual machine with light workload, and improving the isolation between the target virtual machines.

[0095] Figure 5 A flowchart of a scheduling method according to an embodiment of the present disclosure is shown. Figure 1 The graphics card driver and GPU scheduler configured on the host side in the distributed virtualization system are executed as shown in FIG. Figure 5As shown, the graphics card driver first obtains the throughput statistics of the PCie link (including: the amount of data of the PCie outbound / inbound read / write in the current PCie link, the number of data packets of the PCie Outbound / inbound read / write in the current PCie link, the average size of each data packet of the PCie Outbound / inbound read / write in the PCie link), the congestion statistics of the PCie link (including: the value of the GPU read_stall register, the value of the GPU write_stall register, the 0 rate of the credit value obtained by the read and write operations in the PCie link); and summarizes this information to the scheduling module in the graphics card driver.

[0096] Then, the scheduling module uses the throughput statistics of the PCie link and the congestion statistics of the PCie link to determine whether the scheduling strategy needs to be modified. If the preset conditions are met, it means that the PCie link is congested and the scheduling strategy needs to be modified. If not, the scheduling strategy does not need to be modified. Exemplarily, the preset conditions may include: (1) The amount of data in the PCie outbound / inbound read / write in the current PCie link exceeds the threshold T1, and the number of data packets in the PCie Outbound / inbound read / write in the current PCie link exceeds the threshold T2, and the average size of each data packet in the PCie Outbound / inbound read / write in the PCie link is less than the threshold T3. (2) The value of the GPU read_stall register exceeds the threshold T4 and the value of the GPU write_stall register exceeds the threshold T5. (3) The 0 rate of the credit value obtained by the read and write operations in the PCie link exceeds the threshold T6. Among them, the values ​​of thresholds T1-T6 can be set according to requirements.

[0097] Furthermore, if congestion is detected in the PCIE link, the scheduling strategy needs to be modified, and the operation information of each target virtual machine (including: the utilization rate of the vGPU corresponding to each target virtual machine, the size of the system memory used by the vGPU corresponding to each target virtual machine, and the average time taken by the target GPU to execute the access command of each target virtual machine to the system memory) is shared with the graphics driver through shared memory. The graphics driver obtains this information from the shared memory and then processes it in the scheduling module. The scheduling module processes this information, determines the scheduling priority of each target virtual machine, and determines the scheduling order corresponding to each target virtual machine in the order of high and low scheduling priorities.

[0098] Finally, the graphics card driver sends the scheduling order corresponding to each target virtual machine to the GPU scheduler; the GPU scheduler prioritizes the access commands to the system memory of the virtual machines with lighter workloads, so that the GPU can respond to the access commands to the system memory of the virtual machines with lighter workloads in a timely manner.

[0099] Based on the same inventive concept of the above method embodiments, an embodiment of the present disclosure further provides a scheduling device, which can be used to execute the technical solution described in the above method embodiments.

[0100] Figure 6 A structural diagram of a scheduling device according to an embodiment of the present disclosure is shown, and the scheduling device is applied to a distributed virtualization system, wherein the distributed virtualization system includes a plurality of virtual machines, wherein the plurality of virtual machines correspond one-to-one to a plurality of virtual graphics processors vGPU formed by virtualizing a target graphics processor GPU; Figure 6 As shown, the device may include: a priority determination module 601, used to determine the scheduling priority of each target virtual machine among the multiple virtual machines in response to congestion in the communication link between the target GPU and the system memory; wherein the scheduling priority is negatively correlated with the workload of the virtual machine; the target virtual machine is the virtual machine currently accessing the system memory; and a scheduling module 602, used to schedule the access commands of each target virtual machine to the system memory based on the scheduling priority of each target virtual machine.

[0101] In one possible implementation, the device also includes: a detection module, used to obtain throughput statistical information of the communication link and / or congestion statistical information of the communication link; in response to the throughput statistical information of the communication link and / or the congestion statistical information of the communication link exceeding a preset threshold, determining that congestion occurs in the communication link.

[0102] In one possible implementation, the priority determination module 601 is further used to: obtain the operating information of each target virtual machine; wherein the operating information of each target virtual machine includes at least one of the following: the utilization rate of the vGPU corresponding to the target virtual machine, the size of the system memory used by the vGPU corresponding to the target virtual machine, and the average time taken by the target GPU to execute the target virtual machine's access command to the system memory; based on the operating information of each target virtual machine, determine the scheduling priority of each target virtual machine.

[0103] In a possible implementation, the throughput statistics information in the communication link includes: at least one of the amount of data transmitted on the communication link, the number of data packets transmitted on the communication link, and the average size of each data packet transmitted on the communication link; and / or, the congestion statistics information of the communication link includes: back pressure information of the target GPU and / or the quota of the system memory; wherein the back pressure information includes the value of the write stall register and / or the value of the read stall register in the target GPU, and the quota is used to indicate the amount of data that can be received by the system memory.

[0104] In a possible implementation, the scheduling module 602 is further used to: determine the order of scheduling priorities as the scheduling order corresponding to the target virtual machines; and schedule the access commands of the target virtual machines to the system memory according to the scheduling order corresponding to the target virtual machines.

[0105] In a possible implementation, the scheduling module 602 is further used to: adjust the scheduling frequency corresponding to each target virtual machine based on the scheduling priority of each target virtual machine; and schedule the access command of each target virtual machine to the system memory according to the scheduling frequency corresponding to each target virtual machine.

[0106] In a possible implementation, the scheduling module 602 is further configured to: for any target virtual machine among the target virtual machines, in response to a scheduling priority of the target virtual machine being lower than a preset priority, reduce a scheduling frequency of the target virtual machine.

[0107] Above Figure 6 The technical effects and specific descriptions of the scheduling device shown and its various possible implementation methods can be found in the above-mentioned scheduling method, which will not be repeated here.

[0108] It should be understood that the division of the modules in the above device is only a division of logical functions, and in actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. In addition, the modules in the device can be implemented in the form of a processor calling software; for example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of the modules of the device, wherein the processor is, for example, a general-purpose processor, such as a CPU or a microprocessor, and the memory is a memory inside the device or a memory outside the device. Alternatively, the modules in the device can be implemented in the form of hardware circuits, and the functions of some or all modules can be realized by designing the hardware circuits, and the hardware circuits can be understood as one or more processors; for example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), and the functions of some or all modules above are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be realized by a programmable logic device (PLD), taking a field programmable gate array (FPGA) as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of some or all modules above. All modules of the above device can be realized in the form of a processor calling software, or in the form of a hardware circuit, or in part by a processor calling software, and the rest by a hardware circuit.

[0109] In the embodiments of the present disclosure, the processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a neural-network processing unit (NPU), a tensor processing unit (TPU), etc.; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules.

[0110] It can be seen that each module in the above device can be one or more processors (or processing circuits) configured to implement the above embodiment method, such as: CPU, GPU, NPU, TPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms. In addition, each module in the above device can be fully or partially integrated together, or can be implemented independently, which is not limited.

[0111] An embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the method of the above embodiment when executing the instructions.

[0112] Figure 7 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 7 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0113] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0114] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0115] An embodiment of the present disclosure provides a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method in the above embodiment is implemented.

[0116] An embodiment of the present disclosure provides a computer program product, which may include, for example, a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code; when the computer program product runs on a computer, the computer executes the method in the above embodiment.

[0117] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0118] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0119] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0120] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0121] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0122] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0123] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0124] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0125] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A scheduling method, characterized in that: Applied to a distributed virtualization system, the distributed virtualization system includes a plurality of virtual machines, wherein the plurality of virtual machines correspond one-to-one to a plurality of virtual graphics processors vGPU formed by virtualizing a target graphics processor GPU; The method comprises: In response to congestion in the communication link between the target GPU and the system memory, determining a scheduling priority of each target virtual machine among the multiple virtual machines; wherein the scheduling priority is negatively correlated with the workload of the virtual machine; and the target virtual machine is a virtual machine currently accessing the system memory; Based on the scheduling priority of each target virtual machine, the access commands of each target virtual machine to the system memory are scheduled.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining throughput statistics information of the communication link and / or congestion statistics information of the communication link; In response to the throughput statistics information of the communication link and / or the congestion statistics information of the communication link exceeding a preset threshold, it is determined that congestion occurs on the communication link.

3. The method according to claim 1, characterized in that The determining the scheduling priority of each target virtual machine among the multiple virtual machines includes: Obtaining the operation information of each target virtual machine; wherein the operation information of each target virtual machine includes at least one of the following: the utilization rate of the vGPU corresponding to the target virtual machine, the size of the system memory used by the vGPU corresponding to the target virtual machine, and the average time taken by the target GPU to execute the access command of the target virtual machine to the system memory; Based on the operation information of each target virtual machine, a scheduling priority of each target virtual machine is determined.

4. The method according to claim 2, characterized in that: The throughput statistics information in the communication link includes: at least one of the amount of data transmitted on the communication link, the number of data packets transmitted on the communication link, and the average size of each data packet transmitted on the communication link; and / or, the congestion statistics information of the communication link includes: back pressure information of the target GPU and / or the quota of the system memory; wherein the back pressure information includes the value of the write stall register and / or the value of the read stall register in the target GPU, and the quota is used to indicate the amount of data that can be received by the system memory.

5. The method according to claim 1, characterized in that The step of scheduling the access commands of the target virtual machines to the system memory based on the scheduling priorities of the target virtual machines includes: Determine the scheduling order corresponding to each target virtual machine in order of scheduling priority; The access commands of the target virtual machines to the system memory are scheduled according to the scheduling order corresponding to the target virtual machines.

6. The method according to claim 1, characterized in that The step of scheduling the access commands of the target virtual machines to the system memory based on the scheduling priorities of the target virtual machines includes: Based on the scheduling priority of each target virtual machine, adjusting the scheduling frequency corresponding to each target virtual machine; The access commands of the target virtual machines to the system memory are scheduled according to the scheduling frequencies corresponding to the target virtual machines.

7. The method according to claim 6, characterized in that The adjusting the scheduling frequency corresponding to each target virtual machine based on the scheduling priority of each target virtual machine includes: For any target virtual machine among the target virtual machines, in response to a scheduling priority of the target virtual machine being lower than a preset priority, reducing a scheduling frequency of the target virtual machine.

8. A scheduling device, characterized in that: Applied to a distributed virtualization system, the distributed virtualization system includes a plurality of virtual machines, wherein the plurality of virtual machines correspond one-to-one to a plurality of virtual graphics processors vGPU formed by virtualizing a target graphics processor GPU; The device comprises: a priority determination module, configured to determine a scheduling priority of each target virtual machine among the multiple virtual machines in response to congestion in a communication link between the target GPU and the system memory; wherein the scheduling priority is negatively correlated with a workload of the virtual machine; and the target virtual machine is a virtual machine currently accessing the system memory; The scheduling module is used to schedule the access commands of each target virtual machine to the system memory based on the scheduling priority of each target virtual machine.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 7 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Video memory management method and device for graphics processor virtualization, server and medium

    CN121277610A