Binding method of virtual machine GPU and NUMA node and related device

Abstract

The application discloses a virtual machine GPU and NUMA node binding method and related equipment, wherein the method comprises: obtaining a GPU binding strategy corresponding to a target virtual machine, wherein the GPU binding strategy is used to indicate a binding relationship between a virtualized GPU of the target virtual machine and a NUMA node; determining a first NUMA node according to the GPU binding strategy and target binding state information, and binding the virtualized CPU and virtualized memory to the first NUMA node, wherein the target binding state information is binding state information of the virtualized CPU and the NUMA node, and the first NUMA node is used to provide CPU resources and memory resources for the target virtual machine through a virtualization technology; determining a target physical GPU corresponding to the target virtual machine from at least one host according to the first NUMA node and the GPU binding strategy, and allocating GPU resources for the virtualized GPU of the target virtual machine through the target physical GPU.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and related equipment for binding a virtual machine GPU and a NUMA node. Background Technology

[0002] NUMA (Non-Uniform Memory Access) is a memory architecture for multiprocessor systems. In a NUMA architecture, each NUMA node includes a Central Processing Unit (CPU) or a CPU core group. This CPU or CPU core group may have local physical memory. CPU access to local memory has low latency, while access to remote memory has high latency. Therefore, to ensure performance, the virtual machine scheduler needs to bind the virtual machine's Virtual Central Processing Unit (vCPU) and virtual memory to the same NUMA node to guarantee the virtual machine's memory access speed.

[0003] However, the introduction of Graphics Processing Units (GPUs) has complicated NUMA topologies. If the topological affinity between GPUs and NUMA is ignored, and only the physical GPU with the largest remaining capacity is selected from the global resource pool to allocate GPU resources to the virtual machine, then data transfer between vCPUs and vGPUs needs to be done through the cross-node interconnect bus, which introduces additional latency and impacts virtual machine performance. If the virtual machine's vCPUs and vGPUs are bound to the same NUMA node, there is an issue of uneven resource utilization. Therefore, how to solve the uneven resource utilization caused by traditional NUMA binding while ensuring virtual machine performance is a critical technical problem that needs to be addressed. Summary of the Invention

[0004] This application provides a method and related equipment for binding a virtual machine GPU to a NUMA node, which can solve the problem of uneven resource utilization caused by traditional NUMA binding while ensuring virtual machine performance.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, a method for binding a virtual machine's GPU to a NUMA node is provided. The method includes: obtaining a GPU binding policy corresponding to a target virtual machine, wherein the GPU binding policy is used to indicate the binding relationship between the virtualized GPU of the target virtual machine and a non-uniform memory access NUMA node; determining a first NUMA node according to the GPU binding policy and target binding status information, and binding the virtualized CPU and virtualized memory of the target virtual machine to the first NUMA node, wherein the target binding status information is the binding status information between the virtualized CPU of the target virtual machine and the NUMA node, and the first NUMA node is used to provide CPU resources and memory resources to the target virtual machine through virtualization technology; determining a target physical GPU corresponding to the target virtual machine from at least one host according to the first NUMA node and the GPU binding policy, and allocating GPU resources to the virtualized GPU of the target virtual machine through the target physical GPU.

[0006] In a second aspect, an electronic device is provided, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0007] Thirdly, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, a computer program product is provided, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the steps of the method described in the first aspect.

[0009] In this embodiment, a first NUMA node is determined based on a GPU binding policy indicating the binding relationship between the virtualized GPU and NUMA node of the target virtual machine, and target binding status information indicating the binding status between the virtualized CPU and NUMA node of the target virtual machine. The virtualized CPU and virtualized memory of the target virtual machine are then bound to the first NUMA node, achieving binding of the CPU and memory according to the NUMA affinity principle. This avoids significant performance loss caused by the CPU accessing memory across multiple nodes. Based on the first NUMA node and the GPU binding policy, a target physical GPU corresponding to the target virtual machine is determined from at least one host. GPU resources are allocated to the virtualized GPU of the target virtual machine through the target physical GPU. This achieves the determination of the optimal physical GPU based on the selection of the first NUMA node. Thus, while ensuring virtual machine performance, it solves the problems of uneven resource utilization, poor flexibility, and resource bottlenecks caused by the strong binding of GPU and CPU to NUMA in traditional methods.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0012] Figure 1 The diagram illustrates a flowchart of a method for binding a virtual machine GPU to a NUMA node according to some embodiments of this application. Figure 2 This application illustrates a schematic diagram of a NUMA architecture provided by some embodiments; Figure 3 This illustration shows a schematic diagram of a first binding strategy provided by some embodiments of this application; Figure 4 This illustration shows a schematic diagram of a second binding strategy provided by some embodiments of this application; Figure 5 This illustration shows another second binding strategy provided by some embodiments of this application; Figure 6 The diagram illustrates a matching check process provided by some embodiments of this application; Figure 7 The diagram illustrates a flowchart of another method for binding a virtual machine GPU to a NUMA node, provided in some embodiments of this application. Figure 8 This paper presents a schematic diagram comparing memory access speeds according to some embodiments of this application. Figure 9 This paper presents another memory access speed comparison diagram provided by some embodiments of this application; Figure 10 This illustration shows another memory access speed comparison diagram provided by some embodiments of this application; Figure 11 The diagram illustrates a structural schematic of a binding device between a virtual machine GPU and a NUMA node, provided in some embodiments of this application. Figure 12 The diagram shows a schematic representation of the structure of an electronic device provided in some embodiments of this application. Detailed Implementation

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0014] Figure 1 This illustration shows a flowchart of a method for binding a virtual machine GPU to a NUMA node according to some embodiments of this application. This method can be executed by an electronic device, which may include a terminal device and a network-side device. In other words, the method can be executed by software or hardware installed on the electronic device, and the method may include the following steps: S110: Obtain the GPU binding policy corresponding to the target virtual machine.

[0015] The GPU binding strategy is used to indicate the binding relationship between the virtualized GPU of the target virtual machine and the NUMA node with non-uniform memory access.

[0016] The virtualized GPU (vGPU) is a virtualized resource of the physical GPU. In a virtualized environment, the vGPU is the GPU resource allocated to the virtual machine.

[0017] Understandably, in related technologies, the virtualized CPU (vCPU) of a virtual machine process is bound to the physical CPU core of a specific NUMA node. However, for vGPUs, a global load balancing strategy is used, binding the vGPU to the NUMA node containing the physical GPU with the largest remaining capacity. This results in the following: when a virtual machine's vCPU is assigned to a remote NUMA node, data transmission between the vCPU and vGPU needs to be routed through the cross-node interconnect bus, significantly increasing data access latency and impacting the performance of latency-sensitive applications such as high-performance computing and AI inference. Forcing all resources related to a virtual machine process (vCPU, memory, vGPU) to the same NUMA node will lead to the following problems: (1) When a NUMA node has sufficient CPU and memory, but its associated GPU resources are exhausted, the NUMA node will no longer be able to create virtual machines that require GPU resources. Even if there are still free GPU resources on other NUMA nodes, the virtual machine cannot be created, resulting in virtual machine creation failure, resource fragmentation and low resource utilization.

[0018] (2) A physical GPU is usually only mounted on a NUMA node, which makes a high-performance physical GPU limited by the CPU and memory resources of a single NUMA node. When there is a scenario where the GPU computing power is excessive but the CPU in a single NUMA is insufficient, the GPU computing power cannot be fully utilized, thus forming a resource bottleneck.

[0019] Therefore, in this embodiment of the application, when creating a target virtual machine, the GPU binding policy corresponding to the target virtual machine can be obtained. The GPU binding policy is used to indicate the binding relationship between the virtualized GPU and NUMA node of the target virtual machine. Subsequently, the fine-grained and flexible management of the virtual machine GPU and NUMA binding is realized based on the GPU binding policy.

[0020] In some embodiments, S110 may include: receiving a virtual machine creation request issued by an administrator through a target interface, wherein the virtual machine creation request carries a resource configuration corresponding to the virtual machine to be created, the resource configuration including a GPU binding policy, and the target interface may be WEB SERVICE or IECS; and extracting the GPU binding policy from the resource configuration.

[0021] In some embodiments, S110 may include: obtaining first status information, wherein the first status information is the binding status information between the target virtual machine and the NUMA node; and obtaining the GPU binding policy if the first status information indicates that the binding between the target virtual machine and the NUMA node is enabled.

[0022] In some embodiments, S110 may include: obtaining second state information, wherein the second state information is binding state information between the virtual machine GPU and the NUMA node; and obtaining the GPU binding policy if the first state information indicates that the binding between the virtual machine GPU and the NUMA node is enabled.

[0023] S120: Based on the GPU binding strategy and target binding status information, determine the first NUMA node and bind the virtualized CPU and virtualized memory of the target virtual machine to the first NUMA node.

[0024] Among them, the target binding status information is the binding status information between the virtualization central processing unit (CPU) of the target virtual machine and the NUMA node. The first NUMA node is used to provide CPU and memory resources to the target virtual machine through virtualization technology.

[0025] The virtualized CPU is a virtualized resource of the physical CPU. In a virtualization environment, vCPU is the CPU resource allocated to a virtual machine.

[0026] Understandably, if GPU binding strategies are not considered and CPU binding is directly performed based on target binding state information, communication between vCPU and vGPU may span multiple NUMA nodes. Interaction across multiple nodes could lead to additional transmission latency and bandwidth bottlenecks on the bus, causing blockages when the vCPU issues instructions or processes data returned by the vGPU. Therefore, it is necessary to determine the first NUMA node based on the GPU binding strategy and target binding state information to avoid significant performance losses caused by communication across multiple nodes, thereby ensuring virtual machine performance.

[0027] S130: Based on the first NUMA node and GPU binding policy, determine the target physical GPU corresponding to the target virtual machine from at least one host, and allocate GPU resources to the virtualization GPU of the target virtual machine through the target physical GPU.

[0028] Each host consists of multiple NUMA nodes. These multiple NUMA nodes form a logical NUMA node group. Each NUMA node is associated with local memory and CPU, and may also be equipped with a physical GPU.

[0029] Understandably, after the target virtual machine's vCPU and virtualized memory are bound to the first NUMA node, the physical GPU it uses is not yet determined. At this time, the selection of the first NUMA node is used as a constraint to determine the optimal target physical GPU for the target virtual machine under this constraint, and the target physical GPU provides GPU resources for the target virtual machine's virtualized GPU.

[0030] In this embodiment, a first NUMA node is determined based on a GPU binding policy indicating the binding relationship between the virtualized GPU and NUMA node of the target virtual machine, and target binding status information indicating the binding status between the virtualized CPU and NUMA node of the target virtual machine. The virtualized CPU and virtualized memory of the target virtual machine are then bound to the first NUMA node, achieving binding of the CPU and memory according to the NUMA affinity principle. This avoids significant performance loss caused by the CPU accessing memory across multiple nodes. Based on the first NUMA node and the GPU binding policy, a target physical GPU corresponding to the target virtual machine is determined from at least one host. GPU resources are allocated to the virtualized GPU of the target virtual machine through the target physical GPU. This achieves the determination of the optimal physical GPU based on the selection of the first NUMA node. Thus, while ensuring virtual machine performance, it solves the problems of uneven resource utilization, poor flexibility, and resource bottlenecks caused by the strong binding of GPU and CPU to NUMA in traditional methods.

[0031] In some embodiments, the GPU binding strategy is one of the following: (1) First binding strategy, wherein the first binding strategy is used to indicate that the virtualized GPU of the target virtual machine is bound to the NUMA node bound to the virtualized CPU of the target virtual machine; It is understandable that the first binding strategy refers to a one-to-one binding of the GPU to a NUMA node, that is, binding the vCPU and vGPU to the same NUMA node. This first binding strategy enables proximity-based access and completely eliminates cross-node access overhead, making it suitable for scenarios requiring low latency and high bandwidth, such as AI training. For example, as... Figure 2 The diagram shows a schematic of a host's NUMA architecture, where the host includes four NUMA node groups: NUMA node group 1, NUMA node group 2, NUMA node group 3, and NUMA node group 4. NUMA node group 1 includes NUMA node 0 and NUMA node 1; NUMA node group 2 includes NUMA node 2 and NUMA node 3; NUMA node group 3 includes NUMA node 4 and NUMA node 5; and NUMA node group 4 includes NUMA node 6 and NUMA node 7. Figure 2 Based on the NUMA architecture shown, assuming NUMA node 0 is connected to an external physical GPU, and NUMA node 1 is not connected to an external physical GPU, such as... Figure 3The diagram shown is a schematic of the first binding strategy. Both vCPU and vGPU are bound to NUMA node 0. The memory in NUMA node 0 provides memory resources for the virtual memory (VM) of the target virtual machine, the CPU in NUMA node 0 provides CPU resources for the vCPU of the target virtual machine, and the GPU in NUMA node 0 provides GPU resources for the vGPU of the target virtual machine.

[0032] Under this first binding strategy, local affinity is achieved between the CPU, memory, and GPU. When the vCPU needs to transfer computing tasks or data to the vGPU, since the two share the same local memory of the same NUMA node, the data only needs to be transferred between the local memory and the GPU memory. There is no need to go through the NUMA interconnect bus for cross-node routing, which ensures that the data exchange between the CPU and the GPU is completed with the highest bandwidth and the lowest latency, thereby significantly improving the performance of the virtual machine.

[0033] (2) Second binding strategy, wherein the second binding strategy is used to indicate that the virtualized GPU of the target virtual machine is bound to the first NUMA node group, and the first NUMA node group is the NUMA node group to which the virtualized CPU of the target virtual machine belongs; Understandingly, the second binding strategy refers to binding GPUs to NUMA nodes in groups, meaning that the NUMA nodes bound to vCPU and vGPU belong to the same NUMA node group. In other words, the NUMA nodes bound to vCPU and vGPU are within the same NUMA node group, allowing CPU and GPU to perform cross-node scheduling within the same group, thereby expanding the available resource pool. This second binding strategy can overcome the resource limitations of a single NUMA node, allowing GPU computing power to be utilized by NUMA nodes with more CPU and memory. This second binding strategy is suitable for scenarios where the application allows for slight latency but needs to overcome the resource bottleneck of a single node, such as moderately loaded cloud desktops and batch rendering. Figure 2 Based on the NUMA architecture shown, assuming NUMA node 0 is connected to an external physical GPU, and NUMA node 1 is not connected to an external physical GPU, such as... Figure 4 The diagram illustrates the second binding strategy. The vCPU is bound to NUMA node 1, with the CPU in NUMA node 1 providing CPU resources for the target virtual machine's vCPU. The VM memory is bound to NUMA node 1, with the memory in NUMA node 1 providing memory resources for the target virtual machine's VM memory. The vGPU is bound to NUMA node 0, with the GPU in NUMA node 0 providing GPU resources for the target virtual machine's vGPU. In other embodiments, such as... Figure 5As shown, assuming that NUMA node 0 is connected to an external physical GPU and NUMA node 1 is also connected to an external physical GPU, but the available resources of the GPU in NUMA node 1 cannot meet the GPU resource requirements of the target virtual machine, the GPU in NUMA node 0 can still provide GPU resources for the vGPU of the target virtual machine.

[0034] Under this second binding strategy, the external GPU is not limited by the CPU and memory resources of a single NUMA node. This allows the GPU's computing power to be utilized by NUMA nodes within the same group that have more CPU and memory, thereby maximizing GPU utilization. Furthermore, although the target virtual machine's vGPU and vCPU are not bound to the same NUMA node, the NUMA nodes they are bound to belong to the same NUMA node group. Therefore, data exchange between the CPU and GPU will not produce significant latency, ensuring virtual machine performance while fully utilizing GPU computing power.

[0035] (3) Third binding strategy, wherein the third binding strategy is used to indicate that the GPU of the target virtual machine is not bound to any NUMA node or any NUMA node group.

[0036] Understandably, the third binding strategy refers to the GPU not being bound to any NUMA node or any NUMA node group. By adopting an unbound mode, it can break the topology restrictions and achieve global GPU resource pooling allocation. This third binding strategy is suitable for scenarios that aim to maximize concurrent throughput and are not sensitive to latency, such as AI inference and offline transcoding.

[0037] In this embodiment, by providing three binding strategies, fine-grained and flexible management of GPU and NUMA binding is achieved. This not only ensures the ultimate performance in high-performance computing scenarios, but also maximizes the flexibility of resource pooling scheduling in general scenarios. Furthermore, it can resolve computing power bottlenecks through cross-node aggregation in resource-scarce scenarios such as "insufficient number of GPU cards" or "insufficient CPU on a single NUMA".

[0038] In some embodiments, determining a first NUMA node based on a GPU binding policy and target binding status information includes: if the target binding status information indicates that virtualization CPU binding with NUMA node is enabled, determining a first candidate NUMA node from at least one host based on the GPU binding policy, and adding the first candidate NUMA node to the list of available NUMA nodes; if the list of available NUMA nodes is not empty, determining the first NUMA node from the list of available NUMA nodes.

[0039] Understandably, when the target binding status information indicates that vCPU and NUMA node binding is enabled, a matching NUMA node can be selected for vCPU to bind. In determining the first NUMA node, it is not necessary to perform a single "GPU matching check within the same NUMA node." Instead, a check is performed based on the GPU binding policy, thereby identifying NUMA nodes that meet the conditions corresponding to the GPU binding policy as first candidate NUMA nodes and adding them to the list of available NUMA nodes. Then, the first NUMA node is determined from the list of available NUMA nodes. In this embodiment, NUMA node matching based on the GPU binding policy ensures that each first candidate NUMA node or the NUMA node group to which the first candidate NUMA node belongs can provide sufficient GPU resources for the target virtual machine's vGPU.

[0040] In some embodiments, a first candidate NUMA node is determined from at least one host according to a GPU binding strategy, including one of the following: (1) When the GPU binding strategy is the first binding strategy or the third binding strategy, at least one NUMA node in the host that meets the target conditions shall be selected as the first candidate NUMA node. The target conditions include: the NUMA node has idle GPU resources that meet the target configuration requirements, and the target configuration requirements are the configuration requirements corresponding to the target virtual machine.

[0041] The target configuration requirements include GPU specifications, which may include, for example, GPU model, video memory capacity, number of GPUs, and computing power. In some embodiments, the target configuration requirements may also include the number of CPU cores and memory capacity. A NUMA node having available idle GPU resources that meet the target configuration requirements means that the NUMA node has an external physical GPU, that physical GPU has remaining unused idle GPU resources, and that these idle GPU resources meet the target configuration requirements. If this target condition is met, the NUMA node is selected as the first candidate NUMA node.

[0042] In some embodiments, selecting at least one NUMA node in a host that meets the target condition as a first candidate NUMA node includes: traversing each NUMA node in each host, and if the currently traversed NUMA node meets the target condition, then selecting the currently traversed NUMA node as a first candidate NUMA node.

[0043] (2) When the GPU binding strategy is the second binding strategy, the NUMA node in each second NUMA node group that meets the target conditions is taken as the first candidate NUMA node, wherein the second NUMA node group is a NUMA node group in at least one host that has idle GPU resources that meet the target configuration requirements.

[0044] Understandably, the second binding strategy is to bind NUMA nodes to GPUs in groups. That is, the NUMA node group to which the GPU is bound needs to have idle GPU resources that meet the target conditions, and then the NUMA nodes in the group that meet the target conditions are taken as the first candidate NUMA nodes.

[0045] In some embodiments, selecting NUMA nodes in each second NUMA node group that meet the target conditions as first candidate NUMA nodes includes: traversing each NUMA node group; if the currently traversed NUMA node group has idle GPU resources that meet the target configuration requirements, then selecting the currently traversed NUMA node group as a second NUMA node group; traversing each NUMA node in each second NUMA node group; if the currently traversed NUMA node meets the target conditions, then selecting the currently traversed NUMA node as a first candidate NUMA node.

[0046] In some embodiments, selecting NUMA nodes that meet the target conditions in each second NUMA node group as first candidate NUMA nodes includes: traversing each NUMA node in each host; if the NUMA node group to which the currently traversed NUMA node belongs has idle GPU resources that meet the target configuration requirements, selecting NUMA nodes that meet the target conditions in the NUMA node group to which the currently traversed NUMA node belongs as first candidate NUMA nodes, and skipping the traversal of each NUMA node included in the NUMA node group.

[0047] In some embodiments, determining the first NUMA node from the list of available NUMA nodes includes: selecting the first candidate NUMA node with the most idle CPU resources that meets the target configuration requirements from the list of available NUMA nodes as the second candidate NUMA node, wherein the target configuration requirements are the configuration requirements corresponding to the target virtual machine; if there is only one second candidate NUMA node, then the second candidate NUMA node is selected as the first NUMA node; or, if there are multiple second candidate NUMA nodes, then the second candidate NUMA node with the highest priority among the multiple second candidate NUMA nodes is selected as the first NUMA node.

[0048] It's understandable that having the most idle CPU resources refers to having the largest number of unused physical CPU cores. The first candidate NUMA node with the most idle CPU resources that meets the target configuration requirements is selected as the second candidate NUMA node. If there is only one second candidate NUMA node, it is selected as the first NUMA node. If there are multiple second candidate NUMA nodes, the first NUMA node is selected according to priority, and the target virtual machine's vCPUs are bound to that first NUMA node. This effectively improves the processing speed of the target virtual machine during operation and reduces stuttering and response latency, thereby improving virtual machine performance.

[0049] In some embodiments, the priority may be determined based on factors such as load, resource utilization, or the service type corresponding to the target virtual machine.

[0050] In some embodiments, determining the target physical GPU corresponding to the target virtual machine from at least one host according to the first NUMA node and GPU binding policy includes: traversing each physical GPU in each host, performing a candidate judgment on each traversed physical GPU according to the first NUMA node and GPU binding policy, and determining the candidate physical GPU; selecting the candidate physical GPU with the most idle GPU resources that meet the target configuration requirements as the target physical GPU, wherein the target configuration requirements are the configuration requirements corresponding to the target virtual machine.

[0051] In this embodiment, selecting the candidate physical GPU with the most idle GPU resources as the target physical GPU ensures that the vGPU obtains sufficient computing power and avoids computing power fragmentation and performance degradation caused by multiple virtual machines sharing the GPU.

[0052] The candidate determination is used to filter candidate physical GPUs that match the GPU binding strategy. In some embodiments, the candidate determination for each traversed physical GPU is performed based on the first NUMA node and the GPU binding strategy, including: if the GPU binding strategy is the third binding strategy, then the currently traversed physical GPU is selected as a candidate physical GPU; if the GPU binding strategy is the first binding strategy or the second binding strategy, then a matching check is performed on the currently traversed physical GPU and the first NUMA node according to the GPU binding strategy, and if the matching check passes, the currently traversed physical GPU is selected as a candidate physical GPU.

[0053] Understandably, if the GPU binding strategy is the third binding strategy, it means that the vGPU is not bound to any NUMA node or any NUMA node group, i.e., the vGPU is in unbound mode. Therefore, no matching check is needed, and each physical GPU is treated as a candidate physical GPU. If the GPU binding strategy is the first binding strategy or the second binding strategy, it means that the vGPU needs to be bound to a certain NUMA node or a certain NUMA node group. That is, the vGPU is in bound mode, and therefore a matching check needs to be performed between the currently traversed physical GPU and the first NUMA node.

[0054] In some embodiments, physical GPUs that meet the screening criteria can be used as candidate physical GPUs, wherein the screening criteria include: the physical GPU has idle GPU resources that meet the target configuration requirements, and the target configuration requirements are the configuration requirements of the target virtual machine.

[0055] In some embodiments, a matching check is performed on the currently traversed physical GPU and the first NUMA node according to the GPU binding strategy, including: if the GPU binding strategy is the first binding strategy and the currently traversed physical GPU belongs to the first NUMA node, then the matching check is determined to be successful; if the GPU binding strategy is the second binding strategy and the currently traversed physical GPU belongs to the first NUMA node group, then the matching check is determined to be successful, wherein the first NUMA node group includes the first NUMA node.

[0056] Understandably, if the GPU binding strategy is the first binding strategy, it means that the vCPU and vGPU need to be bound to the same NUMA node. Currently, the vCPU is bound to the first NUMA node. If the currently traversed physical GPU also belongs to the first NUMA node, then the matching check passes. If the GPU binding strategy is the second binding strategy, it means that the NUMA node bound to the vCPU and the NUMA node bound to the vGPU need to be in the same NUMA node group. Currently, the vCPU is bound to the first NUMA node. If the currently traversed physical GPU belongs to the first NUMA node group, and the first NUMA node group is a group of nodes that includes the first NUMA node, then the matching check passes. In this embodiment, the matching check ensures that the selected candidate physical GPUs meet the corresponding GPU binding strategy.

[0057] Figure 6 The following diagram illustrates a matching check process provided in some embodiments of this application, such as... Figure 6 As shown, the following steps may be included: S601: Obtain GPU binding policy.

[0058] S602: Traverse each physical GPU.

[0059] S603: Get the physical GPU currently being traversed.

[0060] S604: Determine whether the GPU binding strategy is the first binding strategy, the second binding strategy, or the third binding strategy.

[0061] If the first binding strategy is used, proceed to S605; if the second binding strategy is used, proceed to S606; if the third binding strategy is used, proceed to S607.

[0062] S605: Determine whether the physical GPU being traversed belongs to the first NUMA node and whether it has idle GPU resources that match the target configuration.

[0063] If yes, proceed to S608; otherwise, proceed to S609.

[0064] S606: Determine whether the physical GPU being traversed belongs to the first NUMA node group and whether it has idle GPU resources that match the target configuration.

[0065] The first NUMA node group includes the first NUMA node.

[0066] If yes, proceed to S608; otherwise, proceed to S609.

[0067] S607: Determine whether the physical GPU being traversed currently has available free GPU resources that match the target configuration.

[0068] If yes, proceed to S608; otherwise, proceed to S609.

[0069] S608: Determine that the matching check has passed, and select the currently traversed physical GPU as a candidate physical GPU.

[0070] S609: Determine if the traversal is complete.

[0071] If yes, then the process ends; otherwise, proceed to S610.

[0072] S610: Traverse the next physical GPU.

[0073] In some embodiments, after determining the target physical GPU corresponding to the target virtual machine from at least one host according to the first NUMA node and GPU binding policy, the method further includes: if the GPU binding policy is the first binding policy, then binding the virtualized GPU of the target virtual machine to the NUMA node to which the target physical GPU belongs; if the GPU binding policy is the second binding policy, then binding the virtualized GPU of the target virtual machine to the NUMA node group to which the second NUMA node belongs, wherein the second NUMA node is the NUMA node to which the target physical GPU belongs.

[0074] It's understandable that the first binding strategy refers to a one-to-one binding of the GPU to NUMA nodes, meaning the vCPU and vGPU are bound to the same NUMA node. The target physical GPU is determined based on the first NUMA node and the first binding strategy; therefore, the NUMA node to which the target physical GPU belongs is actually the first NUMA node. The second binding strategy refers to GPUs binding to NUMA nodes in groups, meaning the NUMA nodes bound to the vCPU and vGPU belong to the same NUMA node group. The target physical GPU is determined based on both the first and second binding strategies; therefore, the second NUMA node to which the target physical GPU belongs actually belongs to the same NUMA node group as the first NUMA node.

[0075] Figure 7 The following diagram illustrates a matching check provided in some embodiments of this application, such as... Figure 7 As shown, the following steps may be included: S701: Obtain GPU binding policy.

[0076] S702: Determine whether the GPU binding strategy is the first binding strategy, the second binding strategy, or the third binding strategy.

[0077] If the first binding strategy is used, proceed to S703; if the second binding strategy is used, proceed to S704; if the third binding strategy is used, proceed to S705.

[0078] S703: Select at least one NUMA node in the host that meets the target conditions as the first candidate NUMA node.

[0079] The target conditions include: the NUMA node has idle GPU resources that meet the target configuration requirements, and the target configuration requirements are the configuration requirements corresponding to the target virtual machine.

[0080] S704: Select the NUMA nodes in each second NUMA node group that meet the target conditions as the first candidate NUMA nodes.

[0081] The second NUMA node group is a NUMA node group in at least one host that has idle GPU resources that meet the target configuration requirements.

[0082] S705: Select at least one NUMA node in the host that meets the target conditions as the first candidate NUMA node.

[0083] S706: Add the first candidate NUMA node to the list of available NUMA nodes.

[0084] S707: Determine if the list of available NUMA nodes is empty.

[0085] If yes, then the process ends; otherwise, proceed to S708.

[0086] S708: Select the first candidate NUMA node with the most idle CPU resources in the list of available NUMA nodes that meets the target configuration requirements as the first NUMA node.

[0087] S709: Binds the virtualized CPU and virtualized memory of the target virtual machine to the first NUMA node.

[0088] S710: Based on the first NUMA node and the GPU binding strategy, perform candidate judgment on each physical GPU traversed to determine the candidate physical GPU.

[0089] S711: Select the candidate physical GPU with the most available GPU resources that meet the target configuration requirements as the target physical GPU.

[0090] S712: Determine whether the GPU binding strategy is the first binding strategy or the second binding strategy.

[0091] If the first binding strategy is used, proceed to S713; if the second binding strategy is used, proceed to S714.

[0092] S713: Binds the virtualized GPU of the target virtual machine to the NUMA node to which the target physical GPU belongs.

[0093] S714: Bind the virtualized GPU of the target virtual machine to the NUMA node group where the second NUMA node is located, where the second NUMA node is the NUMA node to which the target physical GPU belongs.

[0094] Based on the above embodiments, in Figure 2 Based on the NUMA architecture shown, assume that NUMA node 0 is connected to an external physical GPU: (1) such as Figure 8 As shown: Since only NUMA node 0 has an external GPU, only NUMA node 0 is available in the list of NUMA nodes. Therefore, the first NUMA node with the most idle CPU resources that meets the target configuration requirements is NUMA node 0. If the current binding strategy is the first binding strategy, both vCPU and vGPU are bound to NUMA node 0. The memory in NUMA node 0 provides memory resources for the target virtual machine's VM memory, the CPU in NUMA node 0 provides CPU resources for the target virtual machine's vCPU, and the GPU in NUMA node 0 provides GPU resources for the target virtual machine's vGPU. At this time, the target virtual machine's GPU memory access speed is the fastest. In this way, data exchange between CPU and GPU is completed with the highest bandwidth and lowest latency.

[0095] (2) such as Figure 9 As shown: Assuming that NUMA node 0 has the most idle CPU resources that meet the target configuration requirements, if the current binding strategy is the first one, then both vCPU and vGPU are bound to NUMA node 0. In this case, the GPU memory access speed of the target virtual machine is the fastest. In this way, data exchange between CPU and GPU is completed with the highest bandwidth and lowest latency. Assuming that NUMA node 1 has the most idle CPU resources that meet the target configuration requirements, if the current binding strategy is the second one, then vCPU is bound to NUMA node 1 and vGPU is bound to NUMA node 0. In this case, although it is across nodes, it is within the same node group, so the GPU memory access speed of the target virtual machine is faster. In this way, the affinity between CPU and local NUMA node is guaranteed, and cross-NUMA node access to GPU is allowed within the group.

[0096] (3) such as Figure 10 As shown: Assuming NUMA node 0 has the most idle CPU resources meeting the target configuration requirements, and if the current binding strategy is the first type, both vCPU and vGPU are bound to NUMA node 0. In this case, the target virtual machine's GPU memory access speed is the fastest, and data exchange between the CPU and GPU is completed with the highest bandwidth and lowest latency. Assuming NUMA node 1 has the most idle CPU resources meeting the target configuration requirements, and if the current binding strategy is the second type, then vCPU is bound to NUMA node 1, and vGPU is bound to NUMA node 0. In this case, the target virtual machine's GPU memory access speed is relatively fast. This approach ensures both the affinity between the CPU and the local NUMA node and allows access to the GPU across NUMA nodes within the group. Assuming that NUMA node 6 has the most idle CPU resources that meet the target configuration requirements, if the current strategy is the third binding strategy, then the vCPU is bound to NUMA node 6, and the vGPU is not bound but provides GPU resources through the GPU on NUMA node 0. In this case, due to the cross-group nature, the GPU memory access speed of the target virtual machine is generally slow. In this approach, NUMA nodes with sufficient CPU resources are allowed to use remote GPUs across nodes. Although this may introduce memory access latency, it improves resource utilization.

[0097] like Figure 11As shown in the illustration, this application also provides a device 1100 for binding virtual machine GPUs and NUMA nodes, including a policy configuration and parsing module 1110, a NUMA topology and resource status management module 1120, an available NUMA node list generation module 1130, an optimal NUMA node selection module 1140, and an optimal GPU selection module 1150. The policy configuration and parsing module 1110 is used to receive GPU binding policies and target binding status information issued by the administrator through an interface, wherein the target binding status information is the binding status information of the virtualization central processing unit (CPU) and NUMA nodes of the target virtual machine; the NUMA topology and resource status management module 1120 is used to maintain the host's hardware topology information, wherein the hardware topology information includes at least one of the following: a list of NUMA nodes and their vCPU counts and memory size; NUMA grouping relationships; the NUMA node or group to which each GPU belongs; and the number of remaining vGPUs recorded according to specifications on each GPU; the available NUMA node list generation module 1130 is used to generate a list of available NUMA nodes based on GPU binding... The strategy and target binding status information are processed, all NUMA nodes are traversed, nodes that meet the conditions are filtered out, and a list of available NUMA nodes is output. The optimal NUMA node selection module 1140 is used to select the NUMA node with the most vCPUs from the list of available NUMA nodes as the first NUMA node, and bind the virtualized CPU and virtualized memory of the target virtual machine to the first NUMA node. The optimal GPU selection module 1150 is used to traverse each GPU on the host, calculate its remaining vGPUs, and select the GPU card with the largest remaining vGPUs that matches the GPU binding strategy as the target physical GPU. This target physical GPU is used to provide GPU resources for the virtualized GPU of the target virtual machine.

[0098] The binding device for virtual machine GPU and NUMA node provided in this application embodiment can realize the various processes implemented in the method embodiment of the above-described binding method for virtual machine GPU and NUMA node. To avoid repetition, it will not be described again here.

[0099] The binding device between a virtual machine GPU and a NUMA node in this application embodiment can be a device, or it can be a component, integrated circuit, or chip in an electronic device. This application embodiment does not impose specific limitations.

[0100] The device for binding a virtual machine GPU and a NUMA node in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit the specific operating system used.

[0101] like Figure 12As shown, this application embodiment also provides an electronic device 1200, including a processor 1210 and a memory 1220. The memory 1220 stores a program or instructions that can run on the processor 1210. When the program or instructions are executed by the processor 1210, they implement the various processes of the above-described embodiment of the virtual machine GPU and NUMA node binding method and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0102] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described method for binding virtual machine GPU and NUMA node, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0103] The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium may include a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. In some examples, the readable storage medium may be a non-transient computer-readable storage medium.

[0104] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described embodiment of the virtual machine GPU and NUMA node binding method, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0105] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0106] This application also provides a computer program / program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to implement the various processes of the above-described virtual machine GPU and NUMA node binding method, and achieve the same technical effect. To avoid repetition, these will not be described again here.

[0107] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0108] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as a second memory, RAM, magnetic disk, optical disk, etc.) and includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

[0109] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims

1. A method for binding a virtual machine GPU to a NUMA node, characterized in that, The method includes: Obtain the GPU binding policy corresponding to the target virtual machine, wherein the GPU binding policy is used to indicate the binding relationship between the virtualized GPU of the target virtual machine and the non-uniform memory access NUMA node; Based on the GPU binding strategy and target binding status information, a first NUMA node is determined, and the virtualized CPU and virtualized memory of the target virtual machine are bound to the first NUMA node. The target binding status information is the binding status information between the virtualized central processing unit (CPU) of the target virtual machine and the NUMA node. The first NUMA node is used to provide CPU resources and memory resources to the target virtual machine through virtualization technology. Based on the first NUMA node and the GPU binding policy, a target physical GPU corresponding to the target virtual machine is determined from at least one host, and GPU resources are allocated to the virtualized GPU of the target virtual machine through the target physical GPU.

2. The method according to claim 1, characterized in that, The GPU binding strategy is one of the following: A first binding strategy, wherein the first binding strategy is used to indicate binding the virtualized GPU of the target virtual machine to the NUMA node to which the virtualized CPU of the target virtual machine is bound; The second binding strategy is used to indicate that the virtualized GPU of the target virtual machine is bound to the first NUMA node group, and the first NUMA node group is the NUMA node group to which the virtualized CPU of the target virtual machine is bound. The third binding strategy is used to indicate that the GPU of the target virtual machine is not bound to any NUMA node or any NUMA node group.

3. The method according to claim 2, characterized in that, The step of determining the first NUMA node based on the GPU binding strategy and target binding status information includes: When the target binding status information indicates that the virtualization CPU and NUMA node binding is enabled, a first candidate NUMA node is determined from the at least one host according to the GPU binding policy, and the first candidate NUMA node is added to the list of available NUMA nodes. If the list of available NUMA nodes is not empty, then the first NUMA node is determined from the list of available NUMA nodes.

4. The method according to claim 3, characterized in that, The step of determining the first candidate NUMA node from the at least one host according to the GPU binding strategy includes: When the GPU binding strategy is the first binding strategy or the third binding strategy, the NUMA node in the at least one host that meets the target conditions is selected as the first candidate NUMA node. The target conditions include: the NUMA node has idle GPU resources that meet the target configuration requirements, and the target configuration requirements are the configuration requirements corresponding to the target virtual machine. When the GPU binding strategy is the second binding strategy, the NUMA node in each second NUMA node group that meets the target conditions is selected as the first candidate NUMA node, wherein the second NUMA node group is the NUMA node group in the at least one host that has idle GPU resources that meet the target configuration requirements.

5. The method according to claim 3, characterized in that, Determining the first NUMA node from the list of available NUMA nodes includes: The first candidate NUMA node with the most idle CPU resources in the list of available NUMA nodes that meets the target configuration requirements is selected as the second candidate NUMA node, wherein the target configuration requirements are the configuration requirements corresponding to the target virtual machine; If there is only one second candidate NUMA node, then the second candidate NUMA node will be used as the first NUMA node; or... If there are multiple second candidate NUMA nodes, the second candidate NUMA node with the highest priority among the multiple second candidate NUMA nodes shall be selected as the first NUMA node.

6. The method according to claim 2, characterized in that, The step of determining the target physical GPU corresponding to the target virtual machine from at least one host based on the first NUMA node and the GPU binding policy includes: Traverse each physical GPU in each host, and perform candidate judgment on each traversed physical GPU according to the first NUMA node and the GPU binding strategy to determine the candidate physical GPU; The candidate physical GPU with the most idle GPU resources that meet the target configuration requirements is selected as the target physical GPU, wherein the target configuration requirements are the configuration requirements corresponding to the target virtual machine.

7. The method according to claim 6, characterized in that, Candidate determination is performed for each physical GPU traversed based on the first NUMA node and the GPU binding strategy, including: If the GPU binding strategy is the third binding strategy, then the currently traversed physical GPU is taken as a candidate physical GPU; If the GPU binding strategy is the first binding strategy or the second binding strategy, then a matching check is performed on the currently traversed physical GPU and the first NUMA node according to the GPU binding strategy. If the matching check passes, the currently traversed physical GPU is taken as the candidate physical GPU.

8. The method according to claim 7, characterized in that, The step of performing a matching check on the currently traversed physical GPU and the first NUMA node according to the GPU binding strategy includes: If the GPU binding strategy is the first binding strategy, and the currently traversed physical GPU belongs to the first NUMA node, then the matching check is determined to be successful. If the GPU binding strategy is the second binding strategy, and the currently traversed physical GPU belongs to the first NUMA node group, then the matching check is determined to be successful, wherein the first NUMA node group includes the first NUMA node.

9. The method according to claim 2, characterized in that, After determining the target physical GPU corresponding to the target virtual machine from at least one host based on the first NUMA node and the GPU binding policy, the method further includes: If the GPU binding strategy is the first binding strategy, then the virtualized GPU of the target virtual machine is bound to the NUMA node to which the target physical GPU belongs; If the GPU binding strategy is the second binding strategy, then the virtualized GPU of the target virtual machine is bound to the NUMA node group where the second NUMA node is located, wherein the second NUMA node is the NUMA node to which the target physical GPU belongs.

10. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method as claimed in any one of claims 1 to 9.

11. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-9.

12. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-9.

Images