Virtual machine scheduling method and device, electronic equipment and readable storage medium

Abstract

The disclosure provides a virtual machine scheduling method and device, electronic equipment and readable storage medium, and relates to the technical field of virtual machines. The virtual machine scheduling method based on a non-uniform memory access (NUMA) system comprises: generating a NUMA topology of CPUs based on a NUMA node topology in the NUMA system and a CPU list of the NUMA node; configuring a corresponding local access control group (Cgroup) under a specified directory based on the NUMA topology and the CPU list; in response to scheduling information, determining a virtual machine to be scheduled, and performing a balancing scheduling operation for the virtual machine to be scheduled based on the Cgroup, so that the virtual machine accesses local memory. Through the technical scheme of the disclosure, the virtual machine accesses local memory, thereby facilitating reduction of latency caused by access.

Description

Technical Field

[0001] This disclosure relates to the field of virtual machine technology, and in particular to a virtual machine scheduling method, a virtual machine scheduling device, an electronic device, and a computer-readable storage medium based on a NUMA system. Background Technology

[0002] In recent years, processor development has shifted from continuously increasing the frequency of individual processors to expanding multiprocessor systems. This shift has brought about a problem: the FSB (Front-Side Bus, referring to the data transfer bus between the CPU and the Northbridge chip, where the speed of the front-side bus refers to the data transfer speed) becomes a focal point of competition among multiple processors. To solve this problem, researchers have migrated the Integrated Memory Controller (IMC) from the Northbridge chip to the processor chip, giving each processor its own IMC connected to local memory. Based on this improvement, NUMA (Non-Uniform Memory Access) systems were developed. Currently, multiple virtual machines on NUMA systems still have the following drawbacks:

[0003] (1) If the CPU of one NUMA node in a NUMA system needs to access the memory on another NUMA node, the access signal needs to go through the Fast Interconnect (QPI), which causes memory access delay.

[0004] (2) Since the scheduling algorithm built into the Linux kernel does not distinguish between VCPU threads and ordinary threads in the virtual machine, even if the scheduled threads do not belong to the same virtual machine, the Linux kernel will still schedule them, resulting in multiple VCPUs in the same virtual machine being scheduled to different NUMA nodes, causing remote memory access, which will also lead to memory access latency.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a virtual machine scheduling method, a virtual machine scheduling device, an electronic device, and a computer-readable storage medium based on a NUMA system, which at least to some extent overcome the problem of virtual machine memory access latency in related technologies.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to one aspect of this disclosure, a virtual machine scheduling method based on a NUMA system is provided, comprising: generating a NUMA topology for the CPUs based on the NUMA node topology in the non-consistent memory access NUMA system and a CPU list of the NUMA nodes; configuring corresponding local access control groups (Cgroups) in a specified directory based on the NUMA topology and the CPU list; and, in response to scheduling information, determining virtual machines to be scheduled, and performing a balanced scheduling operation for the virtual machines to be scheduled based on the Cgroups, so that the virtual machines can access local memory.

[0009] In one embodiment of this disclosure, the scheduling information includes new scheduling information for a newly created virtual machine. The step of determining a virtual machine to be scheduled in response to the scheduling information, and performing a balanced scheduling operation on the virtual machine to be scheduled based on the Cgroup, so that the virtual machine can access local memory, specifically includes: determining a newly created first virtual machine as the virtual machine to be scheduled in response to the new scheduling information; determining the number of VCPUs and the required memory for the first virtual machine; performing a first balanced scheduling operation based on the number of VCPUs and the required memory to determine the NUMA node where the first virtual machine will be placed; and placing the virtual machine to be scheduled in the NUMA node based on the Cgroup, so that the virtual machine can access local memory.

[0010] In one embodiment of this disclosure, the step of performing a first balanced scheduling operation based on the number of VCPUs and the required memory to determine the NUMA node for placing the virtual machine to be scheduled specifically includes: sequentially scanning all NUMA nodes based on the NUMA node topology to determine a first node with the most idle CPUs; when it is detected that the number of idle CPUs of the first node is greater than or equal to the number of VCPUs of the first virtual machine, determining the first node as the NUMA node for placing the first virtual machine; when it is detected that the number of idle CPUs of the first node is less than the number of VCPUs of the first virtual machine, detecting whether the total number of idle CPUs of all NUMA nodes is greater than or equal to the number of VCPUs of the first virtual machine; when it is detected that the total number of idle CPUs of all NUMA nodes is greater than or equal to the number of VCPUs of the first virtual machine, determining the NUMA node for placing the first virtual machine based on cross-NUMA node operation.

[0011] In one embodiment of this disclosure, the scheduling information includes deletion scheduling information for deleting virtual machines. The step of determining a virtual machine to be scheduled in response to the scheduling information, and performing a load balancing operation on the virtual machine to be scheduled based on the Cgroup to enable the virtual machine to access local memory, specifically includes: deleting a subdirectory of a second virtual machine in the Cgroup in response to the deletion scheduling information; and determining the virtual machine to be scheduled and its NUMA node among the remaining virtual machines based on the second load balancing operation; and placing the virtual machine to be scheduled in the NUMA node based on the Cgroup to enable the virtual machine to access local memory.

[0012] In one embodiment of this disclosure, the step of determining the virtual machine to be scheduled and the NUMA node of the virtual machine to be scheduled among the remaining virtual machines based on the second balanced scheduling operation specifically includes: when it is detected that there is a cross-node virtual machine deployed across NUMA nodes among the remaining virtual machines, the cross-node virtual machine is determined as the virtual machine to be scheduled, and the NUMA node that deletes the second virtual machine is determined as the NUMA node to place the virtual machine to be scheduled; and / or when a third node with the most idle CPUs and a fourth node with the most allocated CPUs are detected, a third virtual machine is queried in the fourth node, the third virtual machine being the remaining virtual machine with the smallest number of VCPUs; when it is detected that the idle CPUs in the third node can place the third virtual machine, and after placement, the number of remaining idle CPUs in the third node is still greater than or equal to the number of remaining idle CPUs in the fourth node, the third virtual machine is determined as the virtual machine to be scheduled, and the third node is determined as the NUMA node to place the third virtual machine.

[0013] In one embodiment of this disclosure, placing the virtual machine to be scheduled in the NUMA node based on the Cgroup specifically includes: creating a subdirectory in the corresponding Cgroup based on the instance ID of the virtual machine to be scheduled; configuring the CPU list in the NUMA node corresponding to the virtual machine in a first file of the subdirectory; configuring the memory required by the virtual machine in a second file of the subdirectory; configuring the instance ID in a third file of the subdirectory so as to synchronously migrate the memory when it is detected that the memory of the virtual machine is not in the NUMA node; obtaining the PID value corresponding to the instance ID and configuring the PID value in a fourth file of the subdirectory.

[0014] In one embodiment of this disclosure, configuring corresponding local access control groups (Cgroups) in a specified directory based on the NUMA topology and the CPU list specifically includes: configuring individual NUMA nodes and cross-NUMA nodes based on the NUMA topology; configuring corresponding Cgroups in the specified directory for the individual NUMA node and the cross-NUMA node respectively; and writing available CPU numbers into the Cgroups based on the CPU list, wherein the available CPU numbers of the NUMA node are written into the Cgroup corresponding to the individual NUMA node, and all available CPU numbers in the NUMA system are written into the Cgroup corresponding to the cross-NUMA node.

[0015] According to another aspect of this disclosure, a virtual machine scheduling apparatus based on a NUMA system is provided, comprising: a generation module, configured to generate a NUMA topology of the CPUs based on the NUMA node topology and the CPU list of the NUMA nodes in the non-consistent memory access NUMA system; a configuration module, configured to configure corresponding local access control groups (Cgroups) in a specified directory based on the NUMA topology and the CPU list; and a scheduling module, configured to determine the virtual machines to be scheduled in response to scheduling information, and perform a balanced scheduling operation for the virtual machines to be scheduled based on the Cgroups, so that the virtual machines can access local memory.

[0016] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the virtual machine scheduling method of any of the above-mentioned methods via executing the executable instructions.

[0017] In one embodiment of this disclosure, the electronic device is an edge computing node.

[0018] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements a virtual machine scheduling method based on a NUMA system as described above.

[0019] The virtual machine scheduling scheme based on a NUMA system provided in the embodiments of this disclosure collects the NUMA node topology and corresponding CPU topology of the NUMA system, and configures Cgroups for local access based on this information. By leveraging the memory limiting effect of Cgroups, virtual machines can access local memory. Through the configured Cgroups, when a virtual machine that needs to be scheduled is detected, the virtual machine is balanced and scheduled based on a balanced scheduling operation. On the one hand, scheduling and balancing on the NUMA system on a virtual machine basis can fully explore and utilize the characteristics of the NUMA system to improve the reliability of virtual machine deployment and ensure operating efficiency. On the other hand, virtual machine scheduling based on Cgroup technology, including the scheduling of the CPU and memory corresponding to the virtual machine, enables the virtual machine to access local memory, thereby helping to reduce the latency caused by access.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0022] Figure 1 This diagram illustrates a flowchart of a virtual machine scheduling method based on a NUMA system according to an embodiment of the present disclosure;

[0023] Figure 2 A flowchart illustrating another virtual machine scheduling method based on a NUMA system in an embodiment of this disclosure is shown.

[0024] Figure 3 This invention discloses a flowchart illustrating another virtual machine scheduling method based on a NUMA system in an embodiment of the present disclosure.

[0025] Figure 4 A flowchart is shown in another embodiment of the virtual machine scheduling method based on a NUMA system in this disclosure.

[0026] Figure 5 A flowchart is shown in another embodiment of the virtual machine scheduling method based on a NUMA system in this disclosure.

[0027] Figure 6 A flowchart is shown in another embodiment of the virtual machine scheduling method based on a NUMA system in this disclosure.

[0028] Figure 7 A flowchart is shown in another embodiment of the virtual machine scheduling method based on a NUMA system in this disclosure.

[0029] Figure 8 A schematic diagram of a virtual machine scheduling scheme based on a NUMA system is shown in an embodiment of this disclosure;

[0030] Figure 9 This diagram illustrates another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0031] Figure 10 This diagram illustrates another virtual machine scheduling scheme based on a NUMA system in an embodiment of the present disclosure;

[0032] Figure 11 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0033] Figure 12 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0034] Figure 13 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0035] Figure 14 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0036] Figure 15 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0037] Figure 16 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0038] Figure 17 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0039] Figure 18 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0040] Figure 19 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0041] Figure 20 This diagram illustrates yet another virtual machine scheduling scheme based on a NUMA system in an embodiment of this disclosure;

[0042] Figure 21 This diagram illustrates a virtual machine scheduling device based on a NUMA system according to an embodiment of the present disclosure.

[0043] Figure 22 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0044] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the stated details omitted, or other components, devices, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0045] Furthermore, the accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar components, thus repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0046] For ease of understanding, the terms (abbreviations) used in this application are explained below.

[0047] Cgroups, or control groups, provide a mechanism for controlling the use of resources by a specific group of processes. A Cgroup binds a set of processes to one or more subsystems.

[0048] NUMA (Non-Uniform Memory Access) is an architecture whose biggest advantage is that it can combine dozens (or even hundreds) of CPUs into one server, making it easier to better utilize system performance.

[0049] instance ID: The instance name, which is the instanceId property in the InstanceInfo class. It is a unique identifier that distinguishes different instances within the same service.

[0050] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0051] like Figure 1 As shown, the server executes a virtual machine scheduling method based on a NUMA system, including the following steps:

[0052] Step S102: Generate the NUMA topology of the CPU based on the NUMA node topology and the CPU list of the NUMA nodes in the non-consistent memory access NUMA system.

[0053] When the NUMA system starts up, it collects the NUMA topology status of the NUMA system and the list of CPUs reserved by the NUMA system, thereby obtaining the list of physical CPUs available for virtual machines and the NUMA topology of these CPUs. Since a virtual machine includes multiple VCPUs and a NUMA node includes multiple CPUs, CPU-based NUMA topology is beneficial for the rational deployment of virtual machines.

[0054] Step S104: Configure the corresponding local access control group (Cgroup) in the specified directory based on the NUMA topology and CPU list.

[0055] Specifically, by configuring Cgroups, memory limits based on Cgroups are applied within a NUMA node to enable virtual machines to access local memory.

[0056] In addition, based on the collected list of physical CPUs available for use by virtual machines and the NUMA topology of these CPUs, corresponding Cgroups are created in the directory / sys / fs / Cgroup / CPUset.

[0057] Step S106: In response to the scheduling information, determine the virtual machine to be scheduled, and perform a balanced scheduling operation for the virtual machine to be scheduled based on Cgroup so that the virtual machine can access local memory.

[0058] The scheduling information includes, but is not limited to, scheduling information generated by creating a new virtual machine and scheduling information generated by exiting a virtual machine. When creating a new virtual machine, the virtual machine to be scheduled is the newly created virtual machine. When exiting a virtual machine, if it is necessary to maintain the balance of multiple NUMA nodes, at least one remaining virtual machine is the virtual machine to be scheduled.

[0059] Specifically, based on balanced scheduling operations, it is possible to configure all VCPUs and memory for a virtual machine in a NUMA node as much as possible, ensuring access to local memory and reducing memory access latency.

[0060] In this embodiment, by collecting the NUMA node topology and corresponding CPU topology of the NUMA system, and configuring Cgroups for local access based on this information, virtual machines can access local memory based on the memory limiting effect of Cgroups. Through the configured Cgroups, when a virtual machine that needs to be scheduled is detected, the virtual machine is balanced and scheduled based on the balanced scheduling operation. On the one hand, scheduling and balancing on the NUMA system on a virtual machine basis can fully explore and utilize the characteristics of the NUMA system to improve the reliability of virtual machine deployment and ensure operating efficiency. On the other hand, virtual machine scheduling based on Cgroup technology, including the scheduling of the CPU and memory corresponding to the virtual machine, can ensure that the virtual machine accesses local memory, thereby helping to reduce the latency caused by access.

[0061] like Figure 2 As shown, in one embodiment of this disclosure, the scheduling information includes new scheduling information for newly created virtual machines. Step 106, in response to the scheduling information, determines the virtual machine to be scheduled, and performs a balanced scheduling operation on the virtual machine to be scheduled based on Cgroups to enable the virtual machine to access local memory. A specific implementation includes:

[0062] Step S202: In response to the newly created scheduling information, the newly created first virtual machine is determined as the virtual machine to be scheduled.

[0063] Step S204: Determine the number of VCPUs and the required memory for the first virtual machine.

[0064] Step S206: Perform a first balancing scheduling operation based on the number of VCPUs and the required memory to determine the NUMA node where the first virtual machine will be placed.

[0065] Specifically, when the system is preparing to create a new virtual machine, it will provide the number of VCPUs required for the new virtual machine and the memory size of the new virtual machine along with the information of the new virtual machine. Then, the system will find the NUMA node where the virtual machine needs to be placed through the first balancing scheduling operation.

[0066] Step S208: Place the virtual machine to be scheduled in a NUMA node based on Cgroup so that the virtual machine can access local memory.

[0067] In this embodiment, when a new virtual machine is received, the number of VCPUs and memory required for the new virtual machine are determined. Based on the number of VCPUs and memory, a first balancing scheduling operation is performed in the current NUMA system to find the NUMA node where the virtual machine should be placed. The scheduling configuration is based on Cgroups to ensure that the new virtual machine can be configured within a NUMA node as much as possible, thereby ensuring access to local memory.

[0068] like Figure 3 As shown, in one embodiment of this disclosure, step S206, performing a first balancing scheduling operation based on the number of VCPUs and the required memory to determine the NUMA node for placing the virtual machine to be scheduled, specifically includes:

[0069] Step S302: Based on the NUMA node topology, scan all NUMA nodes sequentially to determine the first node with the most idle CPUs.

[0070] Specifically, all NUMA nodes in the system are scanned sequentially, and the NUMA node with the most free CPUs is identified and designated as the first node, NodeX.

[0071] Step S304: When the number of idle CPUs of the first node is detected to be greater than or equal to the number of VCPUs of the first virtual machine, the first node is determined as the NUMA node where the first virtual machine is placed.

[0072] Specifically, if the free CPU number of the first node NodeX is greater than or equal to the number of VCPUs of the newly created first virtual machine, i.e., the VCPU number, then the first node NodeX is the most suitable NUMA node to place this virtual machine, and the node number X is returned.

[0073] Step S306: When it is detected that the number of idle CPUs of the first node is less than the number of VCPUs of the first virtual machine, check whether the total number of idle CPUs of all NUMA nodes is greater than or equal to the number of VCPUs of the first virtual machine.

[0074] Step S308: When the total number of idle CPUs of all NUMA nodes is detected to be greater than or equal to the number of VCPUs of the first virtual machine, the NUMA node to which the first virtual machine is placed is determined based on cross-NUMA node operation.

[0075] Specifically, if the free CPU number of the first NodeX is less than the VCPU number of the newly created virtual machine, but the sum of the free CPU numbers of all NUMA nodes is greater than or equal to the VCPU number of the newly created virtual machine, it means that the virtual machine needs to place VCPUs across NUMA nodes, and a special value 200 is returned.

[0076] If the above conditions are not met, it means that the physical CPU in the system is insufficient to meet the requirements of the new virtual machine, and a 100 error is returned, indicating that the creation has failed.

[0077] In addition, after creating the first virtual machine, update the data structure that maintains the CPU status within the system to clearly distinguish which CPUs have been allocated and which are idle.

[0078] Check the contents of / proc / $virtual_devicePID / NUMA_maps to confirm whether the virtual machine's memory is bound to the desired NUMA node.

[0079] like Figure 4 As shown, in one embodiment of this disclosure, the scheduling information includes deletion scheduling information for deleting virtual machines. Step 106, in response to the scheduling information, determines the virtual machine to be scheduled, and performs a load balancing operation on the virtual machine to be scheduled based on Cgroups. Another specific implementation of this method to enable the virtual machine to access local memory includes:

[0080] In step S402, in response to the deletion of scheduling information, the subdirectory of the second virtual machine in the Cgroup is deleted.

[0081] Specifically, when a virtual machine exits the system, corresponding deletion scheduling information is generated, and the virtual machine's PID needs to be written to the subdirectory " / sys / fs / Cgroup / CPUset / Cgroup.procs", as well as the previously created Cgroup subdirectory corresponding to that virtual machine is deleted.

[0082] Step S404: Based on the second balancing scheduling operation, determine the virtual machines to be scheduled among the remaining virtual machines and the NUMA nodes of the virtual machines to be scheduled.

[0083] Specifically, due to the presence of a second virtual machine that has exited, an imbalance in the number of virtual machines on the NUMA system may occur. For example, some NUMA nodes may be full of virtual machines while others may be relatively idle. Therefore, it is necessary to rescan the current virtual machine allocation status on the NUMA system and make appropriate adjustments based on the scan results. In other words, the NUMA system needs to be rebalanced, and this balancing is achieved by the second balancing scheduling operation.

[0084] Step S406: Place the virtual machine to be scheduled in a NUMA node based on Cgroup so that the virtual machine can access local memory.

[0085] In this embodiment, when a virtual machine exit is detected, the NUMA system is rescanned, and when it is determined that virtual machine deployment balancing needs to be performed again, the virtual machines to be scheduled among the remaining virtual machines are determined based on the second balancing scheduling operation, and scheduling is performed based on Cgroup to ensure the performance balance of the NUMA system.

[0086] like Figure 5As shown, in one embodiment of this disclosure, step S404, based on the second balancing scheduling operation, determines the virtual machines to be scheduled among the remaining virtual machines and the NUMA nodes of the virtual machines to be scheduled. A specific implementation of this includes:

[0087] In step S502, when a cross-node virtual machine is detected among the remaining virtual machines, the cross-node virtual machine is identified as a virtual machine to be scheduled, and the NUMA node where the second virtual machine was deleted is identified as the NUMA node where the virtual machine to be scheduled is placed.

[0088] Specifically, first check if there are any virtual machines in the current NUMA system that are deployed across NUMA nodes. If so, try to migrate the virtual machine from the cross-NUMA node state to a single NUMA node.

[0089] Step S504: When a third node with the most idle CPUs and a fourth node with the most allocated CPUs are detected, query the fourth node for the third virtual machine. The third virtual machine is the remaining virtual machine with the smallest number of VCPUs.

[0090] Specifically, scan all NUMA nodes, find the NUMA node with the most idle CPUs, and name it the third node nodeF. Scan all NUMA nodes, find the node with the most allocated CPUs, and name it the fourth node nodeB. Ensure that nodeF and nodeB are different nodes. If they are the same node, return directly without migrating the virtual machine. In the fourth node nodeB, find the VM with the smallest VCPU number, such as the third virtual machine vmC.

[0091] Step S506: When it is detected that the idle CPU in the third node can accommodate the third virtual machine, and the number of remaining idle CPUs in the third node after placement is still greater than or equal to the number of remaining idle CPUs in the fourth node, the third virtual machine is determined as the virtual machine to be scheduled, and the third node is determined as the NUMA node to place the third virtual machine.

[0092] Specifically, it checks whether the third node nodeF has enough free CPU to place the third virtual machine vmC. If not, it returns an error and stops the operation.

[0093] If so, check whether the number of free cores on the fourth node (nodeB) is greater than that on the third node (nodeF) after the migration of the third virtual machine (vmC) to the third node is completed. If so, do not migrate to avoid ping-pong during migration.

[0094] like Figure 6As shown, in one embodiment of this disclosure, steps 208 and 406, a specific implementation of placing the virtual machine to be scheduled in the NUMA node based on Cgroup, includes:

[0095] Step S602: Based on the instance ID of the virtual machine to be scheduled, create a subdirectory in the corresponding Cgroup.

[0096] Specifically, a subdirectory is created under the corresponding Cgroup, named using the instanceID that is unique to the entire virtual machine system.

[0097] Step S604: Configure the CPU list in the NUMA node corresponding to the virtual machine in the first file of the subdirectory.

[0098] Specifically, fill the list of physical CPUs corresponding to the virtual machine into the first file CPUset.CPUs in the subdirectory.

[0099] Step S606: Configure the memory required by the virtual machine in the second file in the subdirectory.

[0100] Specifically, the memory size required by the virtual machine is entered into the second file CPUset.mems in the subdirectory.

[0101] Step S608: Configure the instance ID in the third file of the subdirectory to synchronously migrate memory when it is detected that the virtual machine's memory is not in the NUMA node.

[0102] Enter the instance ID into the memory_migrate file in the subdirectory to synchronously migrate memory when the virtual machine's memory is not on the corresponding NUMA node.

[0103] Step S610: Obtain the PID value corresponding to the instance ID and configure the PID value in the fourth file in the subdirectory.

[0104] The PID value refers to the process ID, and the instance ID is the application instance ID.

[0105] Specifically, obtain the PID value of the virtual machine instanceId (qemu-kvm) and fill the PID value into the Cgroup.procs file in the subdirectory.

[0106] In one embodiment of this disclosure, configuring corresponding local access control groups (Cgroups) in a specified directory based on the NUMA topology and CPU list specifically includes: configuring individual NUMA nodes and cross-NUMA nodes respectively based on the NUMA topology; configuring corresponding Cgroups in the specified directory for individual NUMA nodes and cross-NUMA nodes respectively; and writing available CPU numbers into the Cgroups based on the CPU list, wherein the available CPU numbers in the NUMA node are written into the Cgroup corresponding to the individual NUMA node, and all available CPU numbers in the NUMA system are written into the Cgroup corresponding to the cross-NUMA node.

[0107] Specifically, taking two NUMA systems, each containing NUMA nodes, as an example, the process of configuring the corresponding local access control group (Cgroup) includes:

[0108] / sys / fs / Cgroup / CPUset / NUMANode0 represents the Cgroup created for virtual machines placed in NUMA Node0.

[0109] / sys / fs / Cgroup / CPUset / NUMANode1 represents the Cgroup created for virtual machines placed in NUMA Node1.

[0110] / sys / fs / Cgroup / CPUset / NUMANode01 represents a Cgroup created for virtual machines across NUMA nodes.

[0111] Additionally, write the corresponding values ​​into the following file:

[0112] Write the physical CPU number of NUMA Node0 in / sys / fs / Cgroup / CPUset / NUMANode0 / CPUset.CPUs.

[0113] Write the physical CPU number of NUMA Node1 in / sys / fs / Cgroup / CPUset / NUMANode1 / CPUset.CPUs.

[0114] The file / sys / fs / Cgroup / CPUset / NUMANode01 / CPUset.CPUs contains the physical CPU numbers of all CPUs in the system, excluding reserved CPUs.

[0115] like Figure 7 As shown, a virtual machine scheduling method based on a NUMA system according to another embodiment of this disclosure specifically includes:

[0116] like Figure 8 As shown, this embodiment uses an Intel CPU with 48 cores and 2 NUMA nodes (i.e., NUMA node 0 and NUMA node 1, both with 24 CPU cores). Among them, 8 cores (4 CPU cores are reserved respectively) are reserved cores, so there are a total of 40 cores, with 20 cores per NUMA node available for the virtual machine.

[0117] Step S702: When the host starts up, the CPU topology of the NUMA host is parsed by parsing the return string of the shell command lsCPU. The list of reserved CPUs of the NUMA host is collected by parsing the CPUaffinity field of / etc / systemd / system.conf. The two are subtracted to obtain the list of physical CPUs on the host that can be used by virtual machines and the NUMA topology of these CPUs.

[0118] Step S704: Based on the list of physical CPUs available for use by virtual machines and the NUMA topology of these CPUs, create the corresponding Cgroups in the directory / sys / fs / Cgroup / CPUset:

[0119] Cgroup NUMANode0 represents a Cgroup created for virtual machines placed in NUMA Node 0.

[0120] Cgroup NUMANode1 represents a Cgroup created for virtual machines placed on NUMA Node 1.

[0121] Cgroup NUMANode01 represents a Cgroup created to house virtual machines spanning two NUMA Nodes.

[0122] Additionally, write the corresponding values ​​into the following file:

[0123] Write the physical CPU number of NUMA Node0 that can be used by the virtual machine, obtained in step 1, to NUMANode0 / CPUset.CPUs.

[0124] Write the physical CPU number of NUMA Node1 that can be used by the virtual machine, obtained in step 1, to NUMANode1 / CPUset.CPUs.

[0125] Write the physical CPU numbers of NUMA Node0 and Node1 obtained in step 1 that are available for use by the virtual machine to NUMANode01 / CPUset.CPUs.

[0126] Step S706: Create a new 16C-32G virtual machine VM1 and call the balancing algorithm findSuitableNode to find the NUMA Node0 to place this virtual machine.

[0127] C refers to the nucleus.

[0128] The balancing algorithm, also known as the first balancing scheduling operation, scans all NUMA nodes in the system and finds the NUMA node with the most available CPU cores.

[0129] Since both NUMA nodes in the system have 20 cores available at this time, the system returns the first matching NUMA node, Node0.

[0130] Take N CPUs from the list of available CPUs in Node0, starting from the front, and assign them to the N VCPUs of the virtual machine.

[0131] Furthermore, the system updates the data structures that maintain these NUMA CPU states internally, clearly distinguishing which are available and which have already been allocated.

[0132] Step S708: Select the Cgroup corresponding to NUMA Node0 to perform the virtual machine placement operation.

[0133] The corresponding Cgroup is: / sys / fs / Cgroup / CPUset / NUMANode0. Perform the following operations:

[0134] Populate the virtual machine information into the corresponding Cgroup file:

[0135] 1. Use the instanceID, which is unique to the virtual machine in the entire system, as the name, and create a subdirectory under the corresponding Cgroup.

[0136] 2. Fill the list of physical CPUs obtained in step 3 for the virtual machine into the file CPUset.CPUs in the subdirectory.

[0137] 3. Enter the memory size required by the virtual machine into the file CPUset.mems in the subdirectory.

[0138] 4. Fill in the file memory_migrate in the subdirectory to indicate that if the virtual machine's memory was not originally on the corresponding NUMA Node, then the memory needs to be migrated synchronously.

[0139] 5. Enter the PID value corresponding to the virtual machine into the Cgroup.procs file in the subdirectory. The PID value corresponding to the virtual machine can be obtained using the following method.

[0140] 5.1 First, use the shell command `PIDof qemu-kvm` to obtain the PIDs of all virtual machines in the system.

[0141] 5.2 Next, iterate through the contents of / proc / $list / cmdline. If an ID matches, it is the corresponding PID number.

[0142] After completion, such as Figure 8 As shown.

[0143] Step S710: Check the specified content to confirm whether the virtual machine's memory is bound to the expected NUMANode.

[0144] The specified content is the content of / proc / $virtual machine PID / NUMA_maps.

[0145] Step S712: Create a new 16C-32G virtual machine VM2, and call the balancing algorithm findSuitableNode to find the most suitable NUMA Node1 for this virtual machine.

[0146] For details, refer to steps S706 to S710.

[0147] After completion, such as Figure 9 As shown.

[0148] Step S714: Create a new 8C-16G virtual machine VM3. The system will call the balancing algorithm findSuitableNode to find the most suitable NUMA Node for this virtual machine.

[0149] For specific details, refer to steps S706 to S710. The difference is that the balancing algorithm will select to place the virtual machine across NUMA nodes because at this time, the system NUMA Node0 only has 4 available cores and NUMA Node1 only has 4 available cores.

[0150] After completion, such as Figure 10 As shown.

[0151] Step S716: Create a new virtual machine VM4 with 1C-2G. The system will call the balancing algorithm findSuitableNode to find the most suitable NUMA Node for this virtual machine.

[0152] Since the system has no available CPU for the virtual machine at this time, an error will be returned, and the creation of the virtual machine will fail.

[0153] After completion, such as Figure 11 As shown.

[0154] In step S718, when virtual machine VM3 exits, if the system is in a balanced state, the rebalance mechanism will not be triggered.

[0155] After completion, such as Figure 12 As shown.

[0156] Step S720: Create a new 4C-8G virtual machine VM5 and call the balancing algorithm findSuitableNode to find the most suitable NUMA Node0 for this virtual machine.

[0157] Specifically, referring to steps S706 to S710, the balancing algorithm will select NUMA Node0 as the target Node.

[0158] After completion, such as Figure 13 As shown.

[0159] Step S722: Create a new 4C8G virtual machine VM6 and call the balancing algorithm findSuitableNode to find the most suitable NUMA Node1 for this virtual machine.

[0160] Specifically, referring to steps S706 to S710, the balancing algorithm will select NUMA Node1 as the target Node.

[0161] After completion, such as Figure 14 As shown.

[0162] In step S724, when virtual machines VM5 and VM6 exit, the entire system is still in a balanced state, so the rebalance mechanism will not be triggered.

[0163] After completion, such as Figure 15 As shown.

[0164] Step S726: Create a new virtual machine VM7 with 1C2G, and call the balancing algorithm findSuitableNode to find the most suitable NUMA Node0 for this virtual machine.

[0165] Specifically, referring to steps S706 to S710, the balancing algorithm will select NUMA Node0 as the target Node.

[0166] After completion, such as Figure 16 As shown.

[0167] Step S728: Create a new 1C2G virtual machine VM8 and call the balancing algorithm findSuitableNode to find the most suitable NUMA Node1 for this virtual machine.

[0168] Similar to steps 3, 4, and 5, the balancing algorithm will select NUMA Node1 as the target Node.

[0169] After completion, such as Figure 17 As shown.

[0170] Step S730: Create a new 4C8G virtual machine VM9 and call the balancing algorithm findSuitableNode to find the most suitable NUMA Node for this virtual machine.

[0171] The steps are similar to steps 3, 4, and 5, except that the balancing algorithm will select to place the virtual machine across NUMA nodes, because at this time the system NUMA Node0 only has 3 available cores and NUMA Node1 only has 3 available cores.

[0172] When placed across NUMA nodes, the first three cores of the virtual machine are placed on NUMA Node0, and the last core is placed on NUMA Node1.

[0173] After completion, such as Figure 18 As shown.

[0174] In step S732, when the virtual machine VM1 (16C32G) exits, the system NUMA Node0 has 16 free cores and node1 has 2 free cores.

[0175] The system will detect that the NUMA node is unbalanced at this time and will automatically trigger a balancing operation, that is, automatically migrate VM9, which spans the NUMA node, to NUMA Node0.

[0176] Refer to step S708 for the migration process.

[0177] After completion, such as Figure 19 As shown.

[0178] At this point, the system detects that the NUMA nodes are still unbalanced and will automatically trigger a balancing operation again, that is, automatically migrate VM7 (1C2G) from NUMA Node0 to NUMA Node1.

[0179] Refer to step S708 for the migration process.

[0180] After completion, such as Figure 20 As shown.

[0181] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0182] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0183] The following reference Figure 21 This invention describes a virtual machine scheduling apparatus 2100 based on a NUMA system according to this embodiment of the invention. Figure 21 The virtual machine scheduling device 2100 based on the NUMA system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0184] The NUMA-based virtual machine scheduling device 2100 is manifested in the form of a hardware module. Components of the NUMA-based virtual machine scheduling device 2100 may include, but are not limited to: a generation module 2102, used to generate the NUMA topology of CPUs based on the NUMA node topology and the CPU list of the NUMA nodes in the NUMA system, based on non-consistent memory access; a configuration module 2104, used to configure the corresponding local access control groups (Cgroups) in a specified directory based on the NUMA topology and the CPU list; and a scheduling module 2106, used to determine the virtual machines to be scheduled in response to scheduling information, and perform a balanced scheduling operation for the virtual machines to be scheduled based on Cgroups, so that the virtual machines can access local memory.

[0185] In one embodiment of this disclosure, the scheduling information includes new scheduling information for a newly created virtual machine. The scheduling module 2106 is further configured to: in response to the new scheduling information, determine the newly created first virtual machine as a virtual machine to be scheduled; determine the number of VCPUs and the required memory for the first virtual machine; perform a first balanced scheduling operation based on the number of VCPUs and the required memory to determine the NUMA node where the first virtual machine will be placed; and place the virtual machine to be scheduled in the NUMA node based on Cgroup.

[0186] In one embodiment of this disclosure, the scheduling module 2106 is further configured to: sequentially scan all NUMA nodes based on the NUMA node topology to determine the first node with the most idle CPUs; when the number of idle CPUs of the first node is detected to be greater than or equal to the number of VCPUs of the first virtual machine, determine the first node as the NUMA node for placing the first virtual machine; when the number of idle CPUs of the first node is detected to be less than the number of VCPUs of the first virtual machine, detect whether the total number of idle CPUs of all NUMA nodes is greater than or equal to the number of VCPUs of the first virtual machine; when the total number of idle CPUs of all NUMA nodes is detected to be greater than or equal to the number of VCPUs of the first virtual machine, determine the NUMA node for placing the first virtual machine based on cross-NUMA node operation.

[0187] In one embodiment of this disclosure, the scheduling information includes deletion scheduling information for deleting virtual machines. The scheduling module 2106 is further configured to: in response to the deletion scheduling information, delete the subdirectory of the second virtual machine in the Cgroup; and based on the second balancing scheduling operation, determine the virtual machines to be scheduled among the remaining virtual machines and the NUMA nodes of the virtual machines to be scheduled; and place the virtual machines to be scheduled in the NUMA nodes based on the Cgroup.

[0188] In one embodiment of this disclosure, the scheduling module 2106 is further configured to: when a cross-node virtual machine deployed across NUMA nodes is detected among the remaining virtual machines, determine the cross-node virtual machine as a virtual machine to be scheduled, and determine the NUMA node where the second virtual machine is deleted as the NUMA node to place the virtual machine to be scheduled; and / or when a third node with the most idle CPUs and a fourth node with the most allocated CPUs are detected, query the fourth node for the third virtual machine, wherein the third virtual machine is the remaining virtual machine with the smallest number of VCPUs; when it is detected that the idle CPUs in the third node can accommodate the third virtual machine, and the number of remaining idle CPUs in the third node after placement is still greater than or equal to the number of remaining idle CPUs in the fourth node, determine the third virtual machine as a virtual machine to be scheduled, and determine the third node as the NUMA node to place the third virtual machine.

[0189] In one embodiment of this disclosure, the scheduling module 2106 is further configured to: create a subdirectory in the corresponding Cgroup based on the instance ID of the virtual machine to be scheduled; configure the CPU list in the NUMA node corresponding to the virtual machine in a first file of the subdirectory; configure the memory required by the virtual machine in a second file of the subdirectory; configure the instance ID in a third file of the subdirectory so as to synchronously migrate the memory when it is detected that the memory of the virtual machine is not in the NUMA node; obtain the PID value corresponding to the instance ID, and configure the PID value in a fourth file of the subdirectory.

[0190] Configuration module 2104 is also used to: configure a single NUMA node and cross-NUMA nodes respectively based on the NUMA topology; configure corresponding Cgroups in a specified directory for a single NUMA node and cross-NUMA nodes respectively; and write available CPU numbers into the Cgroups based on the CPU list, wherein, in the Cgroup corresponding to a single NUMA node, the available CPU numbers in the NUMA node are written, and in the Cgroup corresponding to a cross-NUMA node, all available CPU numbers in the NUMA system are written.

[0191] The following reference Figure 22 To describe an electronic device 2200 according to this embodiment of the present invention. Figure 22 The electronic device 2200 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0192] like Figure 22 As shown, the electronic device 2200 is manifested in the form of a general-purpose computing device. The components of the electronic device 2200 may include, but are not limited to: at least one processing unit 2210, at least one storage unit 2220, and a bus 2230 connecting different system components (including storage unit 2220 and processing unit 2210).

[0193] The storage unit stores program code, which can be executed by the processing unit 2210 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 2210 can perform, as follows: Figure 1 Steps S102 to S106 shown herein, and other steps defined in the virtual machine scheduling method based on a NUMA system of this disclosure.

[0194] Storage unit 2220 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 22201 and / or cache memory 22202, and may further include a read-only memory (ROM) 22203.

[0195] Storage unit 2220 may also include a program / utility 22204 having a set (at least one) program module 22205, such program module 22205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0196] Bus 2230 can represent one or more of several types of bus structures, including memory cell bus or memory cell controller, peripheral bus, graphics acceleration port, processing unit, or local bus using any of the multiple bus structures.

[0197] Electronic device 2200 can also communicate with one or more external devices 2270 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device 2200 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 2250. Furthermore, electronic device 2200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 2260. As shown, network adapter 2260 communicates with other modules of electronic device 2200 via bus 2230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0198] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0199] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the present invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the present invention described in the "Exemplary Methods" section above.

[0200] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0201] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0202] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0203] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0204] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0205] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0206] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0207] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A virtual machine scheduling method based on a NUMA system, characterized in that, include: Based on the NUMA node topology in the non-consistent memory access NUMA system and the CPU list of the NUMA nodes, the NUMA topology of the CPU is generated. Based on the NUMA topology and the CPU list, configure the corresponding local access control group (Cgroup) in the specified directory, including: configuring cross-NUMA nodes based on the NUMA topology, configuring the corresponding Cgroup for the cross-NUMA nodes in the specified directory, and writing all available CPU numbers in the NUMA system into the Cgroup based on the CPU list. In response to scheduling information, a virtual machine to be scheduled is determined, and a load balancing operation is performed on the virtual machine to be scheduled based on the Cgroup to enable the virtual machine to access local memory. The scheduling information includes deletion scheduling information for deleting a virtual machine, including: in response to the deletion scheduling information, deleting the subdirectory of the exiting second virtual machine in the Cgroup; when a cross-node virtual machine deployed across NUMA nodes is detected among the remaining virtual machines, the cross-node virtual machine is determined as the virtual machine to be scheduled, and the NUMA node where the second virtual machine is deleted is determined as the NUMA node where the virtual machine to be scheduled is placed.

2. The virtual machine scheduling method based on a NUMA system according to claim 1, characterized in that, The scheduling information includes new scheduling information for newly created virtual machines. In response to the scheduling information, the virtual machine to be scheduled is determined, and a load balancing operation is performed on the virtual machine to be scheduled based on the Cgroup, so that the virtual machine can access local memory. Specifically, this includes: In response to the newly created scheduling information, the newly created first virtual machine is determined as the virtual machine to be scheduled; Determine the number of VCPUs and the required memory for the first virtual machine; A first balanced scheduling operation is performed based on the number of VCPUs and the required memory to determine the NUMA node where the first virtual machine is placed. The virtual machine to be scheduled is placed in the NUMA node based on the Cgroup, so that the virtual machine can access local memory.

3. The virtual machine scheduling method based on a NUMA system according to claim 2, characterized in that, The first balanced scheduling operation, based on the number of VCPUs and the required memory, to determine the NUMA node for placing the virtual machine to be scheduled, specifically includes: Based on the NUMA node topology, all NUMA nodes are scanned sequentially to determine the first node with the most idle CPUs. When the number of idle CPUs of the first node is detected to be greater than or equal to the number of VCPUs of the first virtual machine, the first node is determined as the NUMA node where the first virtual machine is placed. When it is detected that the number of idle CPUs of the first node is less than the number of VCPUs of the first virtual machine, it is detected whether the total number of idle CPUs of all NUMA nodes is greater than or equal to the number of VCPUs of the first virtual machine. When the total number of idle CPUs of all the NUMA nodes is detected to be greater than or equal to the number of VCPUs of the first virtual machine, the NUMA node to which the first virtual machine is placed is determined based on cross-NUMA node operation.

4. The virtual machine scheduling method based on a NUMA system according to claim 1, characterized in that, The scheduling information includes deletion scheduling information for deleting virtual machines. In response to the scheduling information, the process involves determining the virtual machine to be scheduled, and performing a load balancing operation on the virtual machine based on the Cgroup to enable it to access local memory. Specifically, this includes: Based on the second balancing scheduling operation, the virtual machines to be scheduled and their NUMA nodes are determined from the remaining virtual machines. The virtual machine to be scheduled is placed in the NUMA node based on the Cgroup, so that the virtual machine can access local memory.

5. The virtual machine scheduling method based on a NUMA system according to claim 4, characterized in that, The step of determining the virtual machines to be scheduled and their NUMA nodes from the remaining virtual machines based on the second balancing scheduling operation specifically includes: When a third node with the most idle CPUs is detected, and a fourth node with the most allocated CPUs is detected, a third virtual machine is queried in the fourth node, which is the remaining virtual machine with the smallest number of VCPUs. When it is detected that the idle CPU in the third node can accommodate the third virtual machine, and after placement, the number of remaining idle CPUs in the third node is still greater than or equal to the number of remaining idle CPUs in the fourth node, the third virtual machine is determined as the virtual machine to be scheduled, and the third node is determined as the NUMA node where the third virtual machine is placed.

6. The virtual machine scheduling method based on a NUMA system according to claim 2 or 4, characterized in that, The step of placing the virtual machine to be scheduled in the NUMA node based on the Cgroup specifically includes: Based on the instance ID of the virtual machine to be scheduled, a subdirectory is created in the corresponding Cgroup; Configure the CPU list in the NUMA node corresponding to the virtual machine in the first file of the subdirectory; Configure the memory required by the virtual machine in the second file of the subdirectory; Configure the instance ID in the third file of the subdirectory to synchronously migrate the memory when it is detected that the memory of the virtual machine is not in the NUMA node; Obtain the PID value corresponding to the instance ID, and configure the PID value in the fourth file of the subdirectory.

7. The virtual machine scheduling method based on a NUMA system according to any one of claims 1 to 5, characterized in that, The configuration of the corresponding local access control group (Cgroup) in the specified directory based on the NUMA topology and the CPU list specifically includes: Configure a single NUMA node and cross-NUMA nodes based on the NUMA topology; Configure the corresponding Cgroup in the specified directory for both the individual NUMA node and the cross-NUMA node; Based on the CPU list, the available CPU numbers are written into the Cgroup. Specifically, the available CPU number in the NUMA node is written into the Cgroup corresponding to the individual NUMA node.

8. A virtual machine scheduling device based on a NUMA system, characterized in that, include: A generation module is used to generate the NUMA topology of the CPUs in the NUMA system based on non-consistent memory access and the CPU list of the NUMA nodes. The configuration module is used to configure the corresponding local access control group (Cgroup) in a specified directory based on the NUMA topology and the CPU list, including: configuring cross-NUMA nodes based on the NUMA topology, configuring the corresponding Cgroup for the cross-NUMA nodes in the specified directory, and writing all available CPU numbers in the NUMA system into the Cgroup based on the CPU list. A scheduling module is configured to, in response to scheduling information, determine a virtual machine to be scheduled, and perform a balanced scheduling operation on the virtual machine to be scheduled based on the Cgroup, so that the virtual machine can access local memory. The scheduling information includes deletion scheduling information for deleting a virtual machine, including: in response to the deletion scheduling information, deleting the subdirectory of the exiting second virtual machine in the Cgroup; when a cross-node virtual machine deployed across NUMA nodes is detected among the remaining virtual machines, determining the cross-node virtual machine as the virtual machine to be scheduled, and determining the NUMA node where the second virtual machine was deleted as the NUMA node where the virtual machine to be scheduled is placed.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the virtual machine scheduling method based on any one of claims 1 to 7 via executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the virtual machine scheduling method based on the NUMA system as described in any one of claims 1 to 7.

Images