Memory balancing method and device, electronic equipment and medium

Abstract

The invention provides a memory balancing method and device, electronic equipment and a medium. The method comprises the following steps: determining a primary hot NUMA node and a secondary hot NUMA node from a plurality of NUMA nodes according to the cross-node memory access amount of the plurality of NUMA nodes; according to the cross-NUMA memory access bandwidth of each core in the main hot NUMA node and the secondary hot NUMA node, determining a hot core in the main hot NUMA node and the secondary hot NUMA node; determining a target virtual machine in the virtual machine set according to the cross-NUMA memory space usage amount of each virtual machine in the virtual machine set corresponding to the hotspot core; and according to the NUMA real-time bandwidth and the system NUMA bandwidth, carrying out migration cross-NUMA memory operation on the target virtual machine.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of cloud computing, and particularly relates to a memory balancing method and device, electronic equipment and medium. BACKGROUND

[0002] Non-uniform memory access (NUMA) is a computing system composed of multiple nodes, all of which share aggregated memory. Each CPU is assigned its own local memory and can access memory from other CPUs in the system. NUMA is used to connect multiple central processing units (CPUs) to any number of available memory on a computer.

[0003] For a server virtualization scenario, as multiple virtual machines run for a long time, there is a performance problem of cross- NUMA access. SUMMARY

[0004] To overcome the problems in the related art, the present specification provides a memory balancing method and device, electronic equipment and medium.

[0005] According to a first aspect of an embodiment of the present specification, a memory balancing method is provided, the method comprising: determining a primary hot NUMA node and a secondary hot NUMA node from a plurality of non-uniform memory access (NUMA) nodes according to cross-node memory access amounts of the plurality of NUMA nodes; determining a hot core in the primary hot NUMA node and the secondary hot NUMA node according to cross- NUMA memory access bandwidths of each core in the primary hot NUMA node and the secondary hot NUMA node; determining a target virtual machine in a virtual machine set corresponding to the hot core according to cross- NUMA memory space usage amounts of each virtual machine in the virtual machine set; and performing a migration cross- NUMA memory operation on the target virtual machine according to a NUMA real-time bandwidth and a system NUMA bandwidth.

[0006] According to a second aspect of an embodiment of the present specification, a memory balancing device is provided, comprising: a hot node determination module configured to determine a primary hot NUMA node and a secondary hot NUMA node from a plurality of non-uniform memory access (NUMA) nodes according to cross-node memory access amounts of the plurality of NUMA nodes; a hot core determination module configured to determine a hot core in the primary hot NUMA node and the secondary hot NUMA node according to cross- NUMA memory access bandwidths of each core in the primary hot NUMA node and the secondary hot NUMA node; a target virtual machine determination module configured to determine a target virtual machine in a virtual machine set corresponding to the hot core according to cross- NUMA memory space usage amounts of each virtual machine in the virtual machine set; and a migration module configured to perform a migration cross- NUMA memory operation on the target virtual machine according to a NUMA real-time bandwidth and a system NUMA bandwidth.

[0007] According to a third aspect of the embodiments of the present specification, an electronic device is provided, comprising:

[0008] a processor;

[0009] a memory for storing processor-executable instructions;

[0010] The processor is configured to perform the memory balancing method of the first aspect or any of the corresponding embodiments thereof.

[0011] According to a fourth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, and the computer-readable storage medium stores computer instructions for causing a computer to perform the memory balancing method of the first aspect or any of the corresponding embodiments thereof.

[0012] The technical solutions provided by the embodiments of the present specification can include the following beneficial effects:

[0013] In the embodiments of the present specification, the access heat of the NUMA node and the occupation multi-factor of the NUMA node to the space are comprehensively considered to balance the memory of the NUMA system, which can reduce the influence of the cross- NUMA memory migration on the system performance.

[0014] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present specification. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings incorporated in the specification and constituting a part of the specification illustrate embodiments consistent with the present specification and serve together with the specification to explain the principles of the present specification.

[0016] Figure 1 is a schematic diagram of a NUMA system according to an exemplary embodiment of the present specification.

[0017] Figure 2 is a flowchart of a memory balancing method according to an exemplary embodiment of the present specification.

[0018] Figure 3A is a schematic diagram of the execution result of the numastat command according to an exemplary embodiment of the present specification.

[0019] Figure 3B is a schematic diagram of the execution result of the mlc command according to an exemplary embodiment of the present specification.

[0020] Figure 3C is a schematic diagram of the execution result of the Intelpcm command according to an exemplary embodiment of the present specification.

[0021] Figure 4is a hardware structure diagram of a computer device where a memory balancing apparatus in an embodiment of the present specification is located.

[0022] Figure 5 is a block diagram of a memory balancing apparatus according to an exemplary embodiment of the present specification. DETAILED DESCRIPTION

[0023] The exemplary embodiments will be described in detail herein below with reference to the accompanying drawings. In the following description, the same drawings refer to the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present specification. Rather, they are merely examples of devices and methods consistent with some aspects of the present specification, as detailed in the appended claims.

[0024] The terminology used in the present specification is for the purpose of describing particular embodiments only and is not intended to limit the present specification. As used in the present specification and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0025] It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used merely as labels to identify particular information. For example, a first information can be termed a second information, and similarly, a second information can be termed a first information, without departing from the scope of the present specification. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "in response to determining."

[0026] Next, the embodiments of the present specification will be described in detail.

[0027] The following description is provided in conjunction with Figure 1 The NUMA system to which the memory balancing method and apparatus of the embodiments of the present specification can be applied will be described. It should be noted that, Figure 1 The system architecture shown is merely an example of a system architecture to which the embodiments of the present specification can be applied, to help those skilled in the art understand the technical content of the present specification, but does not mean that the embodiments of the present specification cannot be used for other devices, systems, environments or scenarios.

[0028] Figure 1 is a schematic diagram of a NUMA system according to an exemplary embodiment of the present specification.

[0029] As Figure 1As shown, the NUMA system may, for example, include a plurality of NUMA nodes, such as NUMA1, NUMA2, NUMA3, and NUMA4.

[0030] Each NUMA node has its own physical memory, and a virtual machine in each NUMA node can access local memory and the memory of other NUMA nodes. For example, a virtual machine vm in NUMA1 can access the memory in NUMA2, NUMA3, and NUMA4, and the vm can be configured with other NUMA usage space and access bandwidth.

[0031] Accessing local memory is faster, and accessing remote memory is slower. The access speed is related to the distance between nodes, and the farther the distance, the slower the access speed. When a virtual machine starts, according to the system memory space distribution, the node with more remaining memory space is preferred, and the CPU of the virtual machine is bound to the corresponding core, and the system memory allocation strategy is local, and the virtual machine memory will be allocated on the local NUMA node first.

[0032] When the system single NUMA memory resource cannot fully meet the virtual machine memory, the virtual machine can be allocated a plurality of NUMA nodes, but the virtual machine still has a cross- NUMA access memory situation. The system can periodically evaluate the memory usage of each NUMA, and migrate the virtual machine using the memory of a NUMA, such as NUMA1, to a cross- NUMA to achieve NUMA balancing.

[0033] The memory balancing method provided by the embodiments of the present specification will be described in detail below. As shown in the following Figure 2 As shown, Figure 2 is a flowchart of a memory balancing method according to an exemplary embodiment of the present specification, and the memory balancing method provided by the embodiments of the present specification can include the steps shown below.

[0034] In step 210, a primary hot NUMA node and a secondary hot NUMA node are determined from a plurality of NUMA nodes according to cross-node memory access amounts of the plurality of non-uniform memory access NUMA nodes.

[0035] According to the embodiments of the present specification, for example, the change value of the cross-node memory access bandwidth of each NUMA node in a preset period can be determined according to the cross-node memory access amounts of the plurality of NUMA nodes. The preset period can be set according to actual needs. Then, the NUMA node with the largest change value of the cross-node memory access bandwidth in the preset period is determined as the primary hot NUMA node, and the NUMA node with the second largest change value of the cross-node memory access bandwidth in the preset period is determined as the secondary hot NUMA node.

[0036] For example, the cross-node memory access amount of the NUMA node at time t0 and time t1 within a preset period can be obtained, the cross-node memory access bandwidths at time t0 and time t1 are subtracted to obtain the change value of the cross-node memory access bandwidth of the NUMA node within the preset period. Alternatively, for example, the system tool numastat can be used to query the other_node count of each NUMA node, that is, to obtain the memory amount accessed from other NUMA nodes by the processes running on each NUMA node, that is, the cross-node memory access amount of the NUMA node.

[0037] In step 220, the hotspot core in the main hot NUMA node and the secondary hot NUMA node is determined according to the cross- NUMA memory access bandwidth of each core in the main hot NUMA node and the secondary hot NUMA node.

[0038] According to an embodiment of the present specification, on the one hand, the mean value of the cross- NUMA memory access bandwidth of each core in the main hot NUMA node can be calculated, and the core with the maximum mean value of the cross- NUMA memory access bandwidth in the main hot NUMA node is determined as the hotspot core. On the other hand, the mean value of the cross- NUMA memory access bandwidth of each core in the secondary hot NUMA node can be calculated, and the core with the maximum mean value of the cross- NUMA memory access bandwidth in the secondary hot NUMA node is determined as the hotspot core.

[0039] It can be understood that the operation of determining the hotspot core in the main hot NUMA node and the operation of determining the hotspot core in the secondary hot NUMA node can be executed simultaneously or sequentially in any order.

[0040] Alternatively, for example, the performance monitoring tool can be used to read the cross- NUMA memory access bandwidth of each core in the main hot NUMA node and the secondary hot NUMA node. The performance monitoring tool may, for example, include Intel PCM, or for CPUs of ARM architecture, may, for example, include the SPE tool.

[0041] In step 230, the target virtual machine in the virtual machine set corresponding to the hotspot core is determined according to the cross- NUMA memory space usage of each virtual machine in the virtual machine set.

[0042] According to an embodiment of the present specification, for each hotspot core, for example: the virtual machine processes corresponding to each virtual machine in the virtual machine set corresponding to the hotspot core are determined; the total amount of non- local memory allocation of each virtual machine process is determined; and the virtual machine corresponding to the virtual machine process with the minimum total amount of non- local memory allocation is determined as the candidate virtual machine corresponding to the hotspot core. Then, the target virtual machine in the candidate virtual machine can be determined according to the heat value parameter and the space usage parameter corresponding to each candidate virtual machine.

[0043] The heat value parameter can be equal to the product of the heat value per unit time and a heat value weight, for example. The space usage parameter can be equal to the product of the total amount of non-node memory allocation and a space usage parameter weight, for example. The heat value weight and the space usage parameter weight can be in the range of 0 to 1, and the sum of the heat value weight and the space usage parameter weight is 1. The heat value weight and the space usage parameter weight can be set according to time requirements.

[0044] Optionally, the virtual machine ID occupying the core can be found according to the ID of the hot core, for example, using the virsh vcpuinfo info command. Then the corresponding virtual machine process ID can be identified according to the virtual machine ID, using the process management tool ps aux. Then the total amount of non-node memory allocation corresponding to the virtual machine process ID can be retrieved.

[0045] In step 240, the target virtual machine is migrated for the cross- NUMA memory operation according to the NUMA real-time bandwidth and the system NUMA bandwidth.

[0046] According to an embodiment of the present specification, the system NUMA bandwidth of the plurality of NUMA nodes can be averaged to obtain a system NUMA bandwidth average, for example. The system NUMA bandwidth can include a cross- NUMA bandwidth upper limit. Then, it is determined whether the NUMA real-time bandwidth of the NUMA node corresponding to the target virtual machine is less than the product of the system NUMA bandwidth average and a weight. If the NUMA real-time bandwidth of the NUMA node corresponding to the target virtual machine is less than the product of the system NUMA bandwidth average and the weight, the target virtual machine is migrated for the cross- NUMA memory operation. Otherwise, the target virtual machine is not migrated for the cross- NUMA memory operation.

[0047] The weight can be set according to actual requirements, for example. Illustratively, in the present embodiment, it can be preset that 20% bandwidth is required to migrate from other NUMA nodes to the NUMA node corresponding to the core where the target virtual machine is located, and then the weight = 1-20% = 0.8.

[0048] Optionally, the system NUMA bandwidth of each node in the system can be detected using a performance monitoring tool, such as the mlc (Memory Latency Checker), for example.

[0049] According to an embodiment of the present specification, the memory balancing of the NUMA system is performed by comprehensively considering the access heat of the NUMA node and the space occupation of the NUMA node, which can reduce the impact of cross- NUMA memory migration on system performance.

[0050] Another exemplary memory balancing method is described below. Illustratively, in the present embodiment, the memory balancing method can include the following steps:

[0051] In step 301, a given cycle is set at time t0, and the system tool numastat is used to calculate the other_node count of each NUMA node, which is recorded as numa1t0, numa2t0, etc.

[0052] like Figure 3A , which is a schematic diagram of the execution result of the numastat command according to an exemplary embodiment.

[0053] In step 302, a given cycle is set to time t1, and the system tool numastat is used to calculate the other_node count of each NUMA node, which is recorded as numa1t1, numa2t1, etc.

[0054] In step 303, the maximum change value within the cycle is calculated, numaax1t1-numax1t0=

[0055] Max{numa1t1-numa1t0,numa2t1-numa2t0,……}.

[0056] In step 304, the second largest change value numaax2t1-numax2t0 in the calculation cycle is calculated. <numaax1t1-numax1t0。

[0057] In step 305 , the primary hot NUMA node is selected as numax1 and the secondary hot NUMA node is selected as numax2, and their heat values ​​per unit time are numax1hot and numax2hot respectively.

[0058] In step 306, the performance monitoring tool mlc is used to detect the NUMA bandwidth between nodes in the system, which is recorded as Numa0-1band, Numa0-2band, ...

[0059] like Figure 3B FIG. 1 is a schematic diagram of an execution result of an mlc command according to an exemplary embodiment.

[0060] In step 307, a performance monitoring tool is used to detect the hotspot cores in the numax1 node during the detection period.

[0061] In step 308, a given cycle is set at time t0, and a performance monitoring tool such as Intel PCM (SPE tool is used for ARM CPU) is used to read the cross-NUMA memory access bandwidth of each core of numax1, which is recorded as coreat0, corebt0, ...

[0062] like Figure 3CFig. 3 is a diagram illustrating an execution result of an Intel pcm command according to an exemplary embodiment.

[0063] At step 309, given a period, set time t1, use performance monitoring tools, such as Intel PCM (use SPE tools for ARM CPU) to read the cross-NUMA memory bandwidth usage of numax1 cores, and use the system tool numastat to count the other_node of each NUMA node, denoted as coreat1, corebt1,....

[0064] At step 310, calculate the average value, corea = Avg{coreat0, coreat1,....}, and coreb = Avg{corebt0, corebt1,....}.

[0065] At step 311, assume corea = Max{corea, coreb,....}, select the hot core as numax1corea1, and the corresponding hot value as corea.

[0066] At step 312, similarly, for numax2, find the hot core in the numax2 node as numax2corea2, and the corresponding hot value as coreb.

[0067] For the main NUMA node numax1, perform the following steps:

[0068] At step 313, according to CORE ID: numax1corea1, use virsh vcpuinfo info to find the virtual machine ID occupying the core numax1corea1 as vm1, vm2, vm3.

[0069] At step 314, according to the virtual machine ID, use the process management tool ps aux to identify the corresponding virtual machine process ID as vmproc1, vmproc2, vmproc3.

[0070] At step 315, according to the virtual machine process vmproc1, execute the command / proc / vmproc1 / numa_maps, and in the execution result, retrieve NX = xxx (X represents the node number, and xxx represents the number of pages on the node), calculate the total memory allocation of the memory allocated on the non-node, denoted as vm1numay. Example: N0 = 1234, N1 = 567 indicates that there are 1234 pages on node 0 and 567 pages on node 1.

[0071] At step 316, according to the virtual machine process vmproc2, execute the command / proc / vmproc2 / numa_maps, and in the execution result, retrieve NX = xxx (X represents the node number, and xxx represents the number of pages on the node), calculate the total memory allocation of the memory allocated on the non-node, denoted as vm2numay.

[0072] At step 317, according to the virtual machine process vmproc3, execute the command / proc / vmproc3 / numa_maps, and in the execution result, retrieve NX = xxx (X represents the node number, and xxx represents the number of pages on the node), calculate the total memory allocation of the memory allocated on the non-node, denoted as vm3numay.

[0073] At step 316, according to the virtual machine process vmproc2, NX=xxx of / proc / vmproc2 / numa_maps is retrieved, and the total memory allocation amount of the memory allocated in the non-node is calculated, recorded as vm2numay.

[0074] At step 317, according to the virtual machine process vmproc3, NX=xxx of / proc / vmproc3 / numa_maps is retrieved, and the total memory allocation amount of the memory allocated in the non-node is calculated, recorded as vm3numay.

[0075] At step 318, assuming vm2numay=Min(vm1numay,vm2numay,vm3numay), the initial cold point vm of numaax1 is vm2.

[0076] For the secondary hot NUMA node numax2, the following steps are performed:

[0077] At step 319, according to numax2corea2, the virtual machine ID vm4, vm5, vm6 occupying the core numax2corea2 is found using virsh vcpuinfo.

[0078] At step 320, according to the virtual machine ID, the virtual machine process ID vmproc4, vmproc5, vmproc6 is identified using the process management tool ps aux.

[0079] At step 321, according to the virtual machine process vmproc4, NX=xxx of / proc / vmproc4 / numa_maps is retrieved, and the total memory allocation amount of the memory allocated in the non-node is calculated, recorded as vm4numay.

[0080] At step 322, according to the virtual machine process vmproc5, NX=xxx of / proc / vmproc5 / numa_maps is retrieved, and the total memory allocation amount of the memory allocated in the non-node is calculated, recorded as vm5numay.

[0081] At step 323, according to the virtual machine process vmproc6, NX=xxx of / proc / vmproc6 / numa_maps is retrieved, and the total memory allocation amount of the memory allocated in the non-node is calculated, recorded as vm6numay.

[0082] At step 324, assuming vm4numay=Min(vm4numay,vm5numay,vm6numay), the initial cold point vm of numaax2 is vm4.

[0083] In step 325, preset weight values are obtained, such as a weight coefficient of 0.6 for the hotness value parameter and a weight coefficient of 0.4 for the space usage parameter.

[0084] In step 326, it is assumed that numax1hot*0.6+0.4*vm2numay=Min

[0085] (numax1hot*0.6+0.4*vm2numay, numax2hot*0.6+0.4*vm4numay), the cold point vm is selected as vm2.

[0086] In step 327, for vm2, the numaid corresponding to the core is numax1, and it is preset that migration from other NUMA nodes to numax1 needs to occupy 20% bandwidth.

[0087] According to Numa0-1band, Numa0-2band, …, the upper limit average cross- NUMA bandwidth coreabandtop of numax1corea1 is calculated.

[0088] coreabandtop=Avg(Numax1-1band, Numax1-2band, …), Numax1-1band represents the upper limit of the cross- NUMA bandwidth between the NUMA node where numax1corea1 is located and the first NUMA node.

[0089] In step 328, if corea<0.8*coreabandtop, the migration can be performed, otherwise, the next period is waited until the condition is met.

[0090] In step 329, the migration is performed, and the system memory interface is called to migrate the other NUMA memory of vm2 to numax1.

[0091] In step 330, the round ends, and the next round of memory balancing is performed.

[0092] According to the embodiments of the present specification, for memory balancing of a multi- NUMA system, the current hotness and space occupation of NUMA are comprehensively considered, the hot NUMA node means that it is being accessed, and the space occupation of NUMA means that it can be accessed, and the balancing strategy considers both being accessed and possibly being accessed.

[0093] After selecting the hot point NUMA, a cold point (low hotness) virtual machine is selected instead of a hot point virtual machine, and when migrating the virtual machine cross- NUMA memory, the system NUMA bandwidth is compared, and the virtual machine is migrated cross- NUMA memory only when the system NUMA bandwidth upper limit is not reached. Thus, the influence of cross- NUMA memory migration on system performance can be reduced.

[0094] Corresponding to the embodiments of the foregoing method, the specification also provides embodiments of a memory balancing apparatus and a terminal to which the memory balancing apparatus is applied.

[0095] Embodiments of the memory balancing apparatus in the specification can be applied to a computer device, such as a server or a terminal device. The apparatus embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful apparatus, the apparatus is formed by reading corresponding computer program instructions in a non-volatile memory into a memory for running by a processor in which the apparatus is located. From the hardware level, as shown in Figure 4 , it is a hardware structure diagram of a computer device in which the memory balancing apparatus in the embodiments of the specification is located. In addition to the processor 410, the memory 430, the network interface 420, and the non-volatile memory 440 shown in Figure 4 , the server or the electronic device in which the apparatus 431 in the embodiments is located, according to the actual function of the computer device, can also include other hardware, which will not be described here.

[0096] As shown in Figure 5 , the memory balancing apparatus in the embodiments of the specification includes: Figure 5 is a block diagram of a memory balancing apparatus according to an exemplary embodiment of the specification. The apparatus includes:

[0097] The hot node determination module 510 is configured to determine a primary hot NUMA node and a secondary hot NUMA node from a plurality of NUMA nodes according to cross-node memory access amounts of the plurality of NUMA nodes.

[0098] The hot core determination module 520 is configured to determine a hot core in the primary hot NUMA node and the secondary hot NUMA node according to a cross- NUMA memory access bandwidth of each core in the primary hot NUMA node and the secondary hot NUMA node.

[0099] The target virtual machine determination module 530 is configured to determine a target virtual machine in a virtual machine set corresponding to the hot core according to a cross- NUMA memory space usage amount of each virtual machine in the virtual machine set.

[0100] The migration module 540 is configured to perform a migration cross- NUMA memory operation on the target virtual machine according to a NUMA real-time bandwidth and a system NUMA bandwidth.

[0101] Optionally, the hot node determination module includes:

[0102] The change determination sub-module is configured to determine a change value of a cross-node memory access bandwidth of each NUMA node in a preset period according to the cross-node memory access amounts of the plurality of NUMA nodes.

[0103] The node determining submodule is configured to determine a NUMA node with the maximum change value of the cross-node memory access bandwidth in the preset period as a main hot NUMA node, and determine a NUMA node with the second maximum change value of the cross-node memory access bandwidth in the preset period as a secondary hot NUMA node.

[0104] Optionally, the hotspot core determining module comprises:

[0105] The first mean value calculating submodule is configured to calculate the mean value of the cross- NUMA memory access bandwidth of each core in the main hot NUMA node.

[0106] The first core determining module is configured to determine a core with the maximum mean value of the cross- NUMA memory access bandwidth in the main hot NUMA node as a hotspot core.

[0107] The second mean value calculating submodule is configured to calculate the mean value of the cross- NUMA memory access bandwidth of each core in the secondary hot NUMA node.

[0108] The second core determining module is configured to determine a core with the maximum mean value of the cross- NUMA memory access bandwidth in the secondary hot NUMA node as a hotspot core.

[0109] Optionally, the target virtual machine determining module can comprise:

[0110] The process determining submodule is configured to determine, for each hotspot core, a virtual machine process corresponding to each virtual machine in a virtual machine set corresponding to the hotspot core.

[0111] The memory amount determining submodule is configured to determine the total amount of non-local memory allocation of each virtual machine process.

[0112] The candidate determining submodule is configured to determine a virtual machine corresponding to a virtual machine process with the minimum total amount of non-local memory allocation as a candidate virtual machine corresponding to the hotspot core.

[0113] The virtual machine determining submodule is configured to determine a target virtual machine in the candidate virtual machine according to a heat value parameter and a space usage parameter corresponding to each candidate virtual machine.

[0114] Optionally, the migration module can comprise:

[0115] The bandwidth mean value calculating submodule is configured to perform mean value calculation on the system NUMA bandwidth of the plurality of NUMA nodes to obtain a system NUMA bandwidth mean value.

[0116] The memory migration submodule is configured to perform a migration cross- NUMA memory operation on the target virtual machine in a case where the NUMA real-time bandwidth of the NUMA node corresponding to the target virtual machine is less than the product of the system NUMA bandwidth mean value and the weight.

[0117] Accordingly, the specification also provides an electronic device, comprising a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: determine a primary hot NUMA node and a secondary hot NUMA node from a plurality of non-uniform memory access, NUMA, nodes according to cross-node memory access amounts of the plurality of NUMA nodes; determine a hot core in the primary hot NUMA node and the secondary hot NUMA node according to cross- NUMA memory access bandwidths of each core in the primary hot NUMA node and the secondary hot NUMA node; determine a target virtual machine in a virtual machine set corresponding to the hot core according to cross- NUMA memory space usage amounts of each virtual machine in the virtual machine set; and perform a cross- NUMA memory operation migration on the target virtual machine according to a NUMA real-time bandwidth and a system NUMA bandwidth.

[0118] According to embodiments of the specification, the memory balancing of the NUMA system is performed by comprehensively considering the access hotness of the NUMA nodes and the spatial occupation of the NUMA nodes, which can reduce the influence of the cross- NUMA memory migration on the system performance.

[0119] The implementation processes of the functions and roles of the modules in the above apparatus are specifically described in the implementation processes of the corresponding steps in the above method, which will not be repeated here.

[0120] For the device embodiments, since they basically correspond to the method embodiments, the related parts can be referred to the part of the method embodiments. The above-described device embodiments are only illustrative, and the modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, i.e., they can be located in one place or distributed on multiple network modules. According to actual needs, some or all of the modules can be selected to achieve the purposes of the specification. Those skilled in the art can understand and implement without creative labor.

[0121] The above describes specific embodiments of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired result. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0122] Other embodiments of the present description will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the description be considered as exemplary only and the true scope and spirit of the present description being indicated by the following claims.

[0123] It is to be understood that the present description is not limited to the precise details of apparatus and methodology described herein and as illustrated in the drawings. Various modifications and changes in the precise operations and implementations can be made by those skilled in the art without departing from the scope of the present description. The scope of the present description is indicated only by the appended claims.

[0124] The above description is intended to be illustrative and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of the description should, therefore, be determined not with reference to the above description, but instead with reference to the appended claims, along with their full scope of equivalents.

Claims

1. A memory balancing method, characterized in that: The method comprises: Determine a primary hot NUMA node and a secondary hot NUMA node from the multiple NUMA nodes according to cross-node memory access amounts of the multiple non-uniform memory access NUMA nodes; Determine the hot cores in the primary hot NUMA node and the secondary hot NUMA node according to the cross-NUMA memory access bandwidth of each core in the primary hot NUMA node and the secondary hot NUMA node; Determining a target virtual machine in the set of virtual machines according to the cross-NUMA memory space usage of each virtual machine in the set of virtual machines corresponding to the hot core; A cross-NUMA memory migration operation is performed on the target virtual machine according to the NUMA real-time bandwidth and the system NUMA bandwidth.

2. The method according to claim 1, characterized in that The determining, from the plurality of NUMA nodes according to the cross-node memory access amounts of the plurality of NUMA nodes, a primary hot NUMA node and a secondary hot NUMA node comprises: Determine, based on the cross-node memory access volume of multiple NUMA nodes, a change value of the cross-node memory access bandwidth of each NUMA node within a preset period; The NUMA node with the largest change value of the inter-node memory access bandwidth within the preset period is determined as the primary hot NUMA node, and the NUMA node with the second largest change value of the inter-node memory access bandwidth within the preset period is determined as the secondary hot NUMA node.

3. The method according to claim 1, characterized in that The determining, according to the cross-NUMA memory access bandwidth of each core in the primary hot NUMA node and the secondary hot NUMA node, the hot cores in the primary hot NUMA node and the secondary hot NUMA node, comprises: Calculating the average cross-NUMA memory access bandwidth of each core in the primary hot NUMA node; Determine a core having the largest average cross-NUMA memory access bandwidth in the primary hot NUMA node as the hot core; Calculating the average cross-NUMA memory access bandwidth of each core in the second hottest NUMA node; A core having the largest average value of the cross-NUMA memory access bandwidth in the second hottest NUMA node is determined as the hotspot core.

4. The method according to claim 1, wherein The determining, based on the cross-NUMA memory space usage of each virtual machine in the virtual machine set corresponding to the hotspot core, a target virtual machine in the virtual machine set includes: For each hotspot core, Determine a virtual machine process corresponding to each virtual machine in the set of virtual machines corresponding to the hotspot core; Determine the total amount of non-local node memory allocated to each virtual machine process; Determine the virtual machine corresponding to the virtual machine process with the smallest total memory allocation outside the node as the candidate virtual machine corresponding to the hotspot core; A target virtual machine among the candidate virtual machines is determined according to the heat value parameter and the space usage parameter corresponding to each candidate virtual machine.

5. The method according to claim 1, wherein The migrating cross-NUMA memory operation on the target virtual machine according to the NUMA real-time bandwidth and the system NUMA bandwidth includes: Calculating an average of the system NUMA bandwidths of the multiple NUMA nodes to obtain a system NUMA bandwidth mean; When the NUMA real-time bandwidth of the NUMA node corresponding to the target virtual machine is less than the product of the system NUMA bandwidth average and the weight, a cross-NUMA memory migration operation is performed on the target virtual machine.

6. A memory balancing device, characterized in that: The device comprises: A hot node determination module is used to determine a primary hot NUMA node and a secondary hot NUMA node from the multiple NUMA nodes according to cross-node memory access amounts of the multiple non-uniform memory access NUMA nodes; A hot core determination module is used to determine the hot cores in the primary hot NUMA node and the secondary hot NUMA node according to the cross-NUMA memory access bandwidth of each core in the primary hot NUMA node and the secondary hot NUMA node; a target virtual machine determination module, configured to determine a target virtual machine in the virtual machine set according to the cross-NUMA memory space usage of each virtual machine in the virtual machine set corresponding to the hot core; The migration module is used to perform a cross-NUMA memory migration operation on the target virtual machine according to the NUMA real-time bandwidth and the system NUMA bandwidth.

7. The device according to claim 6, characterized in that The hotspot node determination module includes: A change determination submodule is used to determine a change value of the cross-node memory access bandwidth of each NUMA node within a preset period based on the cross-node memory access amount of multiple NUMA nodes; The node determination submodule is used to determine the NUMA node with the largest change value of the cross-node memory access bandwidth within the preset period as the primary hot NUMA node, and determine the NUMA node with the second largest change value of the cross-node memory access bandwidth within the preset period as the secondary hot NUMA node.

8. The device according to claim 6, characterized in that The hotspot core determination module includes: A first mean calculation submodule is configured to calculate the mean cross-NUMA memory access bandwidth of each core in the primary hot NUMA node; A first core determination module is configured to determine a core having the largest average cross-NUMA memory access bandwidth in the primary hot NUMA node as the hot core; A second mean calculation submodule is used to calculate the mean cross-NUMA memory access bandwidth of each core in the second hot NUMA node; The second core determination module is configured to determine a core having the largest average value of the cross-NUMA memory access bandwidth in the second hot NUMA node as the hot core.

9. An electronic device comprising: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 5.

Images