NUMA system and page migration method in the system

By migrating data between NUMA nodes and adjusting page location according to the frequency and size of memory requests, the problem of remote access delay in NUMA systems is solved, and system performance improvement and cost reduction are achieved.

CN112947851BActive Publication Date: 2024-11-05ALIBABA GROUP HOLDING LTD
View PDF 12 Cites 0 Cited by

Patent Information

Application Number
CN202011301658.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-04-30
Filing Date
2020-11-19
Publication Date
2024-11-05
Estimated Expiration
2040-11-19

AI Technical Summary

Technical Problem

The prior art cannot effectively reduce the delays related to remote access time in non-unified memory access (NUMA) systems, especially in multi-processing systems. Existing methods such as analysis-based optimization cannot adapt to different access modes and require recompilation of code. Or use costly kernel patches, user space tools have limited performance improvements to large data objects.

Method used

By migrating data between NUMA nodes, using the analyzer to sample memory requests, the size and access frequency of the requested data object are determined. If the frequency exceeds the threshold, the page is migrated to the NUMA node that initiated the request to reduce remote access latency.

Benefits of technology

Significantly reduces remote access latency between NUMA nodes, improves system performance, is suitable for large data objects, and provides pure user-space runtime analysis without manual participation, reducing migration costs and runtime overhead.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112947851B_ABST
    Figure CN112947851B_ABST
Patent Text Reader

Abstract

Remote access latency in a non-uniform memory access (NUMA) system can be greatly reduced by monitoring which NUMA nodes are accessing which local memories, and migrating memory pages from the local memory in a first NUMA node to the local memory in a hot NUMA node when the hot NUMA node frequently accesses the local memory in the first NUMA node.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 939,961, filed on November 25, 2019, and U.S. Patent Application No. 16 / 863,954, filed on April 30, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to a non-uniform memory access (NUMA) system, and more particularly, to a NUMA system and a method of migrating pages in the system. Background Art

[0004] A non-uniform memory access (NUMA) system is a multi-processing system with multiple NUMA nodes, wherein each NUMA node has a memory partition and multiple processors connected to the memory partition. In addition, multiple NUMA nodes are connected together so that each processor in each NUMA node treats all memory partitions together as one large memory.

[0005] As the name implies, NUMA systems have non-uniform access times, where local access times to a memory partition of one NUMA node are much shorter than remote access times to a memory partition of another NUMA node. For example, remote access times to a memory partition of another NUMA node may have 30-40% latency than access times to a local memory partition.

[0006] To improve system performance, the latency associated with remote access times needs to be reduced. Existing approaches to date have limitations. For example, profiling-based optimizations use aggregate views that cannot adapt to different access patterns. Additionally, code needs to be recompiled to use previous profiling information.

[0007] As another example, existing dynamic optimizations are typically implemented in the kernel, which requires costly kernel patches whenever any changes are needed. As another example, the few user-space tools that exist use page-level information to reduce remote memory access time, but the performance is poor for large data objects. Therefore, there is a need to reduce the latency associated with remote access time to overcome these limitations. Summary of the invention

[0008] The present invention reduces the delay associated with remote access time by migrating data between NUMA nodes based on the NUMA node with the largest access volume. The present invention includes a method for operating a NUMA system. The method includes: determining a requested data object based on a requested memory address in a sampled memory request, the sampled memory request comes from a NUMA node that initiates the request, and the requested data object represents a memory address range; the method also includes determining whether the size of the requested data object is a page size or less than a page size, or greater than a page size; and when the size of the requested data object is a page size or less than a page size, the method increases a count value, the count value is used to measure the number of times the NUMA node that initiates the request attempts to access the requested data object. The method also determines whether the count value exceeds a threshold within a predetermined time period, and when the count value exceeds the threshold, migrates the page containing the requested data object to the NUMA node that initiates the request.

[0009] The present invention also includes a NUMA system, the NUMA system also includes a memory divided into a plurality of local partitions; a plurality of NUMA nodes connected to the local partitions, each NUMA node having a corresponding local partition of the memory and a plurality of processors connected to the memory; the NUMA system also includes a bus connecting the NUMA nodes together; and an analyzer connected to the bus, the analyzer being used to: determine a requested data object based on a requested memory address in a sampled memory request, the sampled memory request coming from a NUMA node that initiated the request, the requested data object representing a memory address range; the analyzer also determines whether the size of the requested data object is a page size or less than a page size, or greater than a page size; and when the size of the requested data object is a page size or less than a page size, the analyzer increases a count value, the count value being used to measure the number of times the NUMA node that initiated the request attempted to access the requested data object. The analyzer also determines whether the count value exceeds a threshold within a predetermined time period, and when the count value exceeds the threshold, migrates a page containing the requested data object to the NUMA node that initiated the request.

[0010] The present invention further includes a non-transitory computer-readable storage medium in which program instructions are embedded. When the program instructions are executed by one or more processors of a device, the device performs a process of operating a NUMA system, the process comprising: determining a requested data object from a requested memory address in a sampled memory request, the sampled memory request comes from a requesting NUMA node, and the requested data object represents a memory address range; determining a requested data object based on a requested memory address in the sampled memory request, the sampled memory request comes from a requesting NUMA node, and the requested data object represents a memory address range; determining whether the size of the requested data object is a page size or less than a page size, or greater than a page size; and, when the size of the requested data object is a page size or less than a page size, increasing a count value, the count value being used to measure the number of times the requesting NUMA node attempts to access the requested data object, determining whether the count value exceeds a threshold within a predetermined time period, and when the count value exceeds the threshold, migrating a page containing the requested data object to the requesting NUMA node.

[0011] The features and advantages of the present invention may be better understood by referring to the following detailed description and accompanying drawings, which illustrate exemplary embodiments utilizing the principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute limitations on the present application.

[0013] Figure 1 is a block diagram illustrating an example of a non-uniform memory access (NUMA) system 100 according to the present invention.

[0014] Figure 2 is a flow chart illustrating an example of a method 200 of migrating pages in a NUMA system according to the present invention.

[0015] Figure 3 is a flowchart showing an example of a method 300 of analyzing a program according to the present invention. DETAILED DESCRIPTION

[0016] Figure 1 1 is a block diagram showing an example of a non-uniform memory access (NUMA) system 100 according to the present invention. Figure 1 As shown. Figure 1As shown, the NUMA system 100 includes a memory 110 that has been divided into a plurality of local partitions LP1-LPm, a plurality of NUMA nodes NN1-NNm connected to the local partitions LP1-LPm, and a bus 112 connecting the NUMA nodes NN1-NNm together. Each NUMA node NN has a corresponding local partition LP of the memory 110, a plurality of processors 114, each processor having its own local cache 116 connected to the memory 110, and an input / output circuit 118 connected to the processor 114.

[0017] like Figure 1 As further shown, the NUMA system 100 includes an analyzer 120 connected to the bus 112. In operation, the analyzer 120 can be implemented with a CPU, the analyzer 120 samples NUMA node traffic on the bus 112, records the sampled bus traffic, and migrates a page or multiple data objects stored in the first local partition to the second local partition when the sampled bus traffic exceeds a threshold amount. The sampled bus traffic indicates the number of times the second local partition is accessing the data object.

[0018] Figure 2 An example of a method 200 for migrating pages in a non-uniform memory access (NUMA) system according to the present invention is shown. In one embodiment of the present invention, the method 200 can be implemented with the NUMA system 100. The method 200 records a static mapping of the topology of the CPU and the NUMA domain knowledge of the system.

[0019] like Figure 2 As shown, method 200 begins at 210 by determining a plurality of data objects from the code of a program to be executed on a NUMA system. Each data object represents an associated memory address range. For example, the range may be associated with data stored in the memory address range. Heap data: overload memory allocation and free functions may be used to identify data objects, and static data: track the loading and unloading of each module, and read its symbol table. Data objects may be small (having an address range that occupies one page or less) or large (having an address range that is greater than one page).

[0020] Method 200 then moves to 212 to store the data object in a local partition of memory that is associated with a NUMA node of the NUMA system. For example, by examining the code of a program to be executed on NUMA system 100, the data object may be stored in a local partition of the NUMA node that has the first processor to access the data object. For example, referring to Figure 1If the processor 114 in the NUMA node NN1 is the first processor to access the data object (via the requested memory address), the method 200 stores the data object in the local partition LP1 of the NUMA node NN1.

[0021] Next, during execution of a program on a NUMA system such as NUMA system 100, method 200 moves to 214 to sample memory access requests from processors in NUMA nodes of the NUMA system using performance monitoring to generate sampled memory requests. The sampled memory request includes a requested memory address, which can be identified by a block number, a page number in the block, and a row number in the page. The sampled memory request also includes, for example, the NUMA node that initiated the request (the identification of the NUMA node that outputs the sampled memory access request) and the storage NUMA node (the identification of the local partition that stores the requested memory address). In one embodiment, a record of each memory access request issued by each processor in each NUMA node can be generated. These records can then be sampled to obtain sampled memory requests as the records being created.

[0022] Thereafter, method 200 moves to 216 to determine a requested data object (a range of associated memory addresses) based on the requested memory address in the sampled memory request. In other words, method 200 determines the requested data object associated with the memory address in the memory access request.

[0023] For example, if the requested memory address in the sampled memory request falls within the range of memory addresses associated with a data object, then the data object is determined to be the requested data object. In one embodiment, the page number of the requested memory address may be used to identify the requested data object.

[0024] Method 200 then moves to 220 to record memory access information from the sampled memory request, such as the identifier of the NUMA node that initiated the request, the requested data object, the page number, and the identifier of the storage NUMA node. The memory access information also includes timing and congestion data. Other relevant information may also be recorded.

[0025] Thereafter, the method 200 moves to 222 to determine whether the size of the requested data object is the page size, smaller than the page size, or larger than the page size. When the size of the requested data object is the page size or smaller than the page size, the method 200 moves to 224 to increase a count value, which is used to measure the number of times the NUMA node that initiated the request attempted to access the requested data object, that is, a memory access request for a memory address within the range of the requested data object has been generated.

[0026] Next, method 200 moves to 226 to determine whether the count value exceeds a threshold value within a predetermined time period. When the count value is below the threshold value, method 200 returns to 214 to obtain another sample. When the count value exceeds the threshold value, method 200 moves to 230 to migrate the page containing the requested data object to the NUMA node that initiated the request. Alternatively, multiple pages before and after the page containing the requested data object can be migrated simultaneously (adjustable parameter).

[0027] For example, if the threshold of data objects stored in the local partition LP3 of the third NUMA node NN3 is 1,000, the processor in the first NUMA node NN1 has accessed the data object in the local partition LP3 999 times, and if the processor in the second NUMA node NN2 has accessed the data object in the local partition LP3 312 times, the method 200 will migrate the page containing the data object (including the previous and subsequent pages) from the local partition LP3 to the local partition LP1 under the following circumstances: the first NUMA node NN1 accesses the data object in the local partition LP3 for the 1000th time within a predetermined time period.

[0028] Therefore, one of the advantages of the present invention is that no matter where the small data objects are stored in the local partitions of the memory, the present invention continuously migrates the data objects to the active local partitions, ie, the local partitions of the NUMA nodes that currently access the data objects the most.

[0029] For example, if a data object is stored in local partition LP1 because the processor in NUMA node NN1 is the first processor to access a memory address within the data object, but at a subsequent moment in program execution, NUMA node NN2 accesses the data object in large quantities, the present invention migrates the data object from local partition LP1 to local partition LP2, thereby significantly reducing the time required for the processor in NUMA node NN2 to access the data object.

[0030] Reference again Figure 2 , when the size of the requested data object in 222 is greater than the page size, in other words, when the requested data object is a multi-page requested data object, method 200 moves to 240 to determine how page accesses are distributed, and to record how different NUMA nodes access the multi-page requested data object. In other words, method 200 determines which NUMA node that initiated the request accessed the multi-page requested data object, the pages accessed, and the number of times the NUMA node that initiated the request attempted to access the requested data object within a predetermined time period. The distribution of page accesses can be extracted based on a small sample.

[0031] For example, see Figure 1If a multi-page data object is stored in local partition LP1 of NUMA node NN1, method 200 may determine, for example, that NUMA node NN2 accessed page three of the multi-page data object 1 thousand times, while NUMA node NN3 accessed page four of the multi-page data object 312 times.

[0032] Next, the method 200 moves to 242 to determine whether there is a problem with the multi-page requested data object. The problematic data object includes a location domain, multiple access domains, and remote access can trigger congestion. If there is no problem, the method 200 returns to 214 to obtain another sample.

[0033] On the other hand, if it is determined that there is a problem with the multi-page requested data object, such as a data object with a number of pages or more that has exceeded a rebalance threshold, the method 200 moves to 244 to migrate one or more pages of the selected multi-page requested data object to balance / rebalance the multi-page requested data object. For multi-threaded applications, each thread tends to operate on blocks of the entire memory range of the data object.

[0034] For example, method 200 may determine that 1,000 accesses to page three by NUMA node NN2 exceed the rebalance threshold, and in response, migrate page three from local partition LP1 of NUMA node NN1 to local partition LP2 of NUMA node NN2. On the other hand, because the total number of accesses of 312 is less than the rebalance threshold, nothing is migrated to local partition LP3. Therefore, if any page of the multi-page requested data object exceeds the rebalance threshold, method 200 moves to 244 to migrate the page to the requesting NUMA node with the highest access rate.

[0035] Therefore, another advantage of the present invention is that when other NUMA nodes access data objects in large quantities, selected pages of multi-page data objects can be migrated to other NUMA nodes to balance / rebalance data objects and significantly reduce the time required for other NUMA nodes to access information.

[0036] In some cases, data pages in one local partition of memory may be duplicated or copied to another local partition of memory. Duplication can be detected in a number of ways. For example, the tool can be decompiled in the following way: first, the assembly code is obtained from the binary file through a decompilation tool (similar to objdump). Next, the functions of the program are extracted from the assembly code. Then, the allocation and free functions are checked to determine if they are exposing data objects.

[0037] As another example, page migration activity can be monitored via a microbenchmark to detect duplication. The microbenchmark can be run via a tool. Next, system calls are monitored to migrate pages across data objects. If this is not the case, then the migration occurs within the data object and can be seen as semantically aware.

[0038] Figure 3 1 is a flow chart showing an example of a method 300 for analyzing a program according to the present invention. Figure 3 As shown, a program (program.exe) 310 is executed, and an analyzer program (profiler.so) 312 is executed during the program running on a CPU or a processor with similar functions to implement the present invention for the program (program.exe) 310 and generate an optimized executable program (optimized program.exe) 314.

[0039] Therefore, the present invention monitors which NUMA node is accessing which local partition of the memory, and when the hot NUMA node frequently accesses the local partition of the remote NUMA node, and balances / rebalances the memory pages, the remote access latency is substantially reduced by migrating the storage pages from the local partition of the remote NUMA node to the local partition of the hot NUMA node.

[0040] One of the advantages of the present invention is that it provides pure user space runtime analysis without any human involvement. The present invention also handles large and small data well. In addition, group migration of pages reduces migration costs.

[0041] Comparing dynamic analysis to static analysis, simulation based on static analysis results in high runtime overhead. While static analysis based measurements can provide insights with lower overhead, they still need to be performed manually. Kernel based dynamic analysis requires custom patches, which are expensive for commercial use. In addition, existing user space dynamic analysis does not handle large objects well.

[0042] Comparing semantic to non-semantic page-level migration without semantics treats the program as a black box, and it may happen that some pages are moved back and forth, incurring additional overhead. However, semantic-aware analysis can migrate pages in less time. Semantic-aware analysis puts pages together with data objects and computations.

[0043] The above embodiments are only used for illustration and are not used to limit the technical solutions of the present invention. Although the present invention is described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solutions described in the above embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents. These modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions in the embodiments of the present invention.

[0044] It should be understood that the above description is exemplary of the present invention, and various alternatives of the present invention described herein may be used to implement the present invention. Therefore, it is intended that the appended claims define the scope of the present invention, and that structures and methods within the scope of these claims and their equivalents are covered thereby.

Claims

1. A method for operating a non-uniform memory access (NUMA) system, the method comprising: determining a requested data object based on a requested memory address in a sampled memory request, the sampled memory request being from a NUMA node initiating the request, the requested data object representing a memory address range; Determining whether the size of the requested data object is the page size, smaller than the page size, or larger than the page size; and When the size of the requested data object is the page size or smaller than the page size, a count value is increased, the count value is used to measure the number of times the NUMA node that initiated the request attempts to access the requested data object, determine whether the count value exceeds a threshold value within a predetermined time period, and when the count value exceeds the threshold value, migrate the page containing the requested data object to the NUMA node that initiated the request; When the size of the requested data object is larger than the page size, determining a distribution mode of page access, and recording how different NUMA nodes access multiple pages of the requested data object; The method further comprises: when the size of the requested data object is larger than the page size: determining whether there is a problem with the multi-page requested data object, at least when one or more pages of the requested data object have exceeded a rebalancing threshold, the requested data object is determined to be problematic; the problematic data object includes a location domain, multiple access domains, and remote access triggers congestion; If it is determined that the multi-page requested data object has a problem, one or more pages of the selected multi-page requested data object are migrated to balance / rebalance the multi-page requested data object.

2. The method according to claim 1, wherein: The requested data object is determined based on a page number of the requested memory address.

3. The method of claim 1, further comprising sampling memory requests from the requesting NUMA node to generate the sampled memory requests.

4. The method according to claim 1, characterized in that: The method also includes recording memory access information from the sampled memory request, wherein the memory access information includes the NUMA node identifier of the initiating request, the requested data object, a page number, and a storage NUMA node identifier.

5. The method according to claim 1, further comprising: determining a number of data objects based on code of a program to be executed on the NUMA system; and The data object is stored in a local partition of memory.

6. A NUMA system, comprising: a memory, the memory being divided into a plurality of local partitions; a plurality of NUMA nodes connected to the local partition, each NUMA node having a corresponding local partition of the memory, and a plurality of processors connected to the memory; A bus connecting the NUMA nodes together; and an analyzer connected to the bus, the analyzer being configured to: determining a requested data object based on a requested memory address in a sampled memory request, the sampled memory request being from a NUMA node initiating the request, the requested data object representing a memory address range; Determining whether the size of the requested data object is the page size, smaller than the page size, or larger than the page size; and When the size of the requested data object is the page size or smaller than the page size, a count value is increased, the count value is used to measure the number of times the NUMA node that initiated the request attempts to access the requested data object, determine whether the count value exceeds a threshold value within a predetermined time period, and when the count value exceeds the threshold value, migrate the page containing the requested data object to the NUMA node that initiated the request; When the size of the requested data object is larger than the page size, determine the distribution mode of page access, and record how different NUMA nodes access the multi-page requested data object; when the size of the requested data object is larger than the page size: determine whether there is a problem with the multi-page requested data object, at least when one or more pages of the requested data object have exceeded the rebalancing threshold, the requested data object is determined to be problematic; the problematic data object includes a location domain, multiple access domains, and remote access will trigger congestion; If it is determined that the multi-page requested data object has a problem, one or more pages of the selected multi-page requested data object are migrated to balance / rebalance the multi-page requested data object.

7. The NUMA system according to claim 6, wherein: The requested data object is determined based on a page number of the requested memory address.

8. The NUMA system according to claim 6, wherein: The analyzer further samples memory requests from the request-initiating NUMA node to generate the sampled memory requests.

9. The NUMA system according to claim 6, wherein: The analyzer further records memory access information from the sampled memory request, wherein the memory access information includes the NUMA node identifier of the initiating request, the requested data object, a page number, and a storage NUMA node identifier.

10. The NUMA system according to claim 6, wherein: The analyzer is further configured to: determine the number of data objects based on the code of a program to be executed on the NUMA system; and The data object is stored in a local partition of memory.

11. A non-transitory computer-readable storage medium having program instructions embedded therein, which, when executed by one or more processors of a device, cause the device to perform a process for operating a NUMA system, the process comprising: determining a requested data object based on a requested memory address in a sampled memory request, the sampled memory request being from a NUMA node initiating the request, the requested data object representing a memory address range; Determining whether the size of the requested data object is the page size, smaller than the page size, or larger than the page size; and When the size of the requested data object is the page size or smaller than the page size, a count value is increased, the count value is used to measure the number of times the NUMA node that initiated the request attempts to access the requested data object, determine whether the count value exceeds a threshold value within a predetermined time period, and when the count value exceeds the threshold value, migrate the page containing the requested data object to the NUMA node that initiated the request; When the size of the requested data object is larger than the page size, determine the distribution mode of page access, and record how different NUMA nodes access the multi-page requested data object; when the size of the requested data object is larger than the page size: determine whether there is a problem with the multi-page requested data object, at least when one or more pages of the requested data object have exceeded the rebalancing threshold, the requested data object is determined to be problematic; the problematic data object includes a location domain, multiple access domains, and remote access will trigger congestion; If it is determined that the multi-page requested data object has a problem, one or more pages of the selected multi-page requested data object are migrated to balance / rebalance the multi-page requested data object.

12. The medium according to claim 11, wherein The requested data object is determined based on a page number of the requested memory address.

13. The medium according to claim 11, characterized in that The method also includes sampling memory requests from the request-initiating NUMA node to generate the sampled memory requests.

14. The medium according to claim 11, characterized in that The method also includes recording memory access information from the sampled memory request, wherein the memory access information includes the NUMA node identifier of the initiating request, the requested data object, a page number, and a storage NUMA node identifier.

15. The medium of claim 11, further comprising: determining a number of data objects based on code of a program to be executed on the NUMA system; and The data object is stored in a local partition of memory.

Citation Information

Patent Citations

  • NUMA system and method of migrating pages in the system

    US20210157647A1

  • Memory access monitoring method and device

    CN102362464A

  • Method of assigning virtual memory to physical memory, storage controller and computer system

    CN1617113A

  • Dynamic memory placement policies for NUMA architecture

    US20020129115A1

  • I / O conversion method and apparatus for storage system

    US20110231631A1