A memory allocation mechanism to reduce page migration for delay-sensitive applications

Abstract

The application provides a memory allocation mechanism for reducing page migration of delay-sensitive applications, which comprises: (1) an application type classifier for distinguishing delay-sensitive applications (high priority) and other applications (low priority); (2) a memory allocation mechanism mainly comprising a delay-sensitive application page allocation mechanism and an other application page allocation mechanism; and (3) a memory compression mechanism, which can work with the memory allocation mechanism of the method to avoid the migration of delay-sensitive application pages and improve the service quality of the delay-sensitive applications when the system memory pressure is high.

Description

Technical Field

[0001] This invention relates to an operating system (OS) memory page allocation mechanism, and more particularly to an operating system memory page allocation mechanism that reduces memory allocation operations for latency-sensitive application page migration and significantly improves their Quality of Service (QoS). Background Technology

[0002] The memory system plays a crucial role in the cloud environment and is key to improving the quality of cloud application services and the overall performance of the system. Current memory management mechanisms typically rely on various memory granularities for memory management, such as 4KB small pages and 2MB / 1GB large pages. Using various memory granularities can improve system performance, specifically in the following ways: (1) TLB (Translation Lookaside Buffer) misses can significantly hinder system performance. For example, Google's internal tests show that approximately 20% of CPU cycles are blocked by TLB misses. Using 2MB or 1GB large pages can significantly reduce the number of TLB misses, thereby improving the overall performance of the system. (2) Using smaller granular memory pages can reduce the load on page allocation. For example, when allocating 4KB small pages, the operating system usually randomly selects pages from the physical memory pool for allocation, so this allocation method has very low performance overhead. However, allocating 2MB or 1GB large pages relies on memory compression operations, which have higher system overhead. Memory compression is used to merge scattered large pages in the system when there is a lot of memory fragmentation, maintain contiguous physical memory, and allocate large pages.

[0003] It is evident that allocating memory pages with different granularities has many benefits for system performance. The following section first introduces the existing operating system page allocation mechanism, and then analyzes the problems existing in the current page allocation through experiments.

[0004] In a typical cloud server, the system will have latency-sensitive applications and other applications at the same time. However, we have observed that the existing memory allocation mechanism in modern operating systems (such as Linux and FreeBSD) cannot take advantage of the application characteristics of the process to which the page belongs (latency-sensitive applications or other applications). The existing system allocates memory indiscriminately to applications with different characteristics, treating page allocation requests from latency-sensitive applications and other applications the same.

[0005] Typically, when memory pressure is low, the operating system allocates large pages to applications, leveraging their ability to reduce TLB misses and improve system performance. However, large pages can lead to memory bloat. When system memory pressure is high and large pages are difficult to allocate, the overhead of memory compaction and other operations can block the operating system's memory allocation for applications while waiting for free large pages. Therefore, when the operating system is under high memory pressure, it allocates small pages to applications and triggers memory compaction to reserve contiguous physical memory for future large page allocation. Allocating small pages to applications is necessary, but it usually involves expensive memory compaction and migration operations. In Linux, memory compaction typically requires using two pointers to scan the physical memory region: one pointer scans memory from front to back, collecting allocated movable pages; the other pointer scans memory from back to front, collecting unallocated pages. When the two scan pointers meet, all movable pages are copied and migrated to unallocated pages. This reserves contiguous memory space for future large page allocations. However, this memory compaction method cannot determine whether the migrated pages belong to latency-sensitive applications or other applications. Therefore, pages of latency-sensitive applications are frequently migrated by the operating system kernel's memory compaction operations, leading to a degraded service quality for latency-sensitive applications.

[0006] To further investigate this phenomenon, we ran a mixed workload, including latency-sensitive applications and other applications, simultaneously on a typical cloud server with 128GB of DDR4 memory. The latency-sensitive application was Redis (90GB), and the other application was Fluidanimate (20GB). During the 1400 seconds of testing, approximately 896,000 4KB pages belonging to latency-sensitive applications were migrated through memory compression. Therefore, page migration occurred frequently in the address space of latency-sensitive applications, with over 3.4GB of pages belonging to these applications being migrated, representing 10.4% of all pages in latency-sensitive applications. We further demonstrated the fluctuations in the quality of service (QoS) of latency-sensitive applications caused by page migration. Figure 2This demonstration showcases the page migration and service quality changes during the runtime of the latency-sensitive application Redis, using Redis's 99th percentile latency as a service quality metric. As can be seen, in the first 400 seconds, because the system has ample contiguous memory space available for large page allocation, memory compaction does not occur, no pages are migrated, and Redis's service quality remains stable. After 400 seconds, when the operating system cannot allocate large pages, it triggers memory compaction. Since memory compaction involves complex memory page migration, a significant increase in Redis's 99th percentile latency is clearly visible when memory migration occurs. When using larger memory systems in cloud environments, more page migration operations may be involved, potentially leading to greater fluctuations and degradation in the service quality of latency-sensitive applications.

[0007] As described above, we find that existing operating system memory page allocation operations cannot distinguish whether the allocated pages belong to latency-sensitive applications (high priority) or other applications (low priority). This leads to frequent migration of many pages belonging to latency-sensitive applications during memory compaction operations. This migration operation causes a significant decline in the service quality of latency-sensitive applications, thereby affecting the overall system performance. Summary of the Invention

[0008] To address the problems of the aforementioned methods, this invention provides a memory allocation mechanism to reduce page migration for latency-sensitive applications. This mechanism minimizes the migration of pages belonging to latency-sensitive applications during memory compression operations, reduces frequent page migrations, and significantly improves service quality, thereby further enhancing overall system performance. During operation, this invention first distinguishes between different types of applications, namely, latency-sensitive applications and other applications. Then, it is found that the small pages allocated by the operating system have different properties. Small pages surrounding immovable pages (e.g., operating system kernel pages, GPU pages, and pages from other acceleration devices) should be allocated to latency-sensitive applications; while small pages in other locations are allocated to other applications. Finally, when the operating system triggers memory migration, this invention marks a 2MB region surrounding immovable pages and causes memory compression to skip these regions, thus preventing pages of latency-sensitive applications from being migrated by memory compression operations. Compared to methods in existing operating systems (such as Linux), this invention considers the specific characteristics of the application to which the page belongs during page allocation, enabling more detailed page differentiation and allocating pages that will not be migrated in the future to latency-sensitive applications. This avoids frequent blocking of latency-sensitive applications due to page migration operations, improving their service quality. Meanwhile, other applications are relatively insensitive to page migration operations, so the performance impact of this invention on these applications is limited. Overall, this invention can effectively improve system performance.

[0009] Specifically, this invention consists of three parts and operates in a pipeline manner. (1) This invention designs a classifier to distinguish application types, used to differentiate between latency-sensitive applications (high priority) and other applications (low priority). (2) This invention designs a dedicated memory pool for latency-sensitive applications, used to allocate small pages that will not be migrated in the future for latency-sensitive applications. For latency-sensitive applications, the operating system allocates memory from this memory pool, while for other applications, the existing allocation logic of the operating system is used. (3) This invention redesigns the operating system's memory compression mechanism. During memory compression, this invention marks the small pages of latency-sensitive applications, thereby causing memory compression to skip the pages of latency-sensitive applications and preventing the migration of latency-sensitive application pages.

[0010] The working process of each component is as follows:

[0011] (I) Application Type Classifier

[0012] 1. The application type classifier distinguishes the large number of applications running in the operating system and runs within the operating system kernel;

[0013] 2. The system primarily contains two types of applications: latency-sensitive applications and other applications. The application classifier designed in this invention is mainly based on user-defined static priorities, where latency-sensitive applications are high-priority applications, and other applications running in the system are low-priority applications. Users pre-specify the priority of each application using the Linux built-in command `nice / renice`, with values ​​ranging from -20 (highest priority) to 19 (lowest priority). The application classifier reads the user-specified priority value, and the threshold for high-priority applications can be specified by the user within the application classifier, providing ample flexibility.

[0014] 3. By using an application type classifier, the operating system can more accurately classify the numerous applications running in the system, distinguishing the pages of latency-sensitive applications from those of other applications. This allows the system to allocate pages that will not be migrated in the future to latency-sensitive applications during the subsequent page allocation process, and to skip the pages of latency-sensitive applications during memory compression.

[0015] (II) Page Allocation Mechanism

[0016] 1. The memory allocation mechanism of this invention is mainly divided into a latency-sensitive application page allocation mechanism and other application page allocation mechanisms.

[0017] 2. For page allocation requests from high-priority latency-sensitive applications, this invention does not randomly select small pages from the physical memory pool (e.g., the buddy system in Linux) for allocation. Instead, this invention first allocates small pages surrounding immovable pages. Since immovable pages cannot be migrated, memory blocks containing immovable pages cannot be used for large page allocation. Allocating pages for latency-sensitive applications around immovable pages avoids complex memory compression and migration operations. During memory compression, this invention skips movable pages surrounding these immovable pages. However, the dedicated memory pool designed for latency-sensitive applications does not prevent the operating system from promoting small pages of latency-sensitive applications to large pages, further improving the service quality of latency-sensitive applications. Additionally, this invention designs an immovable memory pool that stores other pages that will not be migrated during page compression. When all movable pages surrounding immovable pages have been allocated, this invention allocates pages for latency-sensitive applications from this immovable pool, further preventing pages from being migrated during memory compression.

[0018] 3. For page allocation requests from other low-priority applications, this invention uses the existing operating system's page allocation mechanism. For example, in Linux, suitable small pages are randomly selected from the buddy system for allocation. These pages may be migrated by memory compression when the operating system's memory pressure is high in the future. However, since these applications are not sensitive to page migration operations, the memory allocation mechanism of this invention can improve system performance overall.

[0019] (III) Memory Compression Mechanism

[0020] This invention redesigns the memory compression mechanism of existing operating systems so that it can work in conjunction with the memory allocation mechanism of this invention. During memory compression, this invention uses two pointers to scan a specified memory region. During the scan, the two pointers skip pages from latency-sensitive applications, thus preventing page migration for these applications. Therefore, the memory compression mechanism designed in this invention completely avoids page migration for latency-sensitive applications, improving their service quality. Attached Figure Description

[0021] Figure 1 This is a flowchart of a memory allocation mechanism for reducing latency-sensitive application page migration proposed in this invention.

[0022] Figure 2 This is a schematic diagram illustrating the adverse effects of the existing Linux page allocation mechanism on the service quality of latency-sensitive applications, as detected by this invention.

[0023] Figure 3 The latency-sensitive application page allocation strategy designed for this invention.

[0024] Figure 4 The memory compression mechanism designed for this invention. Specific implementation methods

[0025] The invention will be further described in detail with reference to the accompanying drawings.

[0026] like Figure 1 and Figure 2 This invention mainly consists of two mechanisms: a page allocation mechanism for different applications and a page compression mechanism when system memory pressure is high. These two mechanisms work together. The page allocation mechanism includes a page allocation mechanism for latency-sensitive applications and a page allocation mechanism for other applications. The two mechanisms address the page allocation process for latency-sensitive applications and other applications, respectively. When an application requests a page from the operating system, this invention first uses an application type classifier to distinguish between applications, classifying high-priority applications as latency-sensitive applications and low-priority applications as other applications. This invention originally proposes a page allocation strategy specifically for latency-sensitive applications. Figure 3 As shown, this invention establishes a latency-sensitive application memory pool to allocate small pages that will not be migrated in the future for latency-sensitive applications when the operating system cannot allocate large pages. This memory pool first uses small pages surrounding immovable pages for allocation. Figure 3 A 2MB memory block containing non-movable pages cannot be used for large page allocation. Therefore, compaction operations on this memory block cannot reorganize it into contiguous physical memory, and thus memory compaction of this portion should be avoided. This invention utilizes this characteristic to allocate small pages for latency-sensitive applications around the non-movable pages, thereby preventing the migration of pages for latency-sensitive applications. The allocatable pages around the non-movable pages are limited. Once all these pages are allocated, this invention uses a non-movable memory pool to interact with the operating system's memory pool, further allocating pages for latency-sensitive applications from the non-movable memory pool. This non-movable memory pool can move pages back to the operating system's memory pool when there are enough pages, and move pages from the operating system's memory pool into the non-movable memory pool when there are insufficient pages.

[0027] During page compression, the memory compression mechanism designed in this invention can skip pages of latency-sensitive applications (pages surrounding non-movable pages and pages allocated from non-movable memory pools), such as... Figure 4As shown, the memory compression mechanism of this invention uses two pointers to scan a specified memory region. During the scan, the pointers skip a 2MB area surrounding non-movable pages. When the two pointers meet, the movable pages on one side are moved to the free pages on the other side, thereby reserving contiguous physical memory for the system to allocate large pages. In this way, the memory compression mechanism designed in this invention avoids frequent page migrations in latency-sensitive applications.

[0028] A memory allocation mechanism to reduce page migration for latency-sensitive applications includes the following steps: Step 1, establishing an application type classifier to distinguish latency-sensitive applications from other applications, wherein latency-sensitive applications are high-priority and other applications are low-priority; Step 2, establishing a memory allocation mechanism, including a page allocation mechanism for latency-sensitive applications and page allocation mechanisms for other applications; Step 3, establishing a memory compression mechanism, enabling the memory compression mechanism to work in conjunction with the memory allocation mechanism to prevent the migration of pages for latency-sensitive applications when system memory pressure is high, thereby improving system service quality.

[0029] Step 1.1: The application type classifier distinguishes a large number of applications running in the operating system and runs within the operating system kernel. Step 1.2: The application type classifier classifies applications based on user-defined static priorities. Users pre-specify the priority of each application using the Linux built-in commands nice / renice, with values ​​ranging from -20 (highest priority) to 19 (lowest priority). The application type classifier reads the user-specified priority value, and the threshold for high-priority applications can be specified by the user within the classifier, providing ample flexibility. Step 1.3: Using the application type classifier, the operating system can more accurately classify the numerous applications running in the system, separating latency-sensitive applications from the pages of other applications. This allows for the allocation of pages that will not be migrated in the future to latency-sensitive applications during subsequent page allocation, avoiding the impact of frequent page migration operations on their service quality.

[0030] Step 2 includes the following steps: Step 2.1, This memory allocation mechanism resides in the operating system kernel and is triggered when a page fault occurs in the operating system. When the operating system experiences high memory pressure and cannot allocate large pages, small pages are allocated to applications; Step 2.2, Based on the application type classifier, pages in the system can also be classified into two types: pages for latency-sensitive applications and pages for other applications. Both types of applications request memory from the operating system. Before the operating system allocates pages, it obtains the results from the application type classifier; Step 2.3, For page allocation requests from high-priority latency-sensitive applications, small pages located around non-movable pages are allocated first. Since non-movable pages cannot be migrated, memory blocks containing non-movable pages cannot be used for large page allocation. Allocating pages for latency-sensitive applications around non-movable pages avoids complex memory compression and memory migration operations. The number of allocatable pages is limited. Once all these pages are allocated, a non-movable memory pool interacts with the operating system's memory pool to further allocate pages for latency-sensitive applications from the non-movable memory pool. This non-movable memory pool can move pages back to the operating system's memory pool when there are enough pages; and when there are not enough pages, pages are moved from the operating system's memory pool into the non-movable memory pool. Small pages allocated from the periphery of the non-movable pages and small pages allocated from the non-movable memory pool are recorded so that these areas can be skipped during subsequent memory compression. In step 2.4, for page allocation requests from other low-priority applications, the existing operating system's page allocation mechanism is used, including in Linux, where suitable small pages are randomly selected from the buddy system for allocation. These pages may be migrated by memory compression in the future when the operating system's memory pressure is high, but since these applications are not sensitive to page migration operations, the memory allocation mechanism can improve system performance overall.

[0031] Step 3 includes the following steps: Step 3.1, This memory compression mechanism is also located in the operating system kernel. When the system memory pressure is high and large pages cannot be allocated, the operating system allocates small pages for applications and triggers the memory compression mechanism. This memory compression mechanism works in conjunction with the memory allocation mechanism to prevent pages belonging to latency-sensitive applications from being migrated. Step 3.2, During memory compression, two pointers are used to scan a specified memory region. One pointer scans memory from front to back and collects the allocated movable pages; the other pointer scans memory from back to front and collects the unallocated pages. During the scanning process, these two scanning pointers skip pages from latency-sensitive applications. When these two scanning pointers meet, all movable pages are copied and migrated to unallocated pages. This reserves contiguous memory space in physical memory for future large page allocation. Skipping pages from latency-sensitive applications during memory compression prevents the migration of pages from latency-sensitive applications.

[0032] This invention provides a memory allocation mechanism to reduce the migration of latency-sensitive application pages, comprising: (1) an application type classifier to distinguish latency-sensitive applications (high priority) from other applications (low priority); (2) a memory allocation mechanism, mainly including a latency-sensitive application page allocation mechanism and other application page allocation mechanisms. Latency-sensitive application pages are allocated around immovable pages, thereby avoiding frequent migration operations of latency-sensitive application pages in the future; (3) a memory compression mechanism, which can work in conjunction with the memory allocation mechanism of this method to prevent the migration of latency-sensitive application pages when system memory pressure is high, thus improving their service quality.

Claims

1. A memory allocation mechanism to reduce page migration in latency-sensitive applications, characterized in that, Includes the following steps: Step 1: Build an application type classifier to distinguish between latency-sensitive applications and other applications. Latency-sensitive applications are given high priority, while other applications are given low priority. Step 2: Establish a memory allocation mechanism, including a page allocation mechanism for latency-sensitive applications and page allocation mechanisms for other applications; Step 3: Establish a memory compression mechanism to work in conjunction with the memory allocation mechanism. When the system memory pressure is high, avoid migrating pages of latency-sensitive applications, thereby improving the quality of system services. Step 2 includes the following steps: Step 2.1: This memory allocation mechanism is located in the operating system kernel and is triggered when a page fault occurs in the operating system. When the operating system is under high memory pressure and cannot allocate large pages, it allocates small pages for the application. Step 2.2: Based on the results of the application type classifier, the pages in the system can also be classified into two types: pages of latency-sensitive applications and pages of other applications. Both types of applications will request memory from the operating system. Before the operating system allocates pages, it will obtain the results of the application type allocator. Step 2.3: For page allocation requests from high-priority latency-sensitive applications, small pages located around non-movable pages are allocated first. Since non-movable pages cannot be migrated, memory blocks containing non-movable pages cannot be used for large page allocation. Allocating pages for latency-sensitive applications around non-movable pages avoids complex memory compression and memory migration operations. The allocable pages around non-movable pages are limited. After all these pages are allocated, a non-movable memory pool interacts with the operating system memory pool to further allocate pages for latency-sensitive applications from the non-movable memory pool. This non-movable memory pool can move pages back to the operating system memory pool when there are enough pages; and when there are not enough pages, pages are moved from the operating system memory pool to the non-movable memory pool. Small pages allocated from the periphery of non-movable pages and small pages allocated from the non-movable memory pool are recorded so that these areas can be skipped during subsequent memory compression. Step 2.4: For page allocation requests from other low-priority applications, use the existing operating system's page allocation mechanism, including in Linux, directly selecting suitable small pages randomly on the buddy system for allocation. These pages may be migrated by memory compression in the future when the operating system's memory pressure is high. However, since these applications are not sensitive to page migration operations, the memory allocation mechanism can improve system performance overall. Step 3 includes the following steps: Step 3.1, this memory compression mechanism is also located in the operating system kernel. When the system memory pressure is high and large pages cannot be allocated, the operating system allocates small pages for the application and triggers the memory compression mechanism at the same time. This memory compression mechanism works in conjunction with the memory allocation mechanism to prevent pages belonging to latency-sensitive applications from being migrated. Step 3.2: During memory compression, two pointers are used to scan a specified memory region. One pointer scans memory from front to back and collects the allocated movable pages; the other pointer scans memory from back to front and collects the unallocated pages. During the scan, these two scanning pointers skip pages from latency-sensitive applications. When the two scanning pointers meet, all movable pages are copied and migrated to the unallocated pages. This reserves contiguous memory space in physical memory for future large page allocation. Skipping pages from latency-sensitive applications during memory compression avoids the migration of pages from latency-sensitive applications.

2. The memory allocation mechanism for reducing latency-sensitive application page migration according to claim 1, characterized in that, Step 1 includes the following steps: Step 1.1: The application type classifier distinguishes the large number of applications running in the operating system and runs in the operating system kernel; Step 1.2: The application type classifier classifies applications based on user-defined static priorities. Users pre-specify the priority of each application using the Linux built-in command nice / renice. The priority value ranges from -20 (highest priority) to 19 (lowest priority). The application type classifier reads the user-specified priority value. The threshold for high-priority applications can be specified by the user in the application type classifier, providing ample flexibility. Step 1.3: By using the application type classifier, the operating system can more accurately classify the numerous applications running in the system, distinguishing the pages of latency-sensitive applications from those of other applications. This allows the system to allocate pages that will not be migrated in the future to latency-sensitive applications during the subsequent page allocation process, thus avoiding the impact of frequent page migration operations on their service quality.

Images