A kernel huge page memory optimization method

Abstract

The application discloses a kernel large page memory optimization method, comprising the following steps: first, taking over idle memory through a buddy memory management subsystem, establishing struct page metadata to manage 4KB memory pages, and setting start parameters and post-start parameters to reserve memory; second, using the established memory pool and two-layer memory management architecture to manage large page application and release; and third, using the established physical memory and virtual address connection to realize memory usage through the memory pool. The application reduces the number of system page metadata management, optimizes kernel self-provided large page organization management, realizes 16MB memory space saving on each 1GB large page memory of the large page management system, and when the server is configured with more large pages, more memory can be released, and the released memory can be directly used for cloud hosts or allocated to the server system for use, thereby improving memory utilization.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of operating system, in particular to a kernel large page memory optimization method. BACKGROUND

[0002] In the current memory management of Linux Kernel, the system uses 4KB physical pages to organize and manage memory, and needs to establish an associated struct page data structure for each physical page to track and record page information. Currently, the size of the struct page data structure has reached 64 bytes on a 64-bit machine. With the increasing physical memory of servers, the memory occupied by the struct page metadata itself is also increasing. Since each 4KB page requires 64 bytes of struct page metadata, 1 / 64 of the total memory will be consumed in struct page data structures. In cloud computing services, in order to provide better performance for cloud hosts, large pages are usually used. By enabling "large page memory", the system only needs to handle fewer page mapping tables, thereby reducing the overhead of accessing / maintaining them. However, with the increasing memory of servers, more and more large page memory is set. Since the memory required for server system operation is small and relatively fixed, it cannot meet the actual setting requirements, resulting in an increase in operating costs. SUMMARY

[0003] The present application aims to provide a kernel large page memory optimization method to solve the problems raised in the background.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical solution: a kernel large page memory optimization method, the optimization method comprising the following steps:

[0005] First, reserve the memory. In the Linux kernel system, the buddy memory management subsystem takes over the idle memory and establishes struct page metadata to manage 4KB memory pages. At the same time, the parameters are set before and after the startup of the Linux kernel system to reserve the memory.

[0006] Second, manage the memory. Apply and release the large page through the memory pool, and establish a two-layer memory management architecture to manage and control the memory.

[0007] Third, use the memory. Establish the connection between physical memory and virtual address, and use the reserved large page memory through the large page memory pool.

[0008] Preferably, in the first step, the memory reservation process comprises the following steps:

[0009] S1: First, traverse the free memblocks in the system, skip the allocated areas, separate the memblocks that meet the requirements, and register them into the dedicated memory pool until the requirements are met;

[0010] S2 isolates the separated large page memory from the memory within the buddy, and does not create struct page objects for this part of the memory pages;

[0011] S3 resolves the numa topology during the Linux kernel system startup phase, updates the memblock information, and then reserves large page memory.

[0012] S4 specifies the memory alignment size.

[0013] Preferably, in S1, the idle memblocks in the traversal system are accessed sequentially to access the collection, data structure, or data element. This process includes determining the starting position of the traversal, iterating, accessing the element, moving the next element, iterating again until the termination condition is triggered to complete the traversal.

[0014] Preferably, the struct page object in S2 is a data structure used to represent a memory page, which corresponds to a physical page in the system memory. In the Linux system, the struct page object includes information about the status of the physical page.

[0015] Preferably, the memory management process includes the following steps:

[0016] A1 is organized according to the numa topology;

[0017] A2 isolates a contiguous memory region within numa.

[0018] Preferably, the numa topology in A1 includes defining the distance between different numa nodes in the system, memory access latency, and the distribution of data among the nodes.

[0019] Preferably, in the memory management, some virtual memory pages can be stored in physical memory, while other pages are stored on the disk to achieve page swapping.

[0020] Preferably, the process of isolating contiguous memory regions within numa is performed by querying whether the local memory in the numa node is contiguous in physical address, and separating the contiguous local memory that meets the conditions.

[0021] Preferably, the operation method of using the reserved large page memory pool includes the following steps;

[0022] B1, entering the directory mounted by the large page file system first, opening a file, and finding a continuous virtual address segment in the virtual address space;

[0023] B2, finding a piece of physical memory in the large page memory pool, and establishing the relationship between the physical memory and the virtual address;

[0024] B3, after the relationship between the physical memory and the virtual address is established, using the reserved large page memory;

[0025] B4, releasing the occupied large page memory and recycling it to the idle warehouse managed by the large page.

[0026] Preferably, the relationship between the physical memory and the virtual address in S2 is established by using a page table or a page mapping, and the abstraction of the virtual address space is realized by using the page table and address conversion, so that the application program can be executed without directly accessing the physical memory.

[0027] Technical effects and advantages of the present application:

[0028] The present application reduces the number of system page metadata management, optimizes the large page organization and management of the kernel, and realizes the saving of 16MB memory space on each 1GB large page memory of the large page management system. When the server is configured with more large pages, more memory can be released, and the released memory can be directly used for cloud host or allocated to the server system for use, thereby improving the memory utilization. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The present application provides a memory reservation diagram.

[0030] Figure 2 The present application provides a memory management diagram. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0032] The present application provides a kernel large page memory optimization method as shown in Figures 1-2 The optimization method comprises the following steps:

[0033] The first step is to reserve the memory, in the Linux system, the buddy memory management subsystem takes over the idle memory, and establishes struct page metadata to manage 4KB memory pages, and sets parameters for memory reservation when the Linux kernel system starts and after starting;

[0034] The second step is to manage the memory, apply and release the large page through the memory pool, and establish a two-layer memory management architecture to manage and control the memory;

[0035] The third step is to use the memory, by establishing the connection between physical memory and virtual address, and using the reserved large page memory through the large page memory pool.

[0036] It should be noted that "memblock" generally refers to a block of memory, which represents a continuous area in memory, and the memory block includes a certain number of memory addresses for storing data or programs. For memory reservation, the numa node is a computer architecture design aimed at optimizing the memory access performance of multi-processor systems. In the NUMA architecture, the computer's memory subsystem is divided into multiple NUMA nodes, each node containing a part of the processor core (CPU) and the memory directly associated with these cores. Since most existing servers have multi-numa structure, and the memory information obtained in the initial stage of system startup does not contain numa-related information, in order to make the large page reservation compatible with the numa node, the numa topology is parsed in the system startup stage, and the memblock information is updated, and then the large page memory is reserved. In this way, the large page memory on each numa node can be set according to the configuration information.

[0037] In the first step, the memory reservation process includes the following steps:

[0038] S1, first traverse the idle memblock in the system, skip the allocated area, separate the memblock that meets the requirements, and register it in the dedicated memory pool until the requirements are met;

[0039] S2, isolate the separated large page memory from the buddy memory, and do not establish struct page objects for this part of memory pages;

[0040] S3, parse the numa topology in the Linux kernel system startup stage, and update the memblock information, and then reserve the large page memory after updating;

[0041] S4, specify the memory alignment size.

[0042] In S1, the idle memblock in the system is sequentially accessed in order by traversing the set, data structure or data element, which includes determining the starting position of the traversal, iterating, accessing the element, moving the next element, iterating again until the end condition is triggered to complete the traversal.

[0043] In S2, the struct page object is a data structure for representing a memory page, which corresponds to a physical page in the system memory. In the Linux system, the struct page object includes information about the state of the physical page.

[0044] Embodiment 1, taking X86 system as an example;

[0045] The information in the kernel startup phase is obtained from the e820 table. Since the memory part in this part of information only provides region description and type, it does not provide numa related information, so it is necessary to wait until the ACPI STAT obtains the numa topology and adjusts the region range and node information of the memblock. At this time, the memory in the memblock has corresponding numa node information. At this time, the large page can be configured. At the same time, the reserved memory is removed from the memblock before the page initialization of the kernel to form a large page memory pool. After that, the memory in the memblock will be included in the buddy subsystem during the page initialization.

[0046] For the above operation, there are two ways for large page memory reservation, the specific operation is as follows:

[0047] The first is to specify the number and size of memory reservation by specifying cthugepages=[value] cthugepage_size=[value] in the kernel startup parameter;

[0048] The second is to use the command mount-t cthugetlbfs-opagesize= <value>cthugetlbfs <mount directory> to mount this huge page file system.

[0049] In the process of performing memory reservation, the memory alignment size can be specified through kernel start parameters or at the time of mounting.

[0050] The management process of the memory includes the following steps:

[0051] A1, organized according to the numa topology;

[0052] A2, isolate the memory region with continuous memory in the numa.

[0053] The numa topology in A1 includes defining the distance between different numa nodes in the system, memory access delay, and the distribution of data between nodes.

[0054] It should be noted that the numa topology refers to the organization structure of physical connection and memory access mode between NUMA nodes in a multi-processor system with NUMA architecture, including NUMA nodes, physical distance, memory distribution, CPU scheduling and performance optimization. In Linux and other operating systems, you can use tools and commands to view the NUMA topology information of the system to help optimize applications and workloads.

[0055] In the management of the memory, part of the virtual memory pages can be stored in the physical memory, while the other pages are stored on the disk to realize page swapping.

[0056] It should be noted that the physical memory is a hardware resource in a computer system, used to store programs and data. It usually includes RAM (Random Access Memory) modules. The size of the physical memory is usually limited, and different computer systems have different physical memory capacities.

[0057] For the memory region with continuous memory isolated in the numa, it is screened by querying whether the local memory in the numa node is continuous in the physical address, and the local memory that meets the continuous condition is separated.

[0058] Embodiment 2, according to Figure 2 Taking 384G memory as an example, during the system startup phase, memory will usually be applied in the memory range of memblock, including the ranges of [95G, 96G], [126G, 128G] and [316G, 320G]. Therefore, the three memory ranges must be skipped, and finally five mem regions are formed. The specific number of memory regions may be different depending on different circumstances.

[0059] The operation method of the third step of the large page memory pool using the reserved large page memory comprises the following steps.

[0060] B1, first enter the directory mounted by the large page file system, open a file, and find a continuous virtual address segment in the virtual address space;

[0061] B2, find a physical memory in the large page memory pool, and establish the relationship between the physical memory and the virtual address;

[0062] B3, after the relationship between the physical memory and the virtual address is established, the reserved large page memory can be used;

[0063] B4, release the occupied large page memory, and store it in the idle warehouse of the large page management.

[0064] The relationship between the physical memory and the virtual address in S2 is established by using a page table or a page mapping, and the abstraction of the virtual address space is realized by using the page table and the address conversion, so that the application program can be executed without directly accessing the physical memory.

[0065] It should be noted that the page table is a core component of the memory management system of the operating system, which is used to establish the mapping relationship between the physical memory and the virtual address. The operating system divides the virtual address into pages (usually 4KB in size), each page is associated with a physical page frame in the physical memory, and the page table records the mapping relationship between the virtual page and the physical page.

[0066] Embodiment 3, using qemu to use the large page memory as an example;

[0067] When qemu wants to use the previously reserved large page memory, it will first enter the directory mounted by the large page file system, open a file, and then find a continuous address segment in the virtual address space of qemu through the mmap system call.

[0068] In actual operation, the physical address corresponding to this virtual address segment usually does not exist, and accessing this address space will trigger the page fault of the memory large page file system. Then a physical memory is found in the large page memory pool, and the relationship with the virtual address is established. The get_user_page function also needs to adapt to this large page memory management file system.

[0069] When qemu obtains the required large page from the large page memory management file system and mmap to its own virtual address space, if at this time the guest reads and writes to a memory, since this memory has not yet established a relationship with the real physical memory, a VM_exit is triggered, and the ept page fault function of the EPT mechanism is entered, further, the kvm module calculates the corresponding pfn through the gpa corresponding to the hva address and the get_user_page function, and then finds the corresponding physical memory through the pagefault function of the large page file system to establish the corresponding relationship between the guest memory and the host memory.

[0070] It should be noted that VM_EXIT is a concept in virtualization technology, which represents the process of switching the virtual machine from the virtualization mode to the non-virtualization mode or from one virtual machine instance to another virtual machine instance, including the virtualization mode (VMX Root Mode): in this mode, the virtual machine monitor (usually the virtualization management program or hypervisor) has complete control and can manage and control multiple virtual machine instances, and the virtual machine mode (VMXNon-Root Mode): in this mode, the virtual machine instance runs in the virtualization environment, and the virtualized operating system and application are executed in this virtual machine.

[0071] At this time, when the life cycle of qemu ends, if the occupied large page memory is released at this time, the present scheme will recycle these large pages and store them in the idle repository in the memory large page management.

[0072] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application, although the present application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced, any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.< / value>

Claims

1. A kernel big page memory optimization method, characterized in that, The optimization method includes the following steps: The first step is to reserve memory. In the Linux kernel system, the buddy memory management subsystem takes over the free memory and establishes struct page metadata to manage the 4KB memory page. At the same time, parameters are set to reserve memory when the Linux kernel system starts up and after startup. The second step is to manage memory by allocating and releasing large pages through a memory pool, and establishing a two-tier memory management architecture to manage and control memory. The third step is to utilize memory by establishing a connection between physical memory and virtual addresses, and by using the reserved large page memory pool. The first step, the memory reservation process includes the following steps; S1: First, traverse the free memblocks in the system, skip the allocated areas, separate the memblocks that meet the requirements, and register them into the dedicated memory pool until the requirements are met; S2 isolates the separated large page memory from the memory within the buddy, and does not create struct page objects for this part of the memory pages; S3 resolves the numa topology during the Linux kernel system startup phase, updates the memblock information, and then reserves large page memory. S4 specifies the memory alignment size; The memory management process includes the following steps: A1 is organized according to the numa topology; A2 isolates a contiguous memory region within numa; The operation method for using the reserved large page memory pool includes the following steps; B1. First, enter the directory where the large page file system is mounted, open a file, and find a contiguous virtual address segment through the virtual address space. B2, a block of physical memory found in the large page memory pool, establishes the relationship between it and the virtual address; B3, once the relationship between physical memory and virtual address is established, the reserved large page memory can be used; B4 releases the occupied large page memory and reclaims it, storing it in the free warehouse of the large page management; In the memory usage described in S2, the relationship between physical memory and virtual address is established through page tables or page mapping. By utilizing page tables and address translation, the abstraction of the virtual address space is achieved, enabling applications to execute without directly accessing physical memory.

2. The kernel big page memory optimization method according to claim 1, characterized in that, In S1, the idle memblocks in the traversal system are accessed sequentially to access sets, data structures, or data elements. This process includes determining the starting position of the traversal, iterating, accessing elements, moving the next element, iterating again until the termination condition is triggered to complete the traversal.

3. The kernel big page memory optimization method according to claim 2, characterized in that, The structpage object in S2 is a data structure used to represent a memory page, which corresponds to a physical page in the system memory. In the Linux system, the structpage object includes information about the physical page status.

4. The kernel big page memory optimization method according to claim 3, characterized in that, The numa topology in A1 includes defining the distance between different numa nodes in the system, memory access latency, and how data is distributed among the nodes.

5. The kernel big page memory optimization method according to claim 4, characterized in that, In the memory management described above, some virtual memory pages can be stored in physical memory, while other pages are stored on the disk, thus enabling page swapping.

6. The kernel big page memory optimization method according to claim 5, characterized in that, The process of isolating contiguous memory regions within numa is achieved by querying whether the local memory in the numa node is contiguous in physical address, and separating contiguous local memory that meets the criteria.

Images