Memory management method, host machine, electronic device, storage medium and program product

Abstract

The embodiment of the present application provides a memory management method, a host machine, an electronic device, a storage medium and a program product, which relate to the field of computer technology. The method includes: in response to detecting a memory allocation request of kernel data of a virtual machine, allocating a virtual memory page having an address mapping relationship with a reserved memory area for the kernel data, wherein the reserved memory area is located in the virtualized physical address space of the virtual machine and has a continuous virtualized physical address, and the virtualized physical address space is a virtual memory provided by the host machine of the virtual machine; updating the address mapping of the virtual memory page on the virtualized physical address space in a first page table, wherein the virtual memory page is located in the virtual address space of the virtual machine, and the first page table is used to indicate the address mapping relationship between the virtual address space and the virtualized physical address space. The technical solution of the embodiment of the present application can avoid the randomness of kernel data memory allocation and reduce the generation of memory fragmentation.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a memory management method, a host machine, an electronic device, a computer-readable storage medium, and a computer program product, which can be applied to the field of virtualized memory management technology. Background Art

[0002] Memory fragmentation means that the available free blocks in the memory are divided into many discontinuous small blocks. The operating system kernel data is distributed in the non-removable memory of the system. Since these kernel data are frequently allocated and released when the system is running, they may form many small and scattered blocks in the memory. In the related technology, memory hot-plugging operations are supported at the granularity of large pages (HP) (such as 2MB). However, the kernel data may be scattered in these 2MB memory blocks, resulting in these memory blocks cannot be released. Although the total amount of free memory may be sufficient, since these memory blocks are scattered, large blocks of continuous memory cannot be allocated. Therefore, how to reduce memory fragmentation is a technical problem that needs to be solved urgently. Summary of the invention

[0003] Embodiments of the present application provide a memory management method, a host machine, an electronic device, a computer-readable storage medium, and a computer program product to alleviate or solve one or more technical problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a memory management method, which is applied to a virtual machine, and the method includes:

[0005] In response to detecting a memory allocation request for kernel data of the virtual machine, a virtual memory page having an address mapping relationship with the reserved memory area is allocated to the kernel data, wherein the reserved memory area is located in a virtualized physical address space of the virtual machine and has a continuous virtualized physical address, and the virtualized physical address space is a virtual memory provided by a host machine of the virtual machine; and an address mapping of the virtual memory page on the virtualized physical address space is updated in a first page table, wherein the virtual memory page is located in the virtual address space of the virtual machine, and the first page table is used to represent the address mapping relationship between the virtual address space and the virtualized physical address space.

[0006] In a second aspect, an embodiment of the present application provides a memory management method, which is applied to a host machine, and the method comprises: allocating a virtualized physical address space to a virtual machine from the physical address space of the host machine, wherein the virtualized physical address space includes a reserved memory area with a continuous virtualized physical address, and the virtual machine is used to allocate virtual memory pages having an address mapping relationship with the reserved memory area to the kernel data of the virtual machine; and using a second page table to represent the address mapping relationship between the physical address space and the virtualized physical address space.

[0007] In a third aspect, an embodiment of the present application provides a host machine, comprising: a non-volatile memory storing a memory management program; a physical memory providing a physical address space; a processor carrying one or more virtual machines, wherein the processor implements any method of the embodiments of the present application when executing the memory management program.

[0008] In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and the processor implements any method of the embodiment of the present application when executing the computer program.

[0009] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of any one of the embodiments of the present application is implemented.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any method of the embodiments of the present application when executed by a processor.

[0011] According to the technical solution of the embodiment of the present application, a reserved memory area is fixedly set in the virtualized physical address space of the Guest side, and when a memory allocation request for kernel data is subsequently detected, the corresponding virtual memory page is allocated to the kernel data from the reserved memory area. In this way, the randomness of kernel data memory allocation can be avoided, and the memory occupied by the kernel data can be concentrated in a fixed area, thereby reducing the generation of memory fragmentation.

[0012] Optionally, by identifying cold pages in virtual memory pages, releasing virtualized physical memory pages corresponding to the cold pages from the reserved memory area, and taking the virtualized physical memory pages corresponding to the cold pages offline to prevent them from being reallocated, on the one hand, kernel memory page migration in the immovable memory area can be achieved, and on the other hand, since the released memory resources can be used by other kernel data, the use of memory resources can be optimized.

[0013] Optionally, by identifying hot pages in virtual memory pages, physical memory pages corresponding to the hot pages are merged into target pages whose sizes are larger than a size threshold, such as large-size pages HP. Based on the use of large-size pages HP, memory fragmentation can be further reduced and memory management overhead caused by too many page table entries can be reduced.

[0014] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0016] Figure 1 A memory management method for a virtual machine in the related art is shown;

[0017] Figure 2A A schematic diagram showing the structure of a host machine 200 provided in an embodiment of the present application is shown;

[0018] Figure 2B A schematic diagram of the system architecture of a host machine 200 provided in an embodiment of the present application is shown;

[0019] Figure 2C The following is an architecture diagram of a memory management system 210 provided in an embodiment of the present application;

[0020] Figure 2D A schematic diagram showing a memory address space of a memory management system 210 provided in an embodiment of the present application;

[0021] Figure 3 A flowchart of a memory management method 300 according to an embodiment of the present application is shown;

[0022] Figure 4 A flowchart of a memory management method 400 according to an embodiment of the present application is shown;

[0023] Figure 5 A block diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0025] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0026] For ease of understanding, let's first introduce the terms that will be used below:

[0027] Kernel state: It is the state or space domain in which the operating system kernel (Kernel) runs. The operating system kernel itself runs in kernel state, including key functions such as memory management, process scheduling, file system management, and drivers.

[0028] User state: refers to the state or space domain where applications run at a lower permission level. User state runs all common applications, such as browsers, text editors, games, etc.

[0029] 32-bit Direct Memory Access 32 (DMA32): is a special memory area used to support devices that require Direct Memory Access (DMA). These devices can only access less than 4GB of physical memory (virtualized physical memory when it is on the virtual machine side). It is used to solve the compatibility issues of some 32-bit hardware devices when performing DMA on 64-bit systems.

[0030] Normal memory: It is a memory area commonly used in the operating system, mainly used for memory allocation of kernel data and normal operations of the operating system.

[0031] Movable memory: It is a movable memory area, usually used for memory allocation in user mode (such as anonymous memory mapping and page cache).

[0032] malloc function: a standard memory allocation interface in user space, used to dynamically allocate memory pages of a specified size. The size and number of allocated memory pages can be dynamically adjusted while the program is running.

[0033] Slab allocator: It is a memory allocation mechanism of the Linux operating system. It reduces memory fragmentation and improves allocation efficiency by dividing memory into fixed-size caches (called Slabs). Slab allocators are usually used to allocate and manage fixed-size memory blocks.

[0034] vmalloc function: is a kernel interface used to allocate virtual memory pages with continuous virtual addresses without requiring the physical memory to be continuous. The continuity of virtual memory pages is achieved by mapping non-contiguous physical pages.

[0035] Page Table (pgtable) interface: It is the management and allocation interface of the page table. The page table is used by the operating system to represent the mapping relationship between the virtual address and the physical address of the page.

[0036] Buddy System: It is an algorithm for memory management. It manages and allocates memory efficiently and reduces memory fragmentation by splitting memory blocks into blocks of sizes that are powers of 2. When the system needs to allocate a block of memory, it first finds the smallest block that meets the requirement. If there is no block of the right size, the system will split the larger block into two equal small blocks until a block of the right size is found. When the memory is no longer in use, the system will release the memory and try to merge it with the adjacent "buddy" block. If both "buddy" blocks are free and the same size, they can be merged into a larger block. This process can be done recursively until it can no longer be merged.

[0037] Memory fragmentation: refers to the free blocks available in the memory being divided into many discontinuous small blocks.

[0038] Figure 1 The memory management method of the virtual machine in the related art is shown, such as Figure 1 As shown in the figure, the virtualized physical address space of the virtual machine (Guest) includes direct memory, non-removable memory and removable memory. Among them, the virtualized physical address space is the virtual memory allocated by the host of the virtual machine. Direct memory can realize direct memory access of certain hardware devices, such as DMA32; non-removable memory is mainly used for memory allocation of kernel data, such as Normal memory; removable memory is mainly used for memory allocation of user data.

[0039] Memory allocation on a virtual machine (Guest) includes memory allocation in kernel mode and memory allocation in user mode. The memory allocation process in user mode includes: the application process calls the user data memory allocation interface to send a memory allocation request, and the system allocates a virtual memory page for it from the removable memory; the memory allocation process in kernel mode includes the kernel process calls the kernel data memory allocation interface to send a memory allocation request, and the system allocates a virtual memory page for it from the non-removable memory. The virtual memory page can be understood as a memory page with a virtual address. The user data memory allocation interface is, for example, the malloc function, a standard memory allocation interface for user mode space. The kernel data memory allocation interface is, for example, the Slab allocator or the vmalloc function or the pgtable interface.

[0040] Since the operating system kernel data is distributed in the system's non-removable memory, and these kernel data are frequently allocated and released when the system is running, they may form many small and scattered blocks in the memory. Even if the memory hot-plug operation is performed with the granularity of large-size pages HP (such as 2MB granularity). However, the kernel data may be scattered in these 2MB memory blocks, resulting in these memory blocks cannot be released. Although the total amount of free memory may be sufficient, since these memory blocks are scattered, large blocks of continuous memory cannot be allocated, so the problem of memory fragmentation still exists, which will affect the container shrinking in the virtualization scenario.

[0041] The embodiments of the present application are intended to provide a memory management method, a host machine, an electronic device, a computer-readable storage medium, and a computer program product to reduce memory fragmentation and improve the utilization of memory resources. Figure 2A and Figure 2B The hardware architecture and software architecture of the embodiment of the present application are introduced.

[0042] Figure 2A A schematic diagram of the structure of a host machine (Host) 200 provided in an embodiment of the present application is shown. Figure 2B A schematic diagram of the system architecture of a host machine provided in an embodiment of the present application is shown. Figure 2A and Figure 2B As shown, the hardware layer structure of the host machine 200 may include a processor 201 , a non-volatile memory 202 , and a physical memory 203 .

[0043] The processor 201 is, for example, a central processing unit (CPU); the physical memory 203 is, for example, a random access memory (RAM), which provides a physical address space of the host 200. The number of the processor 201, the non-volatile memory 202 and the physical memory 203 can be one or more, and they are interconnected through a bus 204 to complete mutual communication.

[0044] When the host machine (Host) 200 is started, the processor 201 first reads the startup program from the non-volatile memory 202 to load the Host operating system 206. The Host operating system 206 runs in the Host kernel space, controls and manages the actual hardware resources (including the processor 201, the non-volatile memory 202 and the physical memory 203, etc.), and runs one or more Host applications 207. The Host user space provides an environment for running the Host application 207 on the Host operating system 206. Further, in the virtualization scenario, the Host operating system 206 will load the Kernel-based Virtual Machine (KVM) module 208. When the user starts one or more virtual machines (Guests) 205 on the HOST operating system 206, the KVM module 208 is responsible for creating virtual hardware resources for the virtual machines 205. The Guest operating system runs in the Guest kernel space, is responsible for managing the virtual hardware resources in the virtual machines 205, and provides an environment for running Guest applications in the Guest user space.

[0045] In the embodiment of the present application, a memory management program is stored in the non-volatile memory 202. When the processor 201 executes the memory management program, the memory management method of the embodiment of the present application can be implemented. Figure 2C and Figure 2D Give a detailed introduction.

[0046] Figure 2C 2 shows an architecture diagram of a memory management system 210 provided in an embodiment of the present application. The memory management system 210 can be run on a host machine (Host) 200. The memory management program stored in the non-volatile memory 202 includes a host memory management subroutine and a guest memory management subroutine. Figure 2CAs shown, after the Host operating system is started, the Host memory management subroutine is read from the non-volatile memory 202, thereby loading the memory management unit 211, and the memory management unit 211 implements the memory management method on the Host side. After the Guest operating system is started, it interacts with the KVM module to read the Guest memory management subroutine from the non-volatile memory 202, thereby loading the memory allocator 212, and the memory allocator 212 implements the memory management method on the Guest side.

[0047] Figure 2D A schematic diagram of the memory address space of the memory management system 210 is shown. The memory management system 210 performs memory management according to the set memory management granularity, and each memory management granularity can be called a page, or can also be called a memory block. Each page corresponds to its memory address. The memory address space can be divided into the physical address space on the host side, the virtualized physical address space on the guest side, and the virtual address space on the guest side.

[0048] like Figure 2C and Figure 2D As shown, the physical memory 202 provides a physical address space on the host side, and the physical address space includes multiple physical address pages, that is, the physical address pages are located in the physical address space.

[0049] The memory management unit 211 on the host side can select physical memory pages with continuous or discontinuous physical addresses from the physical address space, and allocate virtual memory pages with an address mapping relationship with the physical memory page to each process on the guest, thereby constructing a virtualized physical address space on the guest side. In other words, the virtualized physical address space on the guest side contains multiple virtual memory pages allocated by the host side, but from the perspective of the guest, these virtual memory pages are its physical memory pages. Therefore, in an embodiment of the present application, the virtual memory page allocated by the host to the guest is called a virtualized physical memory page. The virtualized physical memory page corresponds to a virtualized physical address and is located in the virtualized physical address space on the guest side.

[0050] The address mapping relationship between the physical address space and the virtualized physical address space is represented by the second page table, that is, the second page table includes the address mapping between the virtualized physical memory page and the physical memory page. Exemplarily, the second page table is managed by the memory management unit 211 on the host side.

[0051] The virtualized physical address space includes a non-removable memory area and a movable memory area. The non-removable memory area is, for example, the Normal memory. Among them, the Host will allocate virtualized physical memory pages for each process on the Guest. These processes can come from the Guest kernel space, such as the Guest operating system process. The allocated virtualized physical memory pages can be further used for memory allocation of the Guest operating system kernel data. Accordingly, this part of the virtualized physical memory pages is located in the non-removable memory area. These processes can also come from the Guest user space, such as the Guest application process. The allocated virtualized physical memory pages can be further used for memory allocation of the Guest user data. Accordingly, this part of the virtualized physical memory pages is located in the movable memory area.

[0052] like Figure 2D As shown, the memory management module on the guest side can select virtualized physical memory pages with continuous or discontinuous virtualized physical addresses from the virtualized physical address space, and allocate virtual memory pages with address mapping relationship with the virtualized physical memory pages to each process on the guest, thereby constructing the virtual address space on the guest side. In other words, the virtual address space on the guest side contains multiple virtual memory pages.

[0053] The address mapping relationship between the virtualized physical address space and the virtual address space is represented by a first page table, that is, the first page table includes address mappings between virtualized physical memory pages and virtual memory pages. Exemplarily, the first page table is managed by the Guest operating system kernel.

[0054] like Figure 2C As shown, the memory allocator 212 can apply to the Host for a reserved memory area 213, which is located in the virtualized physical address space and has a continuous virtualized physical address. Figure 2C and Figure 2D As shown, the reserved memory area 213 is located in the non-movable memory area and is used to allocate virtual memory pages for kernel data of the Guest operating system.

[0055] Exemplarily, the initial memory of the guest can be set in the configuration file of the KVM module 208. For example, the init_mem_size parameter in the configuration file determines how much virtualized physical memory the guest will obtain when it is created or started. After the guest is started, the guest operating system kernel will reserve a continuous area of ​​virtualized physical addresses as a reserved memory area according to the init_mem_size parameter.

[0056] That is to say, in an embodiment of the present application, a reserved memory area is fixedly set in the virtualized physical address space on the Guest side, and subsequent virtual memory pages of kernel data are allocated from the reserved memory area. In this way, the randomness of kernel data memory allocation can be avoided, and the memory occupied by kernel data can be concentrated in a fixed area, thereby reducing the generation of memory fragmentation.

[0057] The following is a detailed description of the host-side memory management method and the guest-side memory management method of the present application with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0058] Figure 3 The flowchart of the memory management method 300 of the embodiment of the present application is shown. The memory management method 300 can be applied to a virtual machine (Guest), for example, implemented by a memory allocator of the virtual machine. It should be noted that the memory allocator runs in the Guest kernel space, that is, the memory management method 300 is used to implement the memory management of the Guest kernel data. Figure 3 As shown, the memory management method 300 may include step S301 and step S302.

[0059] Step S301: In response to detecting a memory allocation request for kernel data of a virtual machine, a virtual memory page having an address mapping relationship with a reserved memory area is allocated for the kernel data, wherein the reserved memory area is located in the virtualized physical address space of the virtual machine and has a continuous virtualized physical address, and the virtualized physical address space is a virtual memory provided by a host machine of the virtual machine.

[0060] Kernel data refers to various data structures and information used and managed by the Guest operating system during operation, including but not limited to process information, memory management data, file system data, driver data, etc. Memory allocation requests are usually issued by different modules or subsystems, such as processes, memory management, device drivers, file systems, or applications. Figure 2C As shown, the memory allocator 212 on the Guest side will detect whether the memory allocation request is a memory allocation request for kernel data.

[0061] In one example, if the memory allocation request comes from a Guest operating system process, it can be determined that the memory allocation request is a memory allocation request for kernel data.

[0062] In another example, the Guest process can call a memory allocation interface to implement a memory allocation request. When the Guest process needs to allocate memory, the information with a memory allocation identifier (such as "GFP_KERNEL" or "GFP_Movable", etc.) is passed to a memory allocation interface (such as kmalloc function, vmalloc function or alloc_pages function or malloc function, etc.) to send a memory allocation request. These information can indicate the source of memory allocation (such as a removable memory area or a non-removable memory area or a DMA memory area). Wherein, the kernel data process corresponds to its exclusive kernel memory allocation identifier, such as "GFP_KERNEL", and the memory allocator 212 detects the memory allocation request of each process, including screening and filtering, to identify the kernel memory allocation identifier. Once the kernel memory allocation identifier is identified in the memory allocation request, it is determined that the memory allocation request is a memory allocation request for kernel data.

[0063] If it is determined that the memory allocation request is a memory allocation request for kernel data, it will be handed over to the memory allocator 212 for management. The memory allocator 212 will select a virtualized physical memory page from the reserved memory area 213 and allocate it to the kernel data. The virtual memory page that has an address mapping relationship with the virtualized physical memory page is the virtual memory page of the kernel data. If it is determined that the memory allocation request is not a memory allocation request for kernel data, the partner system on the Guest will perform memory management. In the embodiment of the present application, the "management" of memory includes any one or more operations of allocating memory, releasing (reclaiming) memory, initializing memory, expanding memory, shrinking memory, aligning memory, etc., which can be determined based on actual conditions.

[0064] Exemplarily, the memory allocator 212 allocates virtual memory pages for kernel data by calling a kernel data memory allocation interface such as a vmalloc function or an alloc_contig_pages function.

[0065] Among them, the vmalloc function can allocate virtual memory pages whose virtual addresses are continuous but whose corresponding virtualized physical addresses may not be continuous. The vmalloc function is applicable to situations where a large amount of continuous virtual address space is required, so that memory can be managed and mapped more flexibly. When it is necessary to migrate the virtual memory page allocated by vmalloc from the current location to another location, the address mapping of the previously allocated virtual memory page on the virtualized physical address space can be released (unmap) to facilitate reallocation of this area, or to migrate the kernel data on this area to other locations, so that the migration goal of the kernel page can be achieved in a virtualized environment, thereby managing and optimizing memory resources more flexibly.

[0066] The alloc_contig_pages function can allocate virtual memory pages that correspond to continuous virtualized physical addresses. When using the alloc_contig_pages function to allocate memory, if there are migratable pages in the target virtualized physical memory area, the system will first migrate these pages to other locations to ensure that the virtualized physical addresses corresponding to the allocated virtual memory pages are continuous.

[0067] Step S302: updating the address mapping of the virtual memory page on the virtualized physical address space in the first page table, wherein the virtual memory page is located in the virtual address space of the virtual machine, and the first page table is used to represent the address mapping relationship between the virtual address space and the virtualized physical address space.

[0068] After the memory allocator 212 selects a virtualized physical memory page from the reserved memory area 213 and allocates it to the kernel data as the virtual memory page of the kernel data, it updates the address mapping between the selected virtualized physical memory page and the virtual memory page of the kernel data in the first page table.

[0069] According to the memory management method 300 of the embodiment of the present application, a reserved memory area is fixedly set in the virtualized physical address space of the Guest side, and when a memory allocation request for kernel data is subsequently detected, the corresponding virtual memory page is allocated to the kernel data from the reserved memory area. In this way, the randomness of kernel data memory allocation can be avoided, and the memory occupied by the kernel data can be concentrated in a fixed area, thereby reducing the generation of memory fragmentation.

[0070] In one embodiment, the reserved memory area includes multiple sub-areas with continuous virtualized physical addresses, the size of the sub-areas is the same as the size of the target page, and the size of the target page is greater than the size threshold. In step S301, the corresponding virtual memory page is allocated to the kernel data from the reserved memory area, which may include: selecting at least one sub-area from the multiple sub-areas to allocate to the virtual memory page.

[0071] The size of a standard page is usually 4KB, while the size of a large page (Huge Pages, HP) can be 2MB, 1GB or larger. In a memory management system, a page table (such as the first page table and the second page table) is required to represent the address mapping relationship, and each set of address mapping relationships corresponds to a page table entry (Page Table Entry, PTE). If a standard page is used as the memory management granularity, a large number of page table entries will be generated, so a large amount of memory will be consumed to manage the page table. The size threshold is, for example, 2M or 4M, that is, the target page is a large page HP.

[0072] In an embodiment of the present application, the reserved memory area is divided into a plurality of sub-areas of HP size, so as to implement memory management granularity with HP size, for example, the reserved memory area is managed with 2MB as granularity, including memory allocation with HP size as granularity and memory release with HP size as granularity. Based on this, the memory management overhead caused by too many page table entries can be reduced, and the address translation of the Translation Lookaside Buffer (TLB) can be accelerated to improve the hit rate of the TLB. Among them, at least one sub-area can be selected from the plurality of sub-areas according to the memory demand and load.

[0073] In one embodiment, the virtual memory pages allocated in step S301 correspond to continuous virtualized physical addresses. For example, the memory allocator 212 selects virtual memory pages with continuous virtualized physical addresses from the reserved memory area and allocates them to kernel data by calling the alloc_contig_pages function. In a virtualized environment, memory allocation and release are frequent. By allocating virtual memory pages with continuous virtualized physical addresses, the efficiency of memory recovery and reuse can be better improved, and the impact of fragmentation can be further reduced.

[0074] In one embodiment, the memory management method 300 of the embodiment of the present application may also include: in response to identifying a cold page in at least one virtual memory page, sending a memory release request for the cold page to the host machine, the memory release request is used to request the host machine to release the physical memory page corresponding to the cold page from the physical address space of the host machine, wherein the cold page is a page whose access frequency within a preset time period is less than a first preset frequency threshold; releasing the virtualized physical memory page corresponding to the cold page from the reserved memory area, and unmapping the address of the cold page in the virtualized physical address space in the first page table.

[0075] Exemplarily, the Guest operating system kernel will periodically check and count the access status of each virtual memory page. For example, the access timestamp of the virtual memory page can be recorded through a soft interrupt; for another example, the access bit (Accessed bit) recorded in the page table entry in the first page table can be used to determine whether the virtual memory page has been accessed; for another example, the Guest operating system kernel can periodically scan the access status of the virtual memory page. The memory allocator 212 can obtain the access status of each virtual memory page, and then count whether the frequency of the virtual memory page being accessed within a preset time is less than the first preset frequency threshold. If it is less than the first preset frequency threshold, the virtual memory page is marked as a cold page.

[0076] After identifying the cold page, the memory allocator 212 may send a memory release request for the cold page to the host. For example, the memory allocator 212 may use the CPU instruction Hypercall to send a memory release request to the memory management unit 211 of the host, and notify the memory management unit 211 of the virtualized physical address corresponding to the cold page. The memory allocator 212 may determine the virtualized physical address corresponding to the cold page by searching the first page table.

[0077] After receiving the memory release request, the memory management unit 211 will search the second page table, determine the physical address corresponding to the cold page according to the virtualized physical address corresponding to the cold page, and release the physical memory page corresponding to the physical address in the physical address space. The memory management unit 211 will write the kernel data on the physical memory page to a disk or other storage device to implement page swapping. If these pages are accessed again, the memory management unit 211 can reload them into the physical address space.

[0078] Furthermore, the memory allocator 212 releases the virtualized physical memory pages corresponding to the cold pages from the reserved memory area, and unmaps the addresses of the cold pages in the virtualized physical address space in the first page table, thereby offline the virtualized physical memory pages corresponding to the cold pages to prevent them from being reallocated.

[0079] Based on this, on the one hand, kernel memory page migration in the non-movable memory area can be realized, and on the other hand, since the released memory resources can be used by other kernel data, the use of memory resources can be optimized.

[0080] In one embodiment, the memory management method 300 of the embodiment of the present application may also include: in response to identifying a hot page in at least one virtual memory page, sending a page integration request to the host machine, the page integration request is used to request the host machine to integrate the physical memory pages corresponding to the hot page into a target page, wherein the hot page is a page whose access frequency within a preset time period is greater than a second preset frequency threshold, and the size of the target page is greater than the size threshold.

[0081] Exemplarily, the memory allocator 212 may obtain access conditions of each virtual memory page, and then count whether the frequency of access to the virtual memory page within a preset time period is greater than a second preset frequency threshold, and if greater than the second preset frequency threshold, mark the virtual memory page as a hot page. The first preset frequency threshold is less than or equal to the second preset frequency threshold.

[0082] After identifying the hot page, the memory allocator 212 may send a page integration request for the hot page to the Host. For example, the memory allocator 212 may use the CPU instruction Hypercall to send a page integration request to the memory management unit 211 of the Host, and notify the memory management unit 211 of the virtualized physical address corresponding to the hot page. The memory allocator 212 may determine the virtualized physical address corresponding to the hot page by searching the first page table.

[0083] The size threshold is, for example, 2M or 4M, that is, the target page is a large-size page HP. After receiving the page integration request, the memory management unit 211 searches the second page table, determines the physical address corresponding to the hot page according to the virtualized physical address corresponding to the hot page, and releases the physical memory page corresponding to the physical address in the physical address space to merge into a large-size page HP.

[0084] Based on the use of HP, memory fragmentation can be further reduced, the memory management overhead caused by too many page table entries can be reduced, and the address translation of TLB can be accelerated to improve the hit rate of TLB.

[0085] Figure 4 FIG. 4 is a flowchart of a memory management method 400 according to an embodiment of the present application. The memory management method 400 may be applied to a host machine (Host), for example, implemented by a memory management unit of the host machine. Figure 4 As shown, the memory management method 400 may include:

[0086] Step S401: allocating a virtualized physical address space to a virtual machine from the physical address space of the host machine, wherein the virtualized physical address space includes a reserved memory area with continuous virtualized physical addresses, and the virtual machine is used to allocate virtual memory pages having an address mapping relationship with the reserved memory area to the kernel data of the virtual machine;

[0087] Step S402: using a second page table to represent an address mapping relationship between the physical address space and the virtualized physical address space.

[0088] In one embodiment, the memory management method 400 may further include: receiving a memory release request sent by a virtual machine for a cold page, wherein the cold page is a page identified by the virtual machine from at least one virtual memory page and whose access frequency within a preset time period is less than a first preset frequency threshold; releasing the physical memory page corresponding to the cold page from the physical address space, and releasing the address mapping of the virtualized physical memory page corresponding to the cold page in the physical address space in the second page table.

[0089] In one embodiment, the memory management method 400 may further include: receiving a page integration request sent by a virtual machine for a hot page, wherein the hot page is identified by the virtual machine from at least one virtual memory page and the frequency of access within a preset time period is greater than a second preset frequency threshold; integrating the physical memory pages corresponding to the hot page into a target page, and updating the address mapping of the integrated target page in the virtualized physical address space in the second page table, wherein the size of the target page is greater than the size threshold.

[0090] Among them, the implementation methods of each step in the memory management method 400 and the corresponding technical effects can refer to the corresponding descriptions in the above-mentioned host machine 200 and the memory management method 300, and will not be repeated here.

[0091] The following describes an exemplary application scenario of the memory management method of an embodiment of the present application.

[0092] The database system includes Runc mode and Rund mode. The Runc mode creates and runs lightweight containers on the host side, while the Rund mode creates and runs secure containers that are strongly isolated from each other on the host side. The Rund mode is suitable for providing tenants with security isolation that is almost the same as that of bare metal in multi-tenant cloud scenarios.

[0093] In Rund mode, in order to better achieve elastic scaling, it is necessary to scale the container. When scaling, more memory and resources need to be allocated to the container. If the memory is severely fragmented, even if the system has enough total memory, the scaling operation may fail because the required contiguous memory blocks cannot be allocated. When scaling, the container attempts to release some memory, but if these memory fragments are occupied or locked by other parts of the system, the memory may not be released effectively, resulting in scaling failure or inefficiency. This wastes memory resources and prevents other containers or applications from using this memory.

[0094] According to the technical solution provided in the embodiment of the present application, a reserved memory area is fixedly set in the virtualized physical address space on the Guest side, and the subsequent virtual memory pages of the kernel data are allocated from the reserved memory area. In this way, the randomness of the kernel data memory allocation can be avoided, and the memory occupied by the kernel data can be concentrated in a fixed area, thereby reducing the generation of memory fragmentation, so that memory resources can be more easily recycled and reallocated. For expansion operations, the system can allocate continuous memory more efficiently; for reduction operations, the system can release memory more easily. By reducing kernel memory fragmentation, the container can quickly obtain the required resources when expansion is needed, and can quickly release resources when reduction is needed, thereby improving overall elasticity. This enables the container to respond to load changes more flexibly, and can operate normally within a larger load range, reducing the risk of failure of expansion and reduction operations.

[0095] It should be noted that the above application scenarios or application examples provided in the embodiments of the present application are for ease of understanding, and the embodiments of the present application do not specifically limit the application of the technical solution.

[0096] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0097] Corresponding to the memory management method 300 provided in the embodiment of the present application, the embodiment of the present application also provides a memory management device, applied to a virtual machine, the device comprising: a virtual memory page allocation module, used to respond to a memory allocation request detected for the kernel data of the virtual machine, and allocate a virtual memory page having an address mapping relationship with a reserved memory area for the kernel data, wherein the reserved memory area is located in the virtualized physical address space of the virtual machine and has a continuous virtualized physical address, and the virtualized physical address space is the virtual memory provided by the host machine of the virtual machine; a first page table update module, used to update the address mapping of the virtual memory page on the virtualized physical address space in a first page table, wherein the virtual memory page is located in the virtual address space of the virtual machine, and the first page table is used to represent the address mapping relationship between the virtual address space and the virtualized physical address space.

[0098] In one embodiment, the device may also include: a memory release request sending module, used to send a memory release request for the cold page to the host machine in response to identifying a cold page in at least one of the virtual memory pages, the memory release request being used to request the host machine to release the physical memory page corresponding to the cold page from the physical address space of the host machine, wherein the cold page is a page whose access frequency within a preset time period is less than a first preset frequency threshold; a virtualized physical memory page release module, used to release the virtualized physical memory page corresponding to the cold page from the reserved memory area, and to release the address mapping of the cold page on the virtualized physical address space in the first page table.

[0099] In one embodiment, the device may also include: a page integration request sending module, which is used to send a page integration request to the host machine in response to identifying a hot page in at least one of the virtual memory pages, and the page integration request is used to request the host machine to integrate the physical memory pages corresponding to the hot pages into target pages, wherein the hot pages are pages whose access frequency within a preset time period is greater than a second preset frequency threshold, and the size of the target page is greater than a size threshold.

[0100] In one embodiment, the reserved memory area includes multiple sub-areas with continuous virtualized physical addresses, the size of the sub-areas is the same as the size of the target page, the size of the target page is greater than the size threshold, and the virtual memory page allocation module is also used to select at least one sub-area from the multiple sub-areas to allocate to the virtual memory page.

[0101] In one implementation, the virtual memory pages correspond to continuous virtualized physical addresses.

[0102] Corresponding to the memory management method 400 provided in the embodiment of the present application, the embodiment of the present application also provides a memory management device, which is applied to a host machine, and the device includes: a virtualized physical address space allocation module, which is used to allocate a virtualized physical address space to a virtual machine from the physical address space of the host machine, wherein the virtualized physical address space includes a reserved memory area with a continuous virtualized physical address, and the virtual machine is used to allocate virtual memory pages having an address mapping relationship with the reserved memory area to the kernel data of the virtual machine; a second page table representation module, which is used to use a second page table to represent the address mapping relationship between the physical address space and the virtualized physical address space.

[0103] In one embodiment, the device may also include: a memory release request receiving module, used to receive a memory release request sent by the virtual machine for a cold page, wherein the cold page is a page identified by the virtual machine from at least one of the virtual memory pages and whose access frequency within a preset time period is less than a first preset frequency threshold; a physical memory page release module, used to release the physical memory page corresponding to the cold page from the physical address space, and release the address mapping of the virtualized physical memory page corresponding to the cold page in the second page table on the physical address space.

[0104] In one embodiment, the device may also include: a page integration request receiving module, used to receive a page integration request sent by the virtual machine for a hot page, wherein the hot page is a page identified by the virtual machine from at least one of the virtual memory pages and whose access frequency within a preset time period is greater than a second preset frequency threshold; a page integration module, used to integrate the physical memory pages corresponding to the hot pages into target pages, and update the address mapping of the integrated target pages in the virtualized physical address space in the second page table, and the size of the target page is greater than the size threshold.

[0105] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0106] Figure 5 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 5 As shown, the electronic device includes: a memory 501 and a processor 502, and the memory 501 stores a computer program that can be run on the processor 502. When the processor 502 executes the computer program, the method in the above embodiment is implemented. The number of the memory 501 and the processor 502 can be one or more. In a specific implementation, the electronic device may also include a communication interface 503 for communicating with external devices and performing data exchange transmission.

[0107] In specific implementation, if the memory 501, the processor 502 and the communication interface 503 are implemented independently, the memory 501, the processor 502 and the communication interface 503 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0108] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0109] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0110] An embodiment of the present application provides a computer program product, including a computer program, which implements the method provided in the embodiment of the present application when executed by a processor.

[0111] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0112] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0113] It should be understood that the above processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0114] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0115] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0116] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0117] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0118] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0119] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0120] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0121] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0122] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A memory management method, applied to a virtual machine, the method comprising: In response to detecting a memory allocation request for kernel data of the virtual machine, a virtual memory page having an address mapping relationship with at least one sub-area of ​​a reserved memory area is allocated to the kernel data, wherein the reserved memory area is located in a non-movable memory area of ​​a virtualized physical address space of the virtual machine and has a continuous virtualized physical address, the non-movable memory area is a Normal memory, and the virtualized physical address space is a virtual memory provided by a host machine of the virtual machine; The address mapping of the virtual memory page on the virtualized physical address space is updated in a first page table, wherein the virtual memory page is located in the virtual address space of the virtual machine, and the first page table is used to represent the address mapping relationship between the virtual address space and the virtualized physical address space.

2. The method according to claim 1, further comprising: In response to identifying a cold page in at least one of the virtual memory pages, sending a memory release request for the cold page to the host machine, the memory release request being used to request the host machine to release a physical memory page corresponding to the cold page from the physical address space of the host machine, the cold page being a page whose access frequency within a preset time period is less than a first preset frequency threshold; The virtualized physical memory page corresponding to the cold page is released from the reserved memory area, and the address mapping of the cold page in the virtualized physical address space is released in the first page table.

3. The method according to claim 1, further comprising: In response to identifying a hot page in at least one of the virtual memory pages, a page integration request is sent to the host machine, wherein the page integration request is used to request the host machine to integrate the physical memory pages corresponding to the hot pages into target pages, the hot pages being pages whose access frequency within a preset time period is greater than a second preset frequency threshold, and the size of the target page is greater than a size threshold.

4. The method according to any one of claims 1 to 3, wherein: The reserved memory area includes a plurality of sub-areas with continuous virtualized physical addresses, the size of the sub-areas is the same as the size of the target page, the size of the target page is greater than a size threshold, and the corresponding virtual memory page is allocated to the kernel data from the reserved memory area, including: At least one of the sub-areas is selected from the plurality of the sub-areas and allocated to the virtual memory page.

5. The method according to any one of claims 1 to 3, wherein: The virtual memory pages correspond to continuous virtualized physical addresses.

6. A memory management method, applied to a host machine, the method comprising: Allocating a virtualized physical address space to a virtual machine from the physical address space of the host machine, wherein the non-removable memory area of ​​the virtualized physical address space includes a reserved memory area with a continuous virtualized physical address, the non-removable memory area is a Normal memory, and the virtual machine is used to allocate a virtual memory page having an address mapping relationship with at least one sub-area of ​​the reserved memory area to kernel data of the virtual machine; A second page table is used to represent the address mapping relationship between the physical address space and the virtualized physical address space.

7. The method according to claim 6, further comprising: Receiving a memory release request sent by the virtual machine for a cold page, wherein the cold page is a page identified by the virtual machine from at least one of the virtual memory pages and whose access frequency within a preset time period is less than a first preset frequency threshold; The physical memory page corresponding to the cold page is released from the physical address space, and the address mapping of the virtualized physical memory page corresponding to the cold page in the physical address space is released in the second page table.

8. The method according to claim 6, further comprising: receiving a page consolidation request sent by the virtual machine for a hot page, wherein the hot page is a page identified by the virtual machine from at least one of the virtual memory pages and whose access frequency within a preset time period is greater than a second preset frequency threshold; The physical memory pages corresponding to the hot pages are integrated into a target page, and the address mapping of the integrated target page in the virtualized physical address space is updated in the second page table, wherein the size of the target page is greater than a size threshold.

9. A host machine, comprising: A non-volatile memory storing a memory management program; Physical memory, which provides physical address space; A processor carrying one or more virtual machines, wherein the processor implements the method according to any one of claims 1 to 8 when executing the memory management program.

10. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images