Memory allocation method, system and equipment in Linux environment and storage medium

Abstract

The embodiment of the invention provides a memory allocation method, system and equipment in a Linux environment and a storage medium, and the method comprises the following steps: when a memory application request of an application program is received, determining a target memory type allocated to the application program according to a preset memory allocation arbitration algorithm; if the target memory type is a dynamic random access (DRAM) memory, allocating the DRAM memory for the application program, and accessing the DRAM memory through a first access interface provided by a system function library GLIBC; and if the target memory type is a high-speed interconnection protocol CXL persistent memory, allocating the CXL persistent memory to the application program, and accessing the CXL persistent memory through a second access interface provided by a CXL persistent memory driver. By utilizing the method, the total memory capacity and bandwidth of the system are greatly expanded, the CXL persistent memory allocation is realized in the user space, the interaction frequency with the kernel is reduced, and the efficient allocation and scheduling of the CXL persistent memory in the Linux environment are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of memory management, and in particular to a memory allocation method, system, device and storage medium in a Linux environment. Background Art

[0002] The CXL (Compute Express Link) protocol is an open high-speed transmission protocol based on the PCI Express (PCIE) physical layer, designed to support efficient sharing of memory and accelerator resources within computing systems. CXL technology can expand memory, allowing the central processing unit (CPU) to access a larger memory pool on connected devices with low latency and high bandwidth, thereby increasing the memory capacity and performance of applications. Persistent memory is a special form of memory storage with the characteristics of persistence and non-lost data. Unlike traditional volatile memory, persistent memory is designed to store data for a long time and retain its content even after power failure or system shutdown. CXL-based persistent memory technology takes advantage of CXL's high-speed transmission. Persistent memory is directly connected to the processor through the memory bus like dynamic random access memory (DRAM). The processor can access persistent memory at byte granularity through memory instructions, and the access latency is very close to DRAM.

[0003] In the Linux environment, there are three types of memory allocators: physical page allocator, kernel heap allocator, and user space allocator. The first type of allocator is directly related to the hardware, generally completed by the driver, and runs in the kernel space. The second type is used when the kernel program uses memory, which is also in kernel state and has no direct relationship with the physical hardware, so it is universal. The third type is used by applications, which is in user state and is the main way for applications to use memory. In view of the characteristics of CXL persistent memory: large latency, large capacity, complex transmission mechanism, etc., how to improve the performance of user space allocators is an urgent problem to be solved by this type of memory allocator. Summary of the invention

[0004] The present invention provides a memory allocation method, system, device and storage medium in a Linux environment, which realizes CXL persistent memory allocation in user space, reduces the interaction frequency with the kernel, and realizes efficient allocation and scheduling of CXL persistent memory in a Linux environment.

[0005] In a first aspect, an embodiment of the present invention provides a memory allocation method in a Linux environment, which is executed by a CXL persistent memory pool allocator in a memory allocation system in a Linux environment, wherein the memory allocation system in the Linux environment includes a hardware layer and a software layer, wherein the hardware layer includes at least one DRAM memory connected to a central processing unit through a memory controller, and at least one CXL persistent memory connected to the central processing unit through a setting interface, and the software layer includes a CXL persistent memory driver located in a Linux kernel space, a CXL persistent memory pool allocator located in a Linux user space, and at least one application program, wherein the method includes:

[0006] When receiving a memory application request from the application, determining a target memory type to be allocated to the application according to a preset memory allocation arbitration algorithm;

[0007] If the target memory type is a dynamic random access DRAM memory, the DRAM memory is allocated to the application, and the DRAM memory is accessed through a first access interface provided by the system function library GLIBC;

[0008] If the target memory type is a high-speed interconnect protocol CXL persistent memory, the CXL persistent memory is allocated to the application, and the CXL persistent memory is accessed through a second access interface provided by the CXL persistent memory driver.

[0009] In a second aspect, an embodiment of the present invention provides a memory allocation system in a Linux environment, comprising: a hardware layer and a software layer;

[0010] The hardware layer includes at least one DRAM memory connected to the central processor via a memory controller, and at least one CXL persistent memory connected to the central processor via a setting interface;

[0011] The software layer includes a CXL persistent memory driver located in the Linux kernel space, a CXL persistent memory pool allocator located in the Linux user space, and at least one application. The CXL persistent memory allocator is used to execute the memory allocation method in the Linux environment as described in the first aspect embodiment.

[0012] In a third aspect, an embodiment of the present invention further provides an electronic device, including:

[0013] at least one processor; and

[0014] a memory communicatively coupled to the at least one processor;

[0015] Wherein, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the memory allocation method in the Linux environment as described in the embodiment of the first aspect.

[0016] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the memory allocation method in a Linux environment as described in the embodiment of the first aspect.

[0017] An embodiment of the present invention provides a memory allocation method, system, device and storage medium in a Linux environment. The method is executed by a CXL persistent memory pool allocator in a memory allocation system in a Linux environment. The memory allocation system in a Linux environment includes a CXL persistent memory driver located in a Linux kernel space, a CXL persistent memory pool allocator located in a Linux user space and at least one application. The method includes: first, when a memory application request of the application is received, according to a preset memory allocation arbitration algorithm, determining a target memory type allocated to the application; then, if the target memory type is a dynamic random access DRAM memory, allocating the DRAM memory to the application, and accessing the DRAM memory through a first access interface provided by the CXL persistent memory driver; if the target memory type is a high-speed interconnect protocol CXL persistent memory, allocating the CXL persistent memory to the application, and accessing the CXL persistent memory through a second access interface provided by the CXL persistent memory driver. The above technical solution enables multi-channel high-speed interaction between CXL persistent memory and CPU. Multiple PCIe high-speed channels can integrate multiple CXL persistent memories, greatly expanding the total memory capacity and bandwidth of the system. At the same time, applications can directly access CXL persistent memory, bypassing the file system and memory-mapped files, which can improve more efficient access. In addition, CXL persistent memory allocation is implemented in user space to reduce the frequency of interaction with the kernel, thereby achieving efficient allocation and scheduling of CXL persistent memory in the Linux environment.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flowchart of a memory allocation method in a Linux environment provided by Embodiment 1 of the present invention;

[0021] Figure 2 A schematic diagram of the spatial layout of the CXL persistent memory provided in the first embodiment of the present invention;

[0022] Figure 3 A schematic diagram of the structure of a persistent memory device linked list provided in Embodiment 1 of the present invention;

[0023] Figure 4 A schematic diagram of the structure of a persistent memory block allocation linked list provided in Embodiment 1 of the present invention;

[0024] Figure 5 A flowchart of another memory allocation method in a Linux environment provided by Embodiment 2 of the present invention;

[0025] Figure 6 An example flow chart of the step of determining the target memory type allocated to an application program in a certain application scenario provided in the second embodiment of the present invention;

[0026] Figure 7 This is an example flow chart of a single data block allocation algorithm provided in Embodiment 2 of the present invention;

[0027] Figure 8 This is an example flow chart of a multi-data block allocation algorithm provided in Embodiment 2 of the present invention;

[0028] Fig. 9 A schematic diagram of the structure of a memory allocation system in a Linux environment provided by Embodiment 3 of the present invention;

[0029] Fig.10 A schematic diagram of the structure of a CXL persistent memory allocator provided in Embodiment 3 of the present invention;

[0030] Fig.11 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] It should be clear that the CXL protocol is an open high-speed transmission protocol based on the PCIE physical layer, which is designed to support efficient sharing of memory and accelerator resources within computing systems. One of the design goals of CXL is to provide more flexible and higher-bandwidth connections between processors, storage, and accelerators. CXL allows different devices (such as processors, accelerators, etc.) to share memory directly without going through the host memory. This memory sharing capability helps improve system performance and reduce data transmission latency. It provides high-bandwidth data transmission and supports multi-channel and high-speed connections to meet the needs of large-scale data transmission and processing.

[0034] CXL technology can expand memory, allowing the CPU to access a larger pool of memory on connected devices with low latency and high bandwidth, thereby increasing the memory capacity and performance of applications. Persistent memory is a special form of memory storage that is persistent and hard to lose data. Unlike traditional volatile memory, persistent memory is designed to store data for a long time and retain its contents even after power failure or system shutdown. CXL-based persistent memory technology takes advantage of CXL's high-speed transmission. Persistent memory is directly connected to the processor through the memory bus like DRAM. The processor can access persistent memory at byte granularity through memory instructions, and the access latency is very close to DRAM.

[0035] In the Linux environment, there are three types of memory allocators. The first type is the physical page allocator, which is based on pages. A page is usually 4KB. The virtual memory subsystem calls the physical page memory allocator to map a new virtual page to the physical page. The second type is the kernel heap allocator. When the kernel runs out of allocated memory, it uses this allocator to allocate new memory to the kernel. The third type is the user space allocator, which is used by the application to allocate the memory required by the application. The first type of allocator is directly related to the hardware, generally completed by the driver, and runs in the kernel space. Its advantage is universality. The second type is used when the kernel program uses memory, which is also in kernel mode and has no direct relationship with physical hardware, so it is universal. The third type is used by applications, which is in user mode and is the main way for applications to use memory. How to improve the efficiency of user space allocators is an urgent problem to be solved for this type of memory allocator.

[0036] Embodiment 1

[0037] Figure 1 A flow chart of a memory allocation method in a Linux environment is provided for the first embodiment of the present invention. The method can be applied to the situation of efficiently allocating and scheduling CXL persistent memory in a Linux environment. The method can be executed by a CXL persistent memory pool allocator in a memory allocation system in a Linux environment. The system can be implemented in the form of hardware and / or software and can be configured in an electronic device.

[0038] like Figure 1 As shown, the memory allocation method in the Linux environment provided in the first embodiment may specifically include the following steps:

[0039] S101. When a memory application request from an application is received, a target memory type to be allocated to the application is determined according to a preset memory allocation arbitration algorithm.

[0040] It should be clear that the memory allocation system in the Linux environment includes a hardware layer and a software layer. The hardware layer includes at least one DRAM memory (i.e., DDR memory) connected to the central processor through a memory controller, and at least one CXL persistent memory connected to the central processor through a setting interface. The software layer includes a CXL persistent memory driver located in the Linux kernel space, a CXL persistent memory pool allocator located in the Linux user space, and at least one application. Among them, the setting interface is a CXL / PCIe interface.

[0041] In this embodiment, the CXL persistent memory pool allocator is the core component for implementing the CXL persistent memory pooling and efficient allocation and scheduling method. It is responsible for managing multiple CXL persistent memories, implementing standard C language function library (glibc) memory functions such as malloc, free, realloc, calloc, etc., maintaining binary compatibility with applications, and applications use the same interface as traditional DRAM to access CXL persistent memory, avoiding repeated development of applications for CXL persistent memory. In addition, it is also responsible for providing DRAM memory allocation functions to applications, directly quoting the standard glibc malloc memory management algorithm. It is also responsible for determining under what conditions DRAM memory is allocated to applications, and under what conditions CXL persistent memory is allocated to applications. The CXL persistent memory driver provides the CXL persistent memory access function interface to the CXL persistent memory pool allocator, and multiple applications use the CXL persistent memory pool allocator to apply for memory.

[0042] In this embodiment, when a memory application request is received from an application, a multi-factor memory allocation arbitration algorithm is used to decide whether to allocate DRAM memory or CXL persistent memory to the application. The memory allocation arbitration algorithm distinguishes between light-load application scenarios and heavy-load application scenarios. In light-load application scenarios, the application has a small amount of memory usage requirements, and the proportion of memory used to the total memory is not high; in heavy-load application scenarios, the application has a large amount of memory usage requirements, and the proportion of memory used to the total memory is high.

[0043] Exemplarily, the idle ratio of the DRAM memory is determined based on the remaining unused idle amount of the DRAM memory at the current moment and the total capacity of the DRAM memory, and the idle ratio is compared with the set threshold value to determine whether the current application scenario is a light-load application scenario or a heavy-load application scenario. Furthermore, different strategies are used for different application scenarios to determine which type of memory is allocated to the application. For light-load application scenarios, a performance priority strategy can be adopted to determine the allocation strategy based on the size of the memory requested by the application. For heavy-load application scenarios, a balanced priority strategy can be adopted to allocate DRAM memory or CXL persistent memory in proportion. The memory type determined to allocate memory to the application is recorded as the target memory type.

[0044] S102: If the target memory type is dynamic random access DRAM memory, allocate DRAM memory for the application, and access the DRAM memory through a first access interface provided by the system function library GLIBC.

[0045] In this embodiment, if the target memory type is DRAM memory, the remaining free memory in the DRAM memory will be allocated to the application according to the memory usage requirements of the application. Specifically, memory is allocated to the application according to the traditional DRAM memory allocation strategy, and no specific restrictions are made here. The hardware layer of the memory allocation system under the Linux environment includes multiple DRAM memories connected to the central processing unit through the memory controller, and provides a DRAM access function interface to the CXL persistent memory pool allocator through GLIBC, which is recorded as the first access interface. GLIBC is the libc library released by GNU, that is, the c runtime library. GLIBC is the lowest-level API in the Linux system, and almost any other runtime library will rely on GLIBC.

[0046] S103: If the target memory type is a high-speed interconnect protocol CXL persistent memory, allocate the CXL persistent memory to the application, and access the CXL persistent memory through a second access interface provided by the CXL persistent memory driver.

[0047] In this embodiment, if the target memory type is CXL persistent memory, the CXL persistent memory will be allocated to the application. By implementing CXL persistent memory allocation in user space, the memory released by the application can be reused without interacting with the kernel every time, and the CXL persistent memory can be efficiently allocated in the user space. It can also support the allocation of multiple continuous persistent memory data blocks to the application. According to the memory usage requirements of the application and the block size of the CXL persistent memory data block, it can be calculated how many data blocks need to be allocated to the application in order to meet the application. For example, the number of CXL persistent memory data blocks allocated to the application may be one or more.

[0048] Continuing with the above description, after determining the number of CXL persistent memory data blocks required, you can traverse the persistent memory device list and the persistent memory block allocation list to determine whether there is a persistent memory block allocation list, whether there are released CXL persistent memory data blocks, and whether the number of consecutive released data blocks meets the memory usage requirements of the application. If such a persistent memory block allocation list currently exists, these released data blocks are allocated to the application. If such a persistent memory block allocation list does not currently exist, new data blocks need to be allocated to the application from the CXL persistent memory pool.

[0049] It should be noted that the hardware layer of the memory allocation system in the Linux environment includes at least one CXL persistent memory connected to the central processing unit through a set interface, and the CXL persistent memory driver provides an access interface of the CXL persistent memory to the CXL persistent memory pool allocator, and this interface is recorded as the second access interface.

[0050] The embodiment of the present invention provides a memory allocation method in a Linux environment, which enables multi-channel high-speed interaction between CXL persistent memory and CPU. Multiple PCIe high-speed channels can integrate multiple CXL persistent memories, greatly expanding the total memory capacity and bandwidth of the system; at the same time, applications can directly access CXL persistent memory, bypassing the file system and memory mapping files, and can improve more efficient access; in addition, CXL persistent memory allocation is implemented in user space, reducing the frequency of interaction with the kernel, and realizing efficient allocation and scheduling of CXL persistent memory in a Linux environment.

[0051] As an optional embodiment of the embodiment of the present invention, based on the above embodiment, the method can be optimized to further include: initializing the spatial layout of the CXL persistent memory and building a CXL persistent memory pool before receiving the memory application request of the application.

[0052] In this embodiment, the CXL persistent memory pool allocator is initialized, and a streamlined and efficient CXL persistent memory space layout is designed, including a metadata (Meta) area and a data area divided into multiple data blocks (Chunks). The CXL persistent memory device is initialized to build a CXL persistent memory pool.

[0053] As a specific implementation manner, the steps of initializing the spatial layout of the CXL persistent memory may be optimized, including: dividing the CXL persistent memory into a metadata area and a data area.

[0054] The metadata area includes a magic number for identifying that the CXL persistent memory has completed initialization, a block size of a data block, a total number of data blocks, a number of free blocks of a data block, and a free state block, and the data area includes at least one data block.

[0055] In this embodiment, a streamlined and efficient CXL persistent memory space layout is designed, including a metadata (Meta) area and a data area divided into multiple data blocks (Chunks). The Meta area includes a 4-byte magic number (Magic_Number) for indicating that the CXL persistent memory has been initialized; a 4-byte block size (Chunk_Size), which is generally set to 16M to 128M; a 4-byte total number of blocks (Total_Chunks), Total_Chunks = (PMem capacity / Chunk_Size)-1, where the first data block is reserved for the metadata area; the number of free state blocks (Free_Chunks); the free state block of the data block, which identifies the free state (Free) of each data block, using 1 bit to indicate whether each data block is free, 0 for not free, 1 for free, and Free = 1 initially. This area occupies a total of Total_Chunks / 8 bytes, aligned to 4 bytes; followed by the data area of ​​N data blocks, N = Total_Chunks.

[0056] It should be noted that the size of the data block is related to the performance of CXL persistent memory access. Assume that the memory requested by the application is n bytes, and the CXL persistent memory access latency is k microseconds, that is, k x10 -6 s, PCIe transmission speed is pGB / s, that is, p x10 9 B / s, the bandwidth of the application accessing CXL persistent memory is BW (unit: B / s), then Assume access efficiency E = BW / PCIe transmission speed, then

[0057] According to research data, the access latency of CXL persistent memory is 0.5 to 10 microseconds, that is, k = 0.5 to 10, k = 1, and the one-way transmission speed of PCIe 5.0x16 is 64GB / s, that is, p = 64. According to the above formula, if the access efficiency E>95% is to be achieved, then n>1.2x 10 6 B is 1.2MB. According to the above analysis, Chunk Size>1.2MB is required, and the general value of Chunk Size is 2MB to 128MB.

[0058] For example, in order to more clearly illustrate the spatial layout of CXL persistent memory provided by the embodiment of the present invention, Figure 2 A schematic diagram of the spatial layout of the CXL persistent memory provided in the first embodiment of the present invention is shown in FIG. Figure 2As shown, in this embodiment, a streamlined and efficient persistent memory (PMem) 15 space layout is designed, including a Meta area 218 and a data area 219 divided into multiple Chunks. The Meta area 218 includes a 4-byte magic number (Magic_Number) 210, such as 0x12345678, which is used to indicate that the PMem 15 has completed initialization; a 4-byte block size (Chunk_Size) 211, which is generally set to 2M to 128M, such as 16M; a 4-byte total number of blocks (Total_Chunks) 212. During initialization, Total_Chunks = (PMem capacity / Chunk_Size) - 1. The first Chunk is reserved for the Meta area. For example, for a 512GB PMem, assuming Chunk Size=16MB, then Total_Chunks=32K-1; the number of free status blocks (Free_Chunks) 213; the free status block of Chunk 214, which identifies the free status Free of each Chunk, uses 1 bit to indicate whether each Chunk is free, 0 indicates not free, 1 indicates free, initially Free=1, this area occupies a total of Total_Chunks / 8 bytes, aligned to 4 bytes; next is the data area of ​​N Chunks 219, N=Total Chunks, including Chunk-1 data 215, Chunk-2 data 216, ..., Chunk-N data 217, for example, N=Total Chunks=32K-1, then there are Chunk-1 data, Chunk-2 data, ..., Chunk-32K-1 data.

[0059] The above technical solution initializes the spatial layout of CXL persistent memory.

[0060] As a specific implementation method, the steps of building a CXL persistent memory pool can be optimized, including: building a persistent memory device linked list to form a CXL persistent memory pool.

[0061] The persistent memory device linked list is used to represent and manage at least one CXL persistent memory device, and the persistent memory device linked list includes the persistent memory device name, the total number of persistent memory device blocks, the number of persistent memory device free blocks, the persistent memory block allocation linked list, and the next list;

[0062] The persistent memory block allocation linked list is used to represent and manage the block allocation information of the CXL persistent memory device. The persistent memory block allocation linked list includes the pointer assigned after allocation, the number of allocated bytes, the number of occupied blocks, the starting block sequence number, the free flag, and the next list.

[0063] In this embodiment, a persistent memory device linked list (PMem_Dev_Linked_List) structure is designed to represent and manage multiple CXL persistent memory devices to form a CXL persistent memory pool. The persistent memory device linked list is a linked list structure, and its members include the persistent memory device name (PMem_Dev), the total number of persistent memory device blocks (Total_Chunks), the number of free blocks of the persistent memory device (Free_Chunks), the persistent memory block allocation linked list (PMem_Alloc_Linked_List), and the next list (Next).

[0064] Continuing with the above description, the persistent memory block allocation linked list is a linked list structure that represents and manages the block allocation information of the persistent memory device. The structure members include the pointer assigned after allocation (Ptr), the number of allocated bytes (Size), the number of occupied blocks (Chunk_Count), the starting block number (Chunk_Num_start), the free flag (Free, Free is 0 for not free, Free is 1 for free), and the next list (Next).

[0065] In order to more clearly describe the persistent memory device linked list and the persistent memory block allocation linked list provided in the embodiment of the present invention, exemplarily, Figure 3 A schematic diagram of the structure of a persistent memory device linked list provided in the first embodiment of the present invention, such as Figure 3 As shown, the persistent memory device linked list (PMem_Dev_Linked_List) 310 is a linked list structure, and the structure members include the persistent memory device name (PMem_Dev) 311, the total number of persistent memory device blocks (Total_Chunks) 312, the number of free blocks of the persistent memory device (Free_Chunks) 313, the persistent memory block allocation linked list (PMem_Alloc_Linked_List) 314, and the next list (Next) 315. Assuming that there are three CXL persistent memory devices with a capacity of 512GB each, Chunk_Size = 16MB: PMem_Dev-1, PMem_Dev-2, PMem_Dev-3, there are three persistent memory device linked lists: PMem_Dev_Linked_List-1, PMem_Dev_Linked_List-2, PMem_Dev_Linked_List-3.

[0066] The structure of PMem_Dev_Linked_List-1 is:

[0067] PMem_Dev=PMem_Dev-1, Total_Chunks=32K-1, Free_Chunks=32K-1, PMem_Alloc_Linked_List=NULL, Next=PMem_Dev_Linked_List-2.

[0068] The structure of PMem_Dev_Linked_List-2 is:

[0069] PMem_Dev=PMem_Dev-2, Total_Chunks=32K-1, Free_Chunks=32K-1, PMem_Alloc_Linked_List=NULL, Next=PMem_Dev_Linked_List-3.

[0070] The structure of PMem_Dev_Linked_List-3 is:

[0071] PMem_Dev=PMem_Dev-3, Total_Chunks=32K-1, Free_Chunks=32K-1, PMem_Alloc_Linked_List=NULL, Next=NULL.

[0072] For example, Figure 4 A schematic diagram of the structure of the persistent memory block allocation linked list provided in the first embodiment of the present invention, such as Figure 4 As shown, the persistent memory block allocation linked list (PMem_Alloc_Linked_List) 314 includes a pointer (Ptr) 317 assigned after allocation, the number of bytes allocated (Size) 318, the number of blocks occupied (Chunk_Count) 319, the starting block sequence number (Chunk_Num_start) 320, the free flag (Free, Free is 0 for not free, Free is 1 for free) 321, and the next list (Next) 322. Assuming that the first Chunk block (Chunk_Size = 16MB) is allocated in the persistent memory device PMem_Dev-1, and the actual number of occupied bytes is 4MB, a persistent memory block allocation linked list PMem_Alloc_Linked_List-1 will be created, and its structure is: Ptr = Ptr-1, Size = 4*1024*1024, Chunk_Count = 1, Chunk_Num_start = 1, Free = 0, Next = NULL.

[0073] The above technical solution implements a CXL persistent memory pooling method, which can manage multiple CXL persistent memory devices to form a pool. The device direct access (DAX) mode and large memory data blocks are used as allocation units to efficiently manage the CXL persistent memory pool.

[0074] Embodiment 2

[0075] Figure 5 A flow chart of another memory allocation method in a Linux environment provided for Embodiment 2 of the present invention. This embodiment is a further optimization of the above embodiment. In this embodiment, "determining the target memory type allocated to the application according to the memory usage requirements and a preset memory allocation arbitration algorithm" is further limited and optimized, as well as "allocating the CXL persistent memory to the application" is limited and optimized.

[0076] like Figure 5 As shown, this embodiment 2 provides a memory allocation method in a Linux environment, which specifically includes the following steps:

[0077] S401. When a memory application request from an application is received, a first idle ratio of the DRAM memory is obtained by dividing the idle amount of the DRAM memory by the total capacity of the DRAM memory.

[0078] In this embodiment, the total capacity of the DRAM memory and the current remaining unused free space can be directly obtained, for example, by reading the / proc system to obtain the corresponding values, and the / proc system is a special virtual file system provided by the Linux kernel, which is mainly used to display kernel and process information. Further, the result of the quotient of the free space of the DRAM memory and the total capacity of the DRAM memory is defined as the free ratio of the DRAM memory, which is recorded as the first free ratio.

[0079] S402: Determine whether the current application scenario is a light-load application scenario or a heavy-load application scenario according to a first idle ratio of the DRAM memory and a set first ratio threshold.

[0080] In this embodiment, the current application scenario can be specifically understood as a scenario in which the application program requests memory allocation at the current moment. To determine whether the current application scenario is a light-load application scenario or a heavy-load application scenario, a threshold is used for control, and the threshold used to determine the application scenario is recorded as a first ratio threshold, which can be set according to historical experience values, for example, the first ratio threshold value is set to 70% or other experience values.

[0081] As a specific implementation manner, the step of determining whether the current application scenario is a light-load application scenario or a heavy-load application scenario according to the first idle ratio of the DRAM memory and the set first ratio threshold can be optimized, including:

[0082] a1) If the first idle ratio of the DRAM memory is greater than the first ratio threshold, it is determined that the current application scenario is a light-load application scenario.

[0083] In this embodiment, if the first idle ratio of the DRAM memory is greater than the first ratio threshold, the current application scenario is determined to be a light load application scenario. Continuing with the above example, if the first idle ratio of the DRAM memory is greater than 70%, it means that the current Linux system is a light load application scenario.

[0084] b1) If the first idle ratio of the DRAM memory is less than or equal to the first ratio threshold, it is determined that the current application scenario is a heavy-load application scenario.

[0085] In this embodiment, if the first idle ratio of the DRAM memory is less than or equal to the first ratio threshold, the current application scenario is determined to be a heavy load application scenario. Continuing with the above example, if the first idle ratio of the DRAM memory is <= 70%, it means that the current Linux system is a heavy load application scenario.

[0086] The above technical solution specifies how to determine whether the current application scenario is a light-load application scenario or a heavy-load application scenario, and provides a basis for the subsequent memory allocation strategy to be adopted.

[0087] S403: If the current application scenario is a light-load application scenario, determine the target memory type to be allocated to the application program according to a performance priority policy.

[0088] In this embodiment, for light-load application scenarios, the allocation strategy is determined according to the memory size requested by the application program according to the performance priority strategy, thereby determining which memory type to allocate to the application program as the target memory type. For example, it can be determined whether to allocate DRAM memory or CXL persistent memory to the application program according to the relationship between the memory usage requirement and the block size of the data block.

[0089] As a specific implementation method, the steps of determining the target memory type to be allocated to the application according to the performance priority policy can be optimized, including:

[0090] a2) If the memory usage requirement is greater than the set multiple of the block size, CXL persistent memory is used as the target memory type allocated to the application.

[0091] In this embodiment, a set multiple of the block size of a data block can be set as a threshold, and the memory usage requirement can be compared with the set multiple of the block size. If the memory usage requirement is greater than the set multiple of the block size, CXL persistent memory is used as the target memory type. Preferably, block size*1 / 4 is used as the threshold. For example, Chunk_Size=16MB, the judgment condition is whether the memory usage requirement is greater than 4MB. If it is judged to be yes, CXL persistent memory is used as the target memory type and CXL persistent memory is allocated to the application.

[0092] b2) If the memory usage requirement is less than or equal to the set multiple of the block size, DRAM memory is used as the target memory type allocated to the application.

[0093] In this embodiment, if the memory usage requirement is less than or equal to the set multiple of the block size, the traditional DRAM memory is used as the target memory type. Continuing with the above example, if it is determined that the memory usage requirement is less than or equal to 4MB, the DRAM memory is used as the target memory type and the DRAM memory is allocated to the application.

[0094] The above technical solution specifies how to determine which type of memory to allocate to the application based on a performance priority strategy in a light-load application scenario.

[0095] S404: If the current application scenario is a heavy-load application scenario, determine the target memory type to be allocated to the application program according to a balancing priority strategy.

[0096] In this embodiment, for heavy-load application scenarios, DRAM memory or CXL persistent memory is allocated proportionally according to a balance priority strategy, thereby determining which memory is allocated to the application as the target memory type.

[0097] As a specific implementation method, the steps of determining the target memory type to be allocated to the application according to the balance priority policy can be optimized, including:

[0098] a3) Taking the total capacity of the DRAM memory and the total capacity of the CXL persistent memory as the quotient, and using the quotient result as the second ratio threshold.

[0099] In this embodiment, the total capacity of the DRAM memory and the total capacity of the CXL persistent memory can obtain corresponding values ​​by reading the / proc system, and the total capacity of the DRAM memory is divided by the total capacity of the CXL persistent memory. The result of the quotient is defined as a threshold for determining which memory to allocate to the application, which is recorded as the second ratio threshold.

[0100] b3) Taking the quotient of the idle amount of the DRAM memory and the idle amount of the CXL persistent memory, a second idle ratio of the DRAM memory to the CXL persistent memory is obtained.

[0101] In this embodiment, the free amount of DRAM memory can be obtained by reading the / proc system to obtain the corresponding value, and the free amount of CXL persistent memory can be obtained by the number of free blocks (Free_Chunks) of the persistent memory device in the persistent memory block allocation linked list (PMem_Dev_Linked_List) structure. The free amount of DRAM memory is divided by the free amount of CXL persistent memory to obtain the free ratio of the two, which is recorded as the second free ratio, and memory is subsequently allocated to the application based on the second free ratio.

[0102] c3) If the second idle ratio is less than the second ratio threshold, CXL persistent memory is used as the target memory type allocated to the application.

[0103] In this embodiment, if the second idle ratio is less than the second ratio threshold, it means that the idle amount of DRAM memory accounts for too small a proportion, and more CXL persistent memory is needed, so CXL persistent memory is used as the target memory type allocated to the application. For example, for a Linux system with a total DRAM capacity of 64GB and a total CXL persistent memory (PMem) capacity of 512GB, the second ratio threshold = 12.5%, and the memory allocation of the application is performed at a ratio of DRAM: PMem = 1:8. Determine whether the second idle ratio is less than 12.5%. If the judgment result is yes, it means that the DRAM memory allocation ratio is relatively small, and DRAM memory allocation is required.

[0104] d3) If the second idle ratio is greater than or equal to the second ratio threshold, the DRAM memory is used as the target memory type allocated to the application.

[0105] In this embodiment, if the second idle ratio is greater than or equal to the second ratio threshold, it means that the idle amount of DRAM memory accounts for too large a proportion, and more DRAM memory needs to be used. Therefore, DRAM memory is used as the target memory type allocated to the application. Continuing with the above example, it is determined whether the second idle ratio is less than 12.5%. If the determination result is no, it means that the CXL persistent memory allocation ratio is relatively small, and DRAM memory needs to be allocated for the application.

[0106] The above technical solution specifies how to determine which type of memory to allocate to the application based on a balanced priority strategy in a heavy-load application scenario.

[0107] In order to more clearly describe how to determine which memory to allocate to an application in the memory allocation method under the Linux environment provided by the present invention, a certain application scenario is used as an example for explanation. Figure 6 This is a flowchart of a step of determining a target memory type to be allocated to an application program in a certain application scenario provided by the second embodiment of the present invention, such as Figure 6 As shown in the figure, the steps to determine the target memory type to be allocated to the application include:

[0108] S510: Taking the idle amount of the DRAM memory as a quotient of the total capacity of the DRAM memory, a first idle ratio of the DRAM memory is obtained.

[0109] S511. Determine whether the first idle ratio of the DRAM memory is greater than the first ratio threshold. If so, determine that the current application scenario is a light-load application scenario, and continue to execute step S512; if not, determine that the current application scenario is a heavy-load application scenario, and continue to execute S513-S515.

[0110] S512. Determine whether the memory usage requirement is greater than the block size*1 / 4. If so, execute step S517; otherwise, execute step S516.

[0111] S513 . Calculate the quotient of the total capacity of the DRAM memory and the total capacity of the CXL persistent memory, and use the quotient result as the second ratio threshold.

[0112] S514: Calculate the quotient of the idle amount of the DRAM memory and the idle amount of the CXL persistent memory to obtain a second idle ratio of the DRAM memory to the CXL persistent memory.

[0113] S515: Determine whether the second idle ratio is less than the second ratio threshold, if so, execute step S517, if not, execute step S516.

[0114] S516: Use DRAM memory as the target memory type allocated to the application.

[0115] S517. Use CXL persistent memory as the target memory type allocated to the application.

[0116] The above are the steps to determine the target memory type allocated for the application.

[0117] S405: If the target memory type is dynamic random access DRAM memory, allocate DRAM memory to the application, and access the DRAM memory through a first access interface provided by the CXL persistent memory driver.

[0118] In this embodiment, if the target memory type is DRAM memory, the function of allocating DRAM memory to the application program can directly refer to the standard glibc malloc memory management algorithm.

[0119] S406: If the target memory type is a high-speed interconnect protocol CXL persistent memory, determine the number of data blocks required to be allocated by the application program according to the memory usage requirement and the block size of the data block.

[0120] In this embodiment, if the target memory type is CXL persistent memory, CXL persistent memory needs to be allocated to the application. On the premise that the size of the application's memory usage requirement and the block size of the data block of the CXL persistent memory are known, the memory usage requirement and the block size of the data block can be compared. If the memory usage requirement is less than or equal to the block size of the data block, only one data block needs to be allocated to the application; if the memory usage requirement is greater than the block size of the data block, multiple data blocks need to be allocated to the application. The specific number of data blocks that need to be allocated to the application can be determined by using the method of memory usage requirement / block size of the data block and rounding up. For example, if the memory usage requirement is 66MB and the block size is 16MB, the number of data blocks that the application needs to allocate is 5.

[0121] S407 , traverse each persistent memory block allocation linked list and each persistent memory block allocation linked list in sequence.

[0122] In this embodiment, the allocation function (malloc) first updates the persistent memory block allocation linked list to obtain the latest free data block of each persistent memory device, and traverses the persistent memory allocation linked list through the persistent memory block allocation linked list.

[0123] S408: Determine whether there is a target persistent memory block allocation linked list whose free flag is free and the number of consecutively occupied blocks is the number of data blocks required to be allocated.

[0124] This step is used to determine whether there is a persistent memory block allocation linked list, whose free flag = 1 && the number of occupied blocks = the number of data blocks required to be allocated. If it exists, it will be recorded as the target persistent memory block allocation list, set to ChunkList.

[0125] S409: If it exists, allocate the released data block in the target persistent memory block allocation list to the application program, set the free flag in the target persistent memory block allocation list to not free, and return the memory pointer.

[0126] In this embodiment, if there is a target persistent memory block allocation linked list, the data blocks of the persistent memory that have been released in the target persistent memory allocation linked list are reused, and the free flag in the target persistent memory block allocation linked list is set to not free, and the memory pointer is returned.

[0127] Exemplarily, for a data block, the persistent memory block allocation linked list is traversed through PMem_Dev_Linked_List, and if the first allocated block with free flag Free=1 and Chunk_Count=1 is found, the memory pointer Ptr is directly returned.

[0128] For the case where multiple data blocks are needed, assume the number of data blocks to be allocated, traverse the CXL persistent memory block allocation list, and then traverse the persistent memory block allocation list, and then determine whether there is a persistent memory block allocation list, set as ChunkList, ChunkList->Free=1&&ChunkList->Chunk_Count->=C? If it exists, execute the mmap system call to allocate C consecutive Chunks of CXL persistent memory at the hardware layer, and assign the Point memory pointer.

[0129] It is important to know that by implementing CXL persistent memory allocation in user space and reusing the memory released by the application, there is no need to interact with the kernel every time. This can achieve efficient allocation of CXL persistent memory in user space and support the allocation of multiple continuous persistent memory data blocks to the application.

[0130] S410 : Otherwise, allocate a data block for the application from the CXL persistent memory pool.

[0131] Specifically, if there is no target persistent memory block allocation linked list, that is, the free flags of all persistent memory block allocation linked lists = 0, that is, there is no released persistent memory block, then a new persistent memory data block needs to be allocated from the CXL persistent memory device.

[0132] As a specific implementation method, the steps of allocating data blocks from the CXL persistent memory pool to the application can be optimized, including:

[0133] a4) Traverse each persistent memory block allocation linked list to determine the target persistent memory device with the most free data blocks.

[0134] In this embodiment, the process of allocating new persistent memory data blocks needs to be locked to ensure the atomicity when multiple applications concurrently allocate persistent memory. The process is: first traverse the persistent memory block allocation linked list to find the persistent memory device with the most free data blocks, which is recorded as the target persistent memory device.

[0135] b4) Load the metadata area of ​​the target persistent memory device.

[0136] Then, the metadata area of ​​the target persistent memory device is loaded.

[0137] c4) Find the free state blocks of the target persistent memory device and determine whether there are free data blocks with free marks of the number of consecutive data blocks required for allocation.

[0138] In this embodiment, the idle state blocks in the data blocks are searched, and the first data block with the idle flag=1 is located and set as Chunki (i is the number of the data block).

[0139] The parameter of the release function (free) is set to Pointer. First, the persistent memory block allocation list is traversed through the persistent memory device list. It should be noted that the release function only marks the released CXL persistent memory as idle, and does not actually release the physical CXL persistent memory. This is because the application will continuously apply for and release memory. Reusing the released persistent memory will greatly reduce the overhead of reapplying for physical CXL persistent memory. Reusing the released CXL persistent memory can achieve more efficient CXL persistent memory allocation.

[0140] d4) If it exists, call to allocate CXL persistent memory at the hardware layer and assign the memory pointer.

[0141] In this embodiment, if a list of memory pointer (Ptr) = Pointer and free flag (Free) = 0 is matched, a kernel function (mmap) system call is executed, the Ptr memory pointer is assigned, and Ptr = NULL and Free = 1 are set.

[0142] e4) Create a new persistent memory block allocation linked list, and set the newly created persistent memory block allocation linked list.

[0143] In this embodiment, a persistent memory block allocation linked list is created, and the memory pointer, block size, number of occupied blocks, starting number of data blocks, free state, and next list are set. For example, Ptr, Size = Allocation memory size, Chunk_Count = 1, Chunk_Num_start = i, Free = 0, and Next = NULL are set.

[0144] f4) Assign the newly created persistent memory classification linked list to the next list in the persistent memory block allocation linked list corresponding to the target persistent memory device, and update the number of persistent memory device free blocks of the target persistent memory device minus the number of data blocks required for allocation.

[0145] In this embodiment, the newly created persistent memory classification list is added to the next list in the persistent memory block allocation linked list corresponding to the target persistent memory device, and the number of free blocks of the persistent memory device of the target persistent memory device is set minus the number of allocated data blocks required. Exemplarily, Free_Chunks is set to Free_Chunks-5.

[0146] g4) Update the metadata area of ​​the persistent memory device, update the number of free blocks of the persistent memory device of the target persistent memory device minus the number of data blocks required to be allocated, and update the free status of the data blocks.

[0147] In this embodiment, the metadata area of ​​the persistent memory device is updated last, Free_Chunks=Free_Chunks-5, and the Free flag of the data block numbered i in the free state block of the data block of the target persistent memory device is set to 0.

[0148] h4) Returns the memory pointer.

[0149] Finally, the Point memory pointer.

[0150] i4) If it does not exist, an error is returned.

[0151] In this embodiment, if there is no match, Error is returned.

[0152] Exemplarily, the execution steps of the allocation algorithm for multiple data blocks can be described as follows: if the memory usage requirement > block size, multiple data blocks need to be allocated, and the number of allocated data blocks is set to C = memory usage requirement / block size, rounded up. First, traverse the persistent memory block allocation list through the persistent memory device list. If the first allocated block with the free flag = 1 and the number of blocks >= C is found, the memory pointer is directly returned. If there is no released persistent memory block that meets the conditions, then multiple new continuous persistent memory blocks need to be allocated from the CXL persistent memory device. Allocating multiple new continuous persistent memory blocks requires locking to ensure the atomicity when multiple applications concurrently allocate persistent memory. The process is: first traverse the persistent memory device list to find the persistent memory device with the most free blocks, then load the metadata area of ​​the persistent memory device, find the free state block of the data block, locate the first C consecutive data blocks with free state = 1 and set them as Chunki~i+C-1 (i is the starting number of the data block), execute the mmap system call, assign the memory pointer, append a list to the persistent memory device list of the persistent memory device, set the memory pointer, size = memory usage requirement, total number of blocks = C, number of starting block = i, free state = 0, next list = NULL, and finally update the metadata area of ​​the persistent memory device, the number of free blocks = the number of original free blocks - C, and set Free = 0 for the data blocks numbered i~i+C-1 in the free state block of the data block.

[0153] For example, in order to more clearly describe the allocation algorithm of a single data block, Figure 7 This is a flow chart of an example of a single data block allocation algorithm provided in the second embodiment of the present invention, such as Figure 7As shown, the execution process of the single data block allocation algorithm 610 can be described as follows: S611, traverse the CXL persistent memory device linked list (PMem_Dev_Linked_List). S612, traverse the persistent memory allocation linked list (PMem_Alloc_Linked_List) in the persistent memory device linked list. S613, determine whether there is a persistent memory allocation linked list, set as ChunkList, whose free state (Free) = 1 && number of blocks (Chunk_Count) = 1? S614, if it exists, you can reuse this released persistent memory data block (Chunk), set ChunkList->free state (Free) = 1, and return ChunkList->memory pointer (Ptr). Otherwise, go to S615 and allocate memory from the CXL persistent memory device pool. S615 traverses the persistent memory device linked list to find the CXL persistent memory device with the most free data blocks (Free_Chunks), set as PMem_Dev. S616, load the metadata (Meta) area of ​​PMem_Dev. S617. Search for the free state block of the data block (Chunk) of PMem_Dev. S618. Determine whether there is a data block (Chunk) with free state (Free) = 1 in the free state block of the data block (Chunk), numbered i? If found, go to step S619, otherwise go to step S622 and return Error. S619. Execute the mmap system call to allocate CXL persistent memory at the hardware layer and assign the Point memory pointer. Next S620. Create a persistent memory allocation linked list (PMem_Alloc_Linked_List), set it to PMem_Alloc, set its memory pointer (Ptr) = Point, size (Size) = memory usage requirement (Allocation memory size), number of occupied blocks (Chunk_Count) = 1, block starting number (Chunk_Num_start) = i, free state (Free) = 0, next list (Next) = NULL. S621. Assign the created PMem_Alloc to the persistent memory allocation linked list->next list (PMem_Alloc_Linked_List->Next) of PMem Dev found in S615, and set the free blocks (Free_Chunks) of PMem_Dev = the number of original free blocks (Free_Chunks)-1.Then update the metadata (Meta) area of ​​the CXL persistent memory device represented by PMem_Dev, set the free blocks (Free_Chunks) = the original number of free blocks (Free_Chunks) - 1, set the free state (Free) of the data block (Chunk) numbered i in the free state block of the data block (Chunk) = 0, and finally S624, return the Point memory pointer.

[0154] Exemplarily, in order to more clearly describe the allocation algorithm of multiple data blocks, Figure 8 This is a flowchart of a multi-data block allocation algorithm provided in Embodiment 2 of the present invention, such as Figure 8As shown, the execution process of the allocation algorithm 710 for multiple data blocks can be described as follows: S711, calculate C = memory usage requirement (Allocation memory size) / block size (Chunk size), for example, Allocation memory size = 66MB, Chunk size = 16MB, then C = 5. S712, traverse the CXL persistent memory device linked list (PMem_Dev_Linked_List). Then S713, traverse the persistent memory allocation linked list (PMem_Alloc_Linked_List). Then S714, determine whether there is a persistent memory allocation linked list, set to ChunkList, ChunkList->Free state (Free) = 1 && ChunkList->Total number of blocks (Chunk_Count)-> = 5? If it exists, execute step S718, set ChunkList>Free state (Free) = 0, return ChunkList->Memory pointer (Ptr), otherwise go to S715. S715, traverse the persistent memory device linked list to find the persistent memory device (PMem_Dev) with the most free blocks (Free_Chunks). Next, execute S716 to load the metadata (Meta) area of ​​PMem_Dev. Next, execute S717 to find the free state block of the data block (Chunk) of PMem_Dev, and then S719 to determine whether there are C consecutive Chunks with Free==1, starting number i? If not, execute step S720 to return Error, otherwise go to step 721. S721, execute the mmap system call to allocate C consecutive data blocks (Chunk) of CXL persistent memory at the hardware layer, assign the Point memory pointer, and then execute S722 to create a persistent memory block allocation linked list, set the memory pointer (Ptr) = Point, size (Size) = 66MB, number of blocks (Chunk Count) = C, starting block number (Chunk_Num_start) = i, free state (Free) = 0, next list (Next) = NULL. Next, execute S723, attach the created persistent memory block allocation list to the persistent memory device list, and set its free data block (Free_Chunks) = original free data block (Free_Chunks) - C. Then execute S724, update the metadata (Meta) area of ​​PMem_Dev, set free data block (Free_Chunks) = original free data block (Free_Chunks) - C, set Free = 0 for the Chunks numbered i to i+4 in the free state block of the data block (Chunk), and finally execute step 725 to return Pointer.

[0155] S411. Access the CXL persistent memory through a second access interface provided by the CXL persistent memory driver.

[0156] The above technical solution takes into account the characteristics of CXL persistent memory. First, the latency of CXL persistent memory is higher than that of DRAM, and large memory blocks are required as allocation units. Second, CXL persistent memory runs in DAX mode, and applications can directly access CXL persistent memory, bypassing the file system and memory-mapped files, which can improve more efficient access. It is usually accessed through kernel function (mmap) system calls. In addition, CXL persistent memory allocation implemented in user space does not need to interact with the kernel every time, and efficient allocation of CXL persistent memory can be achieved in user space. In order to balance the use of DRAM and CXL persistent memory to achieve high efficiency, an arbitration method is adopted. For small-sized memory applications, traditional DRAM memory is preferred, and for large-sized memory applications, CXL persistent memory is preferred. When allocating CXL persistent memory, full consideration is also given to how to reuse released memory to reduce the number of mmap system calls, thereby further improving performance.

[0157] Embodiment 3

[0158] Fig. 9 The present invention provides a schematic diagram of a memory allocation system in a Linux environment according to a third embodiment of the present invention. The device can be used to efficiently allocate and schedule CXL persistent memory in a Linux environment. The memory allocation system in a Linux environment can be configured in an electronic device, such as Fig. 9 As shown, the structure of the memory allocation system in the Linux environment includes a hardware layer 10 and a software layer 16. The hardware layer 10 includes multiple DRAM memories (denoted as DDR in the figure) 11 connected to the CPU 13 through the memory controller 12, and multiple CXL persistent memories (denoted as PMem in the figure) 15 connected to the CPU 13 through CXL / PCIe 14. The software layer 16 includes a CXL persistent memory driver 19 (PMem CXL Driver) located in the Linux kernel space 17 (Kernel Space), a CXL persistent memory pool allocator 20 located in the Linux user space 18, and multiple applications 21. The CXL persistent memory driver 19 provides an access function interface of the CXL persistent memory 15 to the CXL persistent memory pool allocator 20. Multiple applications 21 use the CXL persistent memory pool allocator 20 to apply for memory. The CXL persistent memory allocator is used to execute the memory allocation method in the Linux environment as described in any of the above embodiments.

[0159] In order to more clearly describe the structure and working principle of the CXL persistent memory allocator, for example, Fig.10A schematic diagram of the structure of the CXL persistent memory allocator provided in the third embodiment of the present invention is shown in FIG. Fig.10 As shown, the CXL persistent memory pool allocator 20 (PMPAllocator) includes an initialization module 110, an allocation strategy arbitrator module 113, a CXL persistent memory allocation algorithm module 115, and a traditional DRAM allocation algorithm module 114. The initialization module 110 is responsible for the CXL persistent memory device (PMem Device) initialization 111 and the PMPAllocator 20 initialization 112. The PMPAllocator--CXL persistent memory allocation algorithm module 115 is responsible for managing multiple CXL persistent memories (PMem) 15, implementing standard glibc memory functions such as malloc 116, free 117, realloc 118, calloc 119, and maintaining binary compatibility with the application 21 program. The traditional DRAM allocation algorithm 114 module is responsible for providing the application 21 program with the allocation function of DRAM (DDR memory) 11, and directly references the standard glibc malloc memory management algorithm. The allocation strategy arbiter 113 module decides under what conditions to call the traditional DRAM allocation algorithm module 114 to allocate DRAM memory (DDR) 11 to the application, and under what conditions to call the CXL persistent memory allocation algorithm module 115 to allocate CXL persistent memory (PMem) 15 to the application. The CXL persistent memory pool allocator 20 (PMPAllocator) implements the pooling of CXL persistent memory (PMem) 15, and balances the allocation of DRAM memory (DDR 11) and CXL persistent memory (PMem) 15, achieving a balance in memory performance and memory capacity, and realizing efficient memory allocation and scheduling for the application (App) 21.

[0160] The memory allocation system under the Linux environment provided by the embodiment of the present invention has the functional modules and beneficial effects corresponding to the execution method.

[0161] Embodiment 4

[0162] Fig.11 A schematic diagram of the structure of an electronic device provided for Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as desktop computers, servers, smart phones, etc. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0163] like Fig.11As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0164] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0165] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as a memory allocation method in a Linux environment.

[0166] In some embodiments, the memory allocation method under the Linux environment can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the memory allocation method under the Linux environment described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to execute the memory allocation method under the Linux environment by any other appropriate means (for example, by means of firmware).

[0167] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0168] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0169] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0170] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0171] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0172] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0173] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements a memory allocation method in a Linux environment as provided in any embodiment of the present invention.

[0174] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code for the operation of the present disclosure, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0175] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0176] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A memory allocation method in a Linux environment, characterized in that: The method is executed by a CXL persistent memory pool allocator in a memory allocation system under a Linux environment, wherein the memory allocation system under the Linux environment includes a hardware layer and a software layer, wherein the hardware layer includes at least one DRAM memory connected to a central processing unit through a memory controller, and at least one CXL persistent memory connected to the central processing unit through a setting interface, and the software layer includes a CXL persistent memory driver located in a Linux kernel space, a CXL persistent memory pool allocator located in a Linux user space, and at least one application program, and the method includes: When receiving a memory application request from the application, determining a target memory type to be allocated to the application according to a preset memory allocation arbitration algorithm; If the target memory type is a dynamic random access DRAM memory, the DRAM memory is allocated to the application, and the DRAM memory is accessed through a first access interface provided by the system function library GLIBC; If the target memory type is a high-speed interconnect protocol CXL persistent memory, the CXL persistent memory is allocated to the application, and the CXL persistent memory is accessed through a second access interface provided by the CXL persistent memory driver.

2. The method according to claim 1, characterized in that Before receiving the memory application request of the application, the method further includes: Initialize the spatial layout of the CXL persistent memory and build a CXL persistent memory pool.

3. The method according to claim 2, characterized in that Initializing the spatial layout of the CXL persistent memory includes: Dividing the CXL persistent memory into a metadata area and a data area; The metadata area includes a magic number for identifying that the CXL persistent memory has completed initialization, a block size of a data block, a total number of data blocks, a number of free blocks of the data block, and an idle state block, and the data area includes at least one data block.

4. The method according to claim 2, characterized in that: The construction of the CXL persistent memory pool includes: Constructing a persistent memory device linked list to form the CXL persistent memory pool; The persistent memory device linked list is used to represent and manage at least one CXL persistent memory device, and the persistent memory device linked list includes a persistent memory device name, a total number of persistent memory device blocks, a number of persistent memory device free blocks, a persistent memory block allocation linked list, and a next list; The persistent memory block allocation linked list is used to represent and manage the block allocation information of the CXL persistent memory device. The persistent memory block allocation linked list includes a pointer assigned after allocation, the number of allocated bytes, the number of occupied blocks, the starting block sequence number, the free flag and the next list.

5. The method according to claim 1, characterized in that: The determining, according to the memory usage requirement and a preset memory allocation arbitration algorithm, a target memory type to be allocated to the application program comprises: Taking the idle amount of the DRAM memory as a quotient of the total capacity of the DRAM memory to obtain a first idle ratio of the DRAM memory; Determining whether the current application scenario is a light-load application scenario or a heavy-load application scenario according to the first idle ratio of the DRAM memory and a set first ratio threshold; If the current application scenario is a light-load application scenario, determining the target memory type to be allocated to the application according to a performance priority strategy; If the current application scenario is a heavy-load application scenario, the target memory type allocated to the application is determined according to a balancing priority strategy.

6. The method according to claim 5, characterized in that The determining, according to the first idle ratio of the DRAM memory and the set first ratio threshold, whether the current application scenario is a light-load application scenario or a heavy-load application scenario includes: If the first idle ratio of the DRAM memory is greater than the first ratio threshold, determining that the current application scenario is a light-load application scenario; If the first idle ratio of the DRAM memory is less than or equal to the first ratio threshold, it is determined that the current application scenario is a heavy-load application scenario.

7. The method according to claim 5, characterized in that Determining the target memory type to be allocated to the application according to the performance priority strategy includes: If the memory usage requirement is greater than a set multiple of the block size, the CXL persistent memory is used as the target memory type allocated to the application; If the memory usage requirement is less than or equal to a set multiple of the block size, the DRAM memory is used as the target memory type allocated to the application.

8. The method according to claim 5, characterized in that Determining the target memory type allocated to the application according to the balance priority strategy includes: Taking a quotient of the total capacity of the DRAM memory and the total capacity of the CXL persistent memory, and using the quotient result as a second ratio threshold; Taking the idle amount of the DRAM memory as a quotient of the idle amount of the CXL persistent memory, a second idle ratio of the DRAM memory to the CXL persistent memory is obtained; If the second idle ratio is less than the second ratio threshold, using the CXL persistent memory as a target memory type allocated to the application; If the second idle ratio is greater than or equal to the second ratio threshold, the DRAM memory is used as the target memory type allocated to the application.

9. The method according to claim 4, characterized in that The allocating the CXL persistent memory to the application comprises: Determining the number of data blocks required to be allocated to the application program according to the memory usage requirement and the block size of the data block; Traversing each of the persistent memory device linked lists and each of the persistent memory block allocation linked lists in sequence; Determine whether there is a target persistent memory block allocation linked list whose free flag is free and the number of consecutively occupied blocks is the number of data blocks required to be allocated; If it exists, allocate the released data block in the target persistent memory block allocation linked list to the application, set the free flag in the target persistent memory block allocation linked list to not free, and return the memory pointer; Otherwise, a data block is allocated to the application from the CXL persistent memory pool.

10. The method according to claim 9, characterized in that Allocating a data block to the application from the CXL persistent memory pool includes: Traversing each of the persistent memory block allocation linked lists to determine the target persistent memory device with the most free data blocks; Loading the metadata area of ​​the target persistent memory device; Searching for free blocks of the target persistent memory device to determine whether there are free blocks with free marks for the number of consecutive data blocks required for allocation; If it exists, call to allocate CXL persistent memory at the hardware layer and assign the memory pointer; Create a new persistent memory block allocation linked list and set the newly created persistent memory block allocation linked list; Assign the newly created persistent memory classification linked list to the next list in the persistent memory block allocation linked list corresponding to the target persistent memory device, and update the number of persistent memory device free blocks of the target persistent memory device minus the number of data blocks required for allocation; Update the metadata area of ​​the persistent memory device, update the number of free blocks of the persistent memory device of the target persistent memory device minus the number of data blocks to be allocated, and update the free flag of the data block; Return the memory pointer; If it does not exist, an error is returned.

11. A memory allocation system in a Linux environment, characterized in that: include: Hardware layer and software layer; The hardware layer includes at least one DRAM memory connected to the central processor via a memory controller, and at least one CXL persistent memory connected to the central processor via a setting interface; The software layer includes a CXL persistent memory driver located in the Linux kernel space, a CXL persistent memory pool allocator located in the Linux user space, and at least one application program. The CXL persistent memory allocator is used to execute the memory allocation method in the Linux environment as described in any one of claims 1 to 10.

12. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the memory allocation method in the Linux environment as described in any one of claims 1-10.

13. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the memory allocation method in a Linux environment according to any one of claims 1 to 10.

Images