Memory management method, apparatus, device, and storage medium

Abstract

The application discloses a memory management method, device, equipment and medium, and the method comprises the following steps: constructing a memory management driver module in a kernel layer of a server, determining the number of page frames occupied by a page frame descriptor table; determining the number of page frames occupied by an idle page number table; determining the number of page frames occupied by a page table unit; applying page frame memory according to the number of page frames occupied by the page frame descriptor table; applying the page frame space occupied by idle pages according to the number of page frames occupied by the idle page number table; applying the page frame space occupied by the page table unit and general memory pages according to the number of page frames occupied by the page table unit; initializing the page table unit, and mapping the page frame descriptor table area, the idle page number table area and the page table unit area to a user layer of the server, so that the user manages the memory of the server through the user layer. The method solves the problems of high cost, high risk and low efficiency of traditional memory management by using a hierarchical management architecture and a direct management mechanism of the user layer, and realizes efficient and stable management of a large amount of memory.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer communication, and particularly relates to a memory management method, device, equipment and storage medium. BACKGROUND

[0002] With the development of science and technology, a storage device, as an architecture for centrally storing data in one or more special storage systems, is widely used in supercomputing, cloud storage and AI training platforms and the like, and becomes a key facility for centralized storage and management of data. The storage device provides data storage services for multiple clients or servers through a unified storage resource pool, and has the characteristics of high efficiency, reliability, sharing and easy management. However, the current traditional memory management mechanism has high overhead when managing large-scale memory, and it is difficult to meet the efficient management needs of massive storage scenarios. SUMMARY

[0003] The purpose of the application is to provide a memory management method, which solves the problems of high overhead, high risk and low efficiency of traditional memory management through a hierarchical management architecture and a user layer direct management mechanism, and realizes efficient and stable management of massive memory.

[0004] The technical scheme adopted by the application is a memory management method, comprising:

[0005] A memory management driver module is constructed in the kernel layer of the server, and the memory management driver module comprises a page frame descriptor table, a free page number table and a page table unit.

[0006] The number of page frames occupied by the page frame descriptor table is determined.

[0007] The number of page frames occupied by the free page number table is determined.

[0008] The number of page frames occupied by the page table unit is determined.

[0009] Page frame memory is applied according to the number of page frames occupied by the page frame descriptor table.

[0010] The page frame space occupied by the free page is applied according to the number of page frames occupied by the free page number table.

[0011] The page table unit occupies the page frame space and general memory pages according to the number of page frames occupied by the page table unit.

[0012] The page table unit is initialized, and the page frame descriptor table area, the free page number table area and the page table unit area are mapped to the user layer of the server, so that the user manages the memory of the server through the user layer.

[0013] The application also has the following characteristics:

[0014] The page frame descriptor table adopts a two-level index structure to manage all physical page frames, and the two-level index structure comprises a first-level index structure and a second-level index structure; a table entry of the first-level index structure points to a first address of the second-level index structure table.

[0015] The number of page frames occupied by the page frame descriptor table is calculated in the following manner:

[0016] The number of page frames occupied by the page frame descriptor table = total memory page frame number of the server * 8 / 4096.

[0017] The number of page frames occupied by the free page number table is calculated in the following manner:

[0018] The number of page frames occupied by the free page number table = (total memory page frame number of the server - number of page frames occupied by the page frame descriptor table) * 4 / 4096.

[0019] The number of page frames occupied by the page table unit is calculated in the following manner:

[0020] The number of page frames occupied by the page table unit = (number of page frames occupied by the page frame descriptor table + number of page frames occupied by the free page number table) / 8 * number of bytes occupied by the page table unit / 4096 + (total memory page frame number of the server - number of page frames occupied by the page frame descriptor table - number of page frames occupied by the free page number table) * 8 * number of bytes occupied by the page table unit / 4096.

[0021] Mapping the descriptor table area, the free page number table area and the page table unit area to the user layer of the server comprises:

[0022] Mapping the page frame descriptor table area, the free page number table area and the page table unit area to the user layer of the server by the memory mapping mmap system respectively.

[0023] Initializing the page table unit comprises:

[0024] The page table unit is divided into a page table unit area recording the page frame descriptor, a page table unit area recording the free page number table, a page table unit area recording the self-management structure of the page table unit and a page table unit area recording the general internal table.

[0025] Another technical solution of the present application is a memory management device, comprising:

[0026] A construction module is configured to construct a memory management drive module in the kernel layer of the server, and the memory management drive module comprises a page frame descriptor table, a free page number table and a page table unit.

[0027] A first determination module is configured to determine the number of page frames occupied by the page frame descriptor table.

[0028] A second determination module is configured to determine the number of page frames occupied by the free page number table.

[0029] a third determining module configured to determine the number of page frames occupied by the page table unit;

[0030] a first applying module configured to apply page frame memory according to the number of page frames occupied by the page frame descriptor table;

[0031] a second applying module configured to apply page frame space occupied by the free page according to the number of page frames occupied by the free page number table;

[0032] a third applying module configured to apply page frame space occupied by the page table unit and general memory page according to the number of page frames occupied by the page table unit;

[0033] a mapping module configured to initialize the page table unit, and map the descriptor table area, the free page number table area and the page table unit area to the user layer of the server, so that the user manages the memory of the server through the user layer.

[0034] Another technical solution of the present application is an electronic device, which comprises a processor, a memory, and a computer program stored on the memory and capable of running on the processor, and the computer program implements the steps of the memory management method when executed by the processor.

[0035] Another technical solution of the present application is a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the memory management method when executed by a processor.

[0036] Advantages of the present application:

[0037] The present application discloses a memory management method, device, equipment and storage medium, which significantly reduces the frequency of kernel state and user state switching in the process of physical memory application and release, reduces the management overhead of TB-level memory, by migrating the underlying management mechanism of memory page frames from the kernel layer to the user layer; by tracking page frame allocation and release in real time through the free page number table, and combining the flexible linked list expansion management of the page table unit on the large memory region, the memory resource allocation strategy is effectively optimized, the allocation delay fluctuation is reduced, and the frequent OOM risk is avoided; by mapping the management structure to the user layer, the user layer can directly obtain and manage the free general page frame, and the memory use efficiency and system response speed are further improved. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a step flow chart of the memory management method of the present application;

[0039] Figure 2 is a structural block diagram of the memory management drive module of the present application

[0040] Figure 3 is a flowchart of the memory management method of the present application;

[0041] Figure 4 Figure 1 is a structural block diagram of the memory management device of the present application. DETAILED DESCRIPTION

[0042] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0043] One of the core ideas of the embodiments of the present application is that by determining the number of page frames occupied by each management structure and accurately applying for memory, efficient organization and indexing of physical page frames are achieved, reducing capacity loss caused by memory fragmentation. Secondly, the underlying management mechanism of memory page frames is migrated from the kernel layer to the user layer, significantly reducing the frequency of kernel mode and user mode switching during physical memory application and release, and reducing the management overhead of TB-level memory. At the same time, the free page number table tracks the allocation and release of page frames in real time, and the flexible linked list expansion management of the page table unit for large memory areas effectively optimizes the memory resource allocation strategy, reduces allocation delay fluctuations and avoids frequent OOM risks. Finally, by mapping the management structure to the user layer, the user layer can directly obtain and manage free general page frames, further improving memory usage efficiency and system response speed. In summary, the scheme solves the problems of high overhead, high risk and low efficiency of traditional memory management through hierarchical management architecture and user layer direct management mechanism, and realizes efficient and stable management of massive memory.

[0044] As shown in Figure 1, a step flowchart of a memory management method according to an embodiment of the present application is shown, which can include the following steps: Figure 1

[0045] Step 101, constructing a memory management driver module in the kernel layer of the server, the memory management driver module including a page frame descriptor table, a free page number table, and a page table unit.

[0046] In the embodiment 1 of the present application, the kernel layer refers to the core level of the operating system kernel, which is responsible for directly managing hardware resources (such as memory, CPU), providing underlying services (such as process scheduling, memory allocation), and running in a privileged mode with the highest permission. The memory management driver module refers to a software module in the kernel layer for managing the physical memory of the server, which is responsible for tracking memory usage status, allocating and recycling memory pages, and implementing virtual address and physical address mapping.

[0047] Creating a memory management driver module in the kernel layer of the server can include three core components:

[0048] Page frame descriptor table: used to track the usage status of each physical page frame (such as whether it is allocated or locked).

[0049] Free page number table: records the page frame numbers that are not currently used, supporting efficient allocation and recycling operations. ​

[0050] Page table unit: responsible for maintaining the mapping relationship between virtual address and physical page frame, realizing address conversion.

[0051] Through the process, the basic data structure of memory management in the kernel can be established, providing support for subsequent memory allocation and recycling.

[0052] As Figure 2 , a structural block diagram of a memory management driver module provided by an embodiment of the application is shown, and the memory management driver module can include a page frame descriptor table, a free page number table, and a page table unit.

[0053] Step 102, determine the number of page frames occupied by the page frame descriptor table.

[0054] In the embodiment 1 of the application, determining the number of page frames occupied by the page frame descriptor table refers to calculating the number of physical memory page frames required for storing the page frame descriptor table (Page Frame Descriptor Table) itself. This process is a key step of memory management system initialization, and the purpose is to allocate sufficient physical memory space for the descriptor table;

[0055] Determining the number of page frames occupied by the page frame descriptor table is a basic step of memory management initialization of the operating system. By accurately calculating and allocating physical page frames, it is ensured that the system can effectively track and manage the state of all memory page frames, providing support for subsequent memory allocation, recycling and virtual memory mapping.

[0056] Step 103, determine the number of page frames occupied by the free page number table.

[0057] In the embodiment 1 of the application, determining the number of page frames occupied by the free page number table refers to calculating the number of physical memory page frames required for storing the free page number table (Free Page Number Table) itself. This step is a core link of memory management system initialization, and the purpose is to allocate sufficient physical space for the data structure of recording free page frames, ensuring that the system can efficiently track and manage available memory. The operating system must allocate physical memory for the free page number table first, and then query and modify the page frame state (such as allocation and recycling) in the subsequent process.

[0058] If the allocation is insufficient, the free page information will be lost; if the allocation is excessive, the memory resources will be wasted.

[0059] The operating system can establish the basic data structure for memory allocation and recycling in the initialization phase, ensuring the efficiency and correctness of subsequent memory management operations (such as user process memory application and kernel memory release).

[0060] Step 104, determine the number of page frames occupied by the page table unit.

[0061] In the embodiment 1 of the present application, the number of page frames occupied by the page table unit refers to the number of physical memory page frames required for storing the page table unit (Page Table Unit) data structure itself. This is the core link of virtual address and physical address mapping in memory management, and the purpose is to allocate sufficient physical space for the page table to ensure that the operating system can efficiently complete address translation.

[0062] The page table unit is the basic unit of the page table (Page Table), usually referring to the page table entry (Page Table Entry, PTE) or each level of the multi-level page table structure.

[0063] Page table entry (PTE): records the mapping relationship between virtual page (Virtual Page) and physical page frame (Page Frame), and contains permission bits, access bits, dirty bits, and other metadata.

[0064] Multi-level page table: to reduce page table memory usage, the page table is layered (such as a two-level page table divided into a directory table and a page table), and each level is composed of multiple page table units.

[0065] By accurately calculating the number of page frames of the page table unit, the operating system establishes an efficient and secure mapping bridge between virtual memory and physical memory, which is the cornerstone of the core functions of modern operating systems such as multitasking and memory protection.

[0066] Step 105, according to the number of page frames occupied by the page frame descriptor table, apply for page frame memory.

[0067] In the embodiment 1 of the present application, the page frame descriptor table can be initialized as a two-dimensional array, and a one-level table space (not included in the management memory) is applied, and initialized as a pointer array pointing to a two-level table. According to the group of 8 page frames applied each time, the first address is written into the one-level table entry, the page frame number starts from 0 and accumulates, the two-level table entry is located through the page frame number, and the page frame physical address, reference times, and other information are filled in to realize fast indexing.

[0068] Step 106, according to the number of page frames occupied by the free page number table, apply for the page frame space occupied by the free page.

[0069] In the embodiment 1 of the present application, the free page number table itself is a piece of data that must be stored in a page frame of physical memory.

[0070] For example: if the system has 1000 free page frames, a bitmap is used to record 125 bytes (1000 bits), but the memory can only be allocated by page frames (4KB), so 1 page frame (4096 bytes) is required to store this bitmap.

[0071] In one example, assume that the system has 8GB physical memory in total (page frame size 4KB, total 2097152 page frames), of which 5GB is free (1310720 free page frames).

[0072] Free page frames are recorded using bitmap method:

[0073] Each page frame corresponds to 1 bit, total bits = 1310720 bits = 163840 bytes (160KB).

[0074] Required page frames = 160KB ÷ 4KB = 40 page frames.

[0075] The kernel can apply 40 continuous or non-continuous page frames from the free page frame list through memory management functions (such as Linux's __get_free_pages).

[0076] In step 107, page table unit-occupied page frame space and general memory pages are applied according to the number of page table unit-occupied page frames.

[0077] In embodiment 1 of the present application, the page table unit is a page table entry (PTE), which is the smallest storage unit in the page table, records the mapping relationship between the virtual page and the physical page frame, and the access permission, cache attribute and other metadata, each page table entry corresponds to a virtual page (such as 4KB), and points to a physical page frame.

[0078] Firstly, the total number of page table entries can be calculated according to the size of the virtual address space, the page size and the number of page table levels, and then the total number of page table bytes is obtained, and the page frame number is rounded up.

[0079] Further, the page frames are applied from the kernel-reserved free page frame pool through kernel functions (such as Linux's alloc_page_table), and these page frames are usually located in the high-end memory or a specific reserved area and do not participate in user memory allocation.

[0080] The page table entries are filled in the page table page frames to establish the mapping from the virtual address to the physical address: for the user space page table, the physical page frame number of the general memory page where the user data is located is mapped; for the kernel space page table, the physical address of the kernel code / data is directly mapped.

[0081] The general memory page refers to the ordinary memory page occupied by the user program or kernel data, which is used to store the actual content such as code, data and stack, and is different from the page frame occupied by the “metadata structure” such as the page table.

[0082] In step 108, the page table unit is initialized, and the descriptor table area, the free page number table area and the page table unit area are mapped to the user layer of the server, so that the user manages the memory of the server through the user layer.

[0083] In Embodiment 1 of the present application, after each space is applied, each level of the page table unit of the page table unit can be initialized to establish an initial virtual address to physical address mapping (such as the fixed mapping of the kernel space).

[0084] Further, the virtual address ranges of the descriptor table area, the free page number table area and the page table unit area are exposed to the user layer through the kernel interface.

[0085] The user layer program can directly access the memory management module through the virtual addresses to implement memory allocation, recycling and other operations.

[0086] The memory management interface between the kernel layer and the user layer can be connected through the above steps, so that the user program can directly participate in memory management, and flexibility and efficiency are improved.

[0087] The present application discloses a memory management method, which determines the number of page frames occupied by each management structure and accurately applies memory, realizes efficient organization and indexing of physical page frames, reduces capacity loss caused by memory fragmentation; secondly, the underlying management mechanism of the memory page frame is migrated from the kernel layer to the user layer, significantly reducing the frequency of kernel mode and user mode switching in the physical memory application and release process, and reducing the management overhead of TB-level memory; at the same time, the idle page number table tracks the allocation and release of page frames in real time, and the flexible linked list expansion management of the page table unit to the large memory area effectively optimizes the memory resource allocation strategy, reduces the allocation delay fluctuation and avoids frequent OOM risk; finally, by mapping the management structure to the user layer, the user layer can directly obtain and manage the idle general page frame, further improving the memory use efficiency and system response speed. In summary, the scheme solves the problems of high overhead, high risk and low efficiency of traditional memory management through hierarchical management architecture and user layer direct management mechanism, and realizes efficient and stable management of massive memory.

[0088] In one embodiment of the present application, the page frame descriptor table manages all physical page frames using a two-level index structure, which includes a first-level index structure and a second-level index structure; the table entries of the first-level index structure point to the first address of the second-level index structure table.

[0089] In Embodiment 2 of the present application, the two-level index structure solves the above problems through "layered management", which is similar to the design idea of two-level page table, and the large table is split into smaller blocks, and only the relevant part is loaded when needed, saving memory and improving query efficiency.

[0090] The first-level index structure is essentially an array (or table), each element in the array is called a first-level entry, and each entry stores a pointer pointing to the first address of a second-level index table.

[0091] The size of the first-level index is usually determined by the high bits of the address space (e.g., the first 10 bits of the virtual address), and each first-level table entry occupies a fixed byte (e.g., 8 bytes), so the first-level table itself needs to be resident in memory (but with a smaller scale) even if the corresponding second-level table is not used.

[0092] The second-level index structure is essentially a plurality of small arrays, each array corresponding to a "sub-table" pointed to by a first-level table entry, and each second-level table entry directly corresponds to a specific page frame descriptor, storing detailed information of the page frame. The size of the second-level table is usually determined by the middle bits of the address space (e.g., the middle 10 bits of the virtual address), and each table can manage a fixed number of page frames (e.g., 1024), and only when the first-level table entry is accessed, the corresponding second-level table will be loaded into memory.

[0093] In one example, assuming that the descriptor of the Nth page frame needs to be accessed, the steps are as follows:

[0094] Calculate the first-level index index: use the high bits of the page frame number N as the index to find the corresponding table entry in the first-level table. For example, if the first-level table manages 1024 second-level tables and each second-level table manages 1024 page frames, the high 10 bits of N are the first-level index and the low 10 bits are the second-level index.

[0095] Get the starting address of the second-level table: find the starting address of the corresponding second-level index table through the pointer of the first-level table entry.

[0096] Calculate the second-level index index: use the low bits of the page frame number N as the index to find the specific page frame descriptor in the second-level table.

[0097] The second-level index structure is an efficient method for hierarchical management of large-scale data, and through the "directory + sub-table" mode, it realizes lightweight and fast access to a large number of page frames in memory management.

[0098] In one embodiment 3 of the present application, the number of page frame descriptors is calculated as follows:

[0099] The number of page frame descriptors = the total number of server memory page frames * 8 / 4096.

[0100] In embodiment 3 of the present application, the total number of server memory page frames refers to the number of page frames divided by the total physical memory of the server. If the total physical memory of the server is M bytes and each page frame is 4KB = 4096 bytes, then the total number of page frames is M / 4K,

[0101] Each page frame descriptor occupies 8 bytes (e.g., including status bits, reference count, pointer, etc.), and the total descriptor data volume = total page frame number * 8 bytes.

[0102] Each page frame size is 4096 bytes, so the total data amount is divided by the page frame size to obtain the required page frame number.

[0103] If the total page frame number is 4,194,304 (corresponding to 16 GB memory): 4,194,304*8 / 4096=8192 page frames;

[0104] The descriptor of each 4096 page frames occupies 8 page frames (4096*8 / 4096=8), that is, the descriptor table occupies 1 / 512 of the total page frame number (because each descriptor occupies 8 bytes, and each page frame can store 4096 / 8=512 descriptors).

[0105] In an embodiment 3 of the present application, the page frame number occupied by the free page number table is calculated as follows:

[0106] The page frame number occupied by the free page number table=(total memory page frame number of the server-page frame number occupied by the page frame descriptor table)*4 / 4096.

[0107] In an embodiment 3 of the present application, the page frame number occupied by the page table unit is calculated as follows:

[0108] The page frame number occupied by the page table unit=(page frame number occupied by the page frame descriptor table+page frame number occupied by the free page number table) / 8*byte number occupied by the page table unit / 4096+(total memory page frame number of the server-page frame number occupied by the page frame descriptor table-page frame number occupied by the free page number table)*8*byte number occupied by the page table unit / 4096.

[0109] In an embodiment 4 of the present application, the page frame descriptor table area, the free page number table area and the page table unit area are mapped to the user layer of the server, comprising:

[0110] The page frame descriptor table area, the free page number table area and the page table unit area are respectively mapped to the user layer of the server through the memory mapping mmap system.

[0111] In the embodiment 4 of the present application, the mmap system can map the region of a file, a device or a physical memory to the virtual address space of a process, to realize the memory sharing between the user state and the kernel state.

[0112] Specifically, the physical start address and the length of the descriptor table, the free page number table and the page table unit can be obtained through a kernel module or a special interface (such as ioremap). For example, the physical start address of the descriptor table is 0x1000000, and the length is 64 KB (16 page frames).

[0113] The physical start address of the free page number table is 0x1100000, and the length is 4 KB (1 page frame).

[0114] Page table unit physical start address: 0x1200000, length: 8KB (2 page frames).

[0115] Further, the kernel memory is mapped to the user space through mmap.

[0116] The present application simplifies development, improves flexibility, and supports customized strategies by opening the memory management control to the user space.

[0117] In one embodiment 4 of the present application, the page table unit is initialized, including:

[0118] The page table unit is divided into a page table unit area corresponding to the record page frame descriptor, a page table unit area corresponding to the record free page number table, a page table unit area recording the self-management structure of the page table unit, and a page table unit area recording the general internal table.

[0119] In the embodiment 4 of the present application, when the page table unit is initialized, it can be divided into different functional areas, thereby realizing fine management of memory resources and improving memory allocation efficiency and system stability.

[0120] Specifically, after hardware initialization, the operating system kernel first initializes the self-management structure area (area 3), sets the page table base register, establishes the page table mapping of the kernel space, and then:

[0121] The free page number table area (area 2) is initialized, the physical memory is scanned, and the unused page frames are marked (such as obtaining the memory layout through BIOS).

[0122] Further, the page frame descriptor area (area 1) is allocated for a new process, an independent page table structure is created, and the user space virtual address is mapped.

[0123] Finally, the general resources (such as shared libraries) required by the process are mapped to the general internal table area (area 4).

[0124] When allocating memory, the free page frames can be obtained from the free page number table (area 2), and the page frame descriptor (area 1) and the free page table (area 2) are updated.

[0125] When recycling memory, the page frame can be marked as free, and the mapping relationship in the page table descriptor is cleared. Through this functional partition design, the page table unit can efficiently manage memory resources, balance performance, security, and resource utilization.

[0126] For example, Figure 3The flowchart of the memory management method of the application is shown, the maximum memory value supported by the server can be set first, then the values (the number of page frames occupied by the page frame descriptor table, the number of page frames occupied by the free page number table, the number of page frames occupied by the page table unit) are calculated, then the parameters in the actual application are determined according to the actual application memory amount, and further, the first-level page frame descriptor table memory and the second-level page frame descriptor table memory are calculated, and then the first-level page frame descriptor table and the second-level descriptor table related to the page of the descriptor table are initialized.

[0127] Further, the free number table related page frame can be applied and the related descriptor table item is initialized, the page table unit related page frame can be applied and the related descriptor table item is initialized, and the remaining general page frame can be applied and the related descriptor table item is initialized.

[0128] After the application, the map list of the starting value of the theoretical page number and the starting value of the actual number of each management area can be recorded, the page table unit area is initialized, the management area occupied by the page table unit structure is written into the corresponding page table unit area, and the page table unit area for describing the remaining general memory frame is reserved, and finally, the miscellaneous device / dev / mm is registered, the icotl and mmap interfaces are registered, and the user can manage the memory of the server through the corresponding interface.

[0129] The application discloses a memory management method, which determines the number of page frames occupied by each management structure and accurately applies memory, realizes efficient organization and indexing of physical page frames, reduces capacity loss caused by memory fragmentation; secondly, the underlying management mechanism of the memory page frame is migrated from the kernel layer to the user layer, which significantly reduces the frequency of kernel mode and user mode switching in the process of physical memory application and release, and reduces the management overhead of TB-level memory; at the same time, the free page number table tracks the allocation and release of page frames in real time, and the flexible linked list expansion management of the page table unit to the large memory area effectively optimizes the memory resource allocation strategy, reduces the allocation delay fluctuation and avoids the frequent OOM risk; finally, by mapping the management structure to the user layer, the user layer can directly obtain and manage the free general page frame, further improving the memory use efficiency and system response speed. In summary, through the hierarchical management architecture and the user layer direct management mechanism, the high overhead, high risk and low efficiency problems of the traditional memory management are systematically solved, and efficient and stable management of massive memory is realized.

[0130] It should be noted that, for the method embodiment, in order to simply describe, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the application are not limited by the order of the described actions, because according to the embodiments of the application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of the application.

[0131] Referring to Figure 4 , a structural block diagram of a memory management device provided by an embodiment 5 of the present application is shown, the device can include:

[0132] The construction module 201 is configured to construct a memory management driver module at a kernel layer of a server, the memory management driver module including a page frame descriptor table, a free page number table, and a page table unit.

[0133] The first determination module 202 is configured to determine a number of page frames occupied by the page frame descriptor table.

[0134] The second determination module 203 is configured to determine a number of page frames occupied by the free page number table.

[0135] The third determination module 204 is configured to determine a number of page frames occupied by the page table unit.

[0136] The first application module 205 is configured to apply for page frame memory according to the number of page frames occupied by the page frame descriptor table.

[0137] The second application module 206 is configured to apply for page frame space occupied by free pages according to the number of page frames occupied by the free page number table.

[0138] The third application module 207 is configured to apply for page frame space occupied by the page table unit and general memory pages according to the number of page frames occupied by the page table unit.

[0139] The mapping module 208 is configured to initialize the page table unit and map the descriptor table area, the free page number table area, and the page table unit area to a user layer of the server, so that the user manages the memory of the server through the user layer.

[0140] The present application discloses a memory management device, by determining the number of page frames occupied by each management structure and accurately applying for memory, efficient organization and indexing of physical page frames are realized, capacity loss caused by memory fragmentation is reduced; secondly, the underlying management mechanism of memory page frames is migrated from the kernel layer to the user layer, the frequency of kernel mode and user mode switching in the process of physical memory application and release is significantly reduced, and the management overhead of TB-level memory is reduced; at the same time, the free page number table tracks page frame allocation and release in real time, combined with the flexible linked list expansion management of the page table unit on the large memory area, the memory resource allocation strategy is effectively optimized, the allocation delay fluctuation is reduced and the frequent OOM risk is avoided; finally, by mapping the management structure to the user layer, the user layer can directly obtain and manage the free general page frame, further improving the memory use efficiency and system response speed. In summary, through the hierarchical management architecture and the user layer direct management mechanism, the high overhead, high risk and low efficiency problems of traditional memory management are systematically solved, and efficient and stable management of massive memory is realized.

[0141] In one embodiment 5 of the present application, the page frame descriptor table manages all physical page frames by using a two-level index structure, which includes a first-level index structure and a second-level index structure; the table entry of the first-level index structure points to the first address of the second-level index structure table.

[0142] In one embodiment 5 of the present application, the number of page frames occupied by the page frame descriptor table is calculated as follows:

[0143] The number of page frames occupied by the page frame descriptor table = the total number of memory page frames of the server * 8 / 4096.

[0144] In one embodiment 5 of the present application, the number of page frames occupied by the page table unit is calculated as follows:

[0145] The number of page frames occupied by the page table unit = (the number of page frames occupied by the page frame descriptor table + the number of page frames occupied by the free page number table) / 8 * the number of bytes occupied by the page table unit / 4096 + (the total number of memory page frames of the server - the number of page frames occupied by the page frame descriptor table - the number of page frames occupied by the free page number table) * 8 * the number of bytes occupied by the page table unit / 4096.

[0146] In one embodiment 5 of the present application, the mapping module comprises:

[0147] The mapping submodule is configured to map the descriptor table area, the free page number table area and the page table unit area to the user layer of the server respectively by using the memory mapping (mmap) system.

[0148] In one embodiment 5 of the present application, the number of page frames occupied by the free page number table is calculated as follows:

[0149] The number of page frames occupied by the free page number table = (the total number of memory page frames of the server - the number of page frames occupied by the page frame descriptor table) * 4 / 4096.

[0150] In one embodiment 5 of the present application, the mapping module comprises:

[0151] The division submodule is configured to divide the page table unit into a page table unit area recording the page frame descriptor, a page table unit area recording the free page number table, a page table unit area recording the self-management structure of the page table unit, and a page table unit area recording the general internal table.

[0152] In one embodiment 5 of the present application, the page table unit area recording the page frame descriptor is configured to store the mapping relationship between the page frame descriptor (PFD) and the virtual address, and the page frame descriptor records the state of the physical page frame (such as whether it is occupied, access permission, modification flag, etc.), while the page table unit maps the virtual address to a specific physical page frame.

[0153] When a process accesses a virtual address, the CPU looks up the corresponding physical page frame through the page table unit. The page table entry in this area is directly associated with the metadata of the physical memory, ensuring fast conversion from virtual address to physical address.

[0154] The page table unit area recording the free page number table manages the free page number table, that is, tracks the free page frames in the physical memory that are not allocated. The free page number table records the numbers of the free page frames, and the corresponding page table unit marks the "allocatable" state of these page frames.

[0155] When the system needs to allocate a new physical page frame (such as a process applying for memory), it first queries the page table unit corresponding to the free page number table, finds the page frame marked as "free", and updates its state to "allocated". This process ensures the efficiency and atomicity of memory allocation.

[0156] The page table unit area recording the management structure of the page table unit itself is used to store the management metadata of the page table unit itself, such as the hierarchical structure of the page table (such as multi-level page table), access statistics of the page table entry (such as the time of the most recent access), consistency markers of the page table (such as whether it needs to be synchronized to the disk), etc.

[0157] The page table itself may occupy a large amount of memory, and through an independent management area, the system can efficiently maintain the integrity and consistency of the page table. For example, in a multi-level page table structure, this area may store the base addresses of the page tables at each level, validity markers of the page table entries, etc.

[0158] The page table unit area recording the general internal table is used to map the general internal tables (General Internal Tables) used by the operating system kernel, such as the process control block (PCB), the file descriptor table, the control structure of the device driver, etc. These internal tables are usually stored in physical memory and need to be mapped to the kernel address space through the page table unit.

[0159] The general internal table is a basic data structure for the operation of the operating system kernel, and through an independent page table unit area, the kernel space is isolated from the user space, and the efficiency of kernel access to these structures is improved.

[0160] The present application divides the page table unit into four functional areas, achieving hierarchical management of memory mapping, balancing efficiency, stability, and security. This design is one of the core technologies of modern operating system virtual memory management, especially suitable for server-level high-performance computing scenarios, capable of supporting the memory needs of large-scale concurrent processes, while ensuring the long-term reliable operation of the system.

[0161] The application discloses a memory management device, which realizes efficient organization and indexing of physical page frames by determining the number of page frames occupied by each management structure and accurately applying memory, and reduces capacity loss caused by memory fragmentation; secondly, the underlying management mechanism of the memory page frame is migrated from the kernel layer to the user layer, significantly reducing the kernel mode and user mode switching frequency in the physical memory application and release process, and reducing the management overhead of TB-level memory; at the same time, the idle page number table tracks the allocation and release of page frames in real time, and the flexible linked list expansion management of the page table unit on the large memory area effectively optimizes the memory resource allocation strategy, reduces the allocation delay fluctuation and avoids frequent OOM risk; finally, by mapping the management structure to the user layer, the user layer can directly obtain and manage the idle general page frame, further improving the memory use efficiency and system response speed. In summary, the scheme solves the problems of high overhead, high risk and low efficiency of traditional memory management through the hierarchical management architecture and the user layer direct management mechanism, and realizes efficient and stable management of massive memory.

[0162] For the device embodiment, it is basically similar to the method embodiment, so the description is relatively simple, and the related parts refer to the part of the method embodiment 1-4.

[0163] The embodiment 6 of the application further provides an electronic device, which comprises:

[0164] The computer program is stored on the memory and can run on the processor, and when the computer program is executed by the processor, each process of the memory management method embodiment is realized, and the same technical effect can be achieved, and details are not repeated here.

[0165] The embodiment 7 of the application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, each process of the memory management method embodiment is realized, and the same technical effect can be achieved, and details are not repeated here.

[0166] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to.

[0167] Those skilled in the art should understand that the embodiments of the application can be provided as a method, device or computer program product. Therefore, the embodiments of the application can adopt a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer usable program code.

[0168] The embodiments of the present application are described with reference to the flowchart and / or block diagram of the method, terminal device (system) and computer program product according to the embodiments of the present application. It is understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 Figure 1

[0169] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal devices to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 Figure 1

[0170] These computer program instructions can also be loaded into a computer or other programmable data processing terminal devices, such that a series of operational steps are carried out on the computer or other programmable terminal devices to produce a computer implemented process so that the instructions executed on the computer or other programmable terminal devices provide steps for implementing the function specified in the flowchart and / or block diagram block or blocks. Figure 1 Figure 1

[0171] Although the preferred embodiments of the present application have been described, those skilled in the art will be able to make additional modifications and variations to these embodiments without departing from the scope of the present application. Accordingly, it is intended to include all such modifications and variations as fall within the scope of the embodiments of the present application.

[0172] ​​​​​​Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be intended to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus including the element.

[0173] The above describes in detail the memory management method, device and medium provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A memory management method, characterized in that, include: A memory management driver module is built at the kernel layer of the server. The memory management driver module includes a page frame descriptor table, a free page number table, and page table cells. Determine the number of page frames occupied by the page frame descriptor table; Determine the number of page frames occupied by the free page number table; Determine the number of page frames occupied by the page table unit; Allocate page frame memory based on the number of page frames occupied by the page frame descriptor table; Allocate page frame space for free pages based on the number of page frames occupied by the free page number table; Allocate page frame space and general memory pages for page table units based on the number of page frames occupied by page table units; The allocation and release of page frames are tracked through a free page number table, and the page frame numbers recorded in the free page number table are updated synchronously. Large memory regions are managed by using linked lists in page table units, establishing a mapping relationship between large memory regions and physical page frames; Initialize the page table cells and map the page frame descriptor table area, free page number table area, and page table cell area to the server's user layer, so that users can manage the server's memory through the user layer.

2. The memory management method according to claim 1, characterized in that, The page frame descriptor table uses a two-level index structure to manage all physical page frames. The two-level index structure includes a first-level index structure and a second-level index structure. The entries in the first-level index structure point to the starting address of the second-level index structure table.

3. The memory management method according to claim 1, characterized in that, The number of page frames occupied by the page frame descriptor table is calculated in the following way: The number of page frames occupied by the page frame descriptor table = the total number of memory page frames of the server * 8 / 4096.

4. The memory management method according to claim 1, characterized in that, The number of page frames occupied by the free page number table is calculated in the following way: The number of page frames occupied by the free page number table = (total number of memory page frames on the server - number of page frames occupied by the page frame descriptor table) * 4 / 4096.

5. The memory management method according to claim 1, characterized in that, The number of page frames occupied by the page table unit is calculated in the following way: The number of page frames occupied by a page table unit = (the number of page frames occupied by the page frame descriptor table + the number of page frames occupied by the free page number table) / 8 * the number of bytes occupied by the page table unit / 4096 + (the total number of page frames in the server's memory - the number of page frames occupied by the page frame descriptor table - the number of page frames occupied by the free page number table) / 8 * the number of bytes occupied by the page table unit / 4096.

6. The memory management method according to claim 1, characterized in that, The step of mapping the page frame descriptor table area, the free page number table area, and the page table cell area to the user layer of the server includes: The memory mapping mmap system maps the descriptor table area, free page number table area, and page table cell area to the user layer of the server, respectively.

7. The memory management method according to claim 1, characterized in that, The initialization page table unit includes: The page table unit is divided into a page table unit area that records page frame descriptors, a page table unit area that records free page number tables, a page table unit area that records the page table unit's own management structure, and a page table unit area that records general internal tables.

8. A memory management device, characterized in that, include: The building module is used to build a memory management driver module at the kernel layer of the server. The memory management driver module includes a page frame descriptor table, a free page number table, and page table units. The first determining module is used to determine the number of page frames occupied by the page frame descriptor table; The second determining module is used to determine the number of page frames occupied by the free page number table; The third determining module is used to determine the number of page frames occupied by the page table unit; The first allocation module is used to allocate page frame memory based on the number of page frames occupied by the page frame descriptor table; The second application module is used to apply for the page frame space occupied by free pages based on the number of page frames occupied by the free page number table; The third application module is used to apply for the page frame space occupied by the page table unit and the general memory page based on the number of page frames occupied by the page table unit; The page frame tracking module is used to track the allocation and release status of page frames in real time through the free page number table and update the page frame numbers recorded in the free page number table synchronously. The linked list management module is used to manage large memory regions through page table units using a linked list expansion method, and to establish a mapping relationship between large memory regions and physical page frames; The mapping module is used to initialize page table cells and map the page frame descriptor table area, free page number table area, and page table cell area to the user layer of the server.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the memory management method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements the steps of the memory management method as described in any one of claims 1-7.

Images