A memory page recycling method, device and electronic equipment under a large memory pool

Abstract

Embodiments of the present application disclose a memory page recycling method and device under a large memory pool and electronic equipment, and relate to the technical field of memory management, and are invented to improve memory utilization. The method comprises the following steps: receiving a memory allocation request initiated by an application program, and allocating only virtual memory space for the application program; obtaining a memory page recycling instruction carrying a virtual address to be recycled; locating a first target memory page generating a memory hole in the large memory pool; judging whether there is an effective page region occupied by other virtual addresses distinguished from the virtual address to be recycled in the first target memory page; if there is an effective page region, migrating complete data in the effective page region to a free region of other free memory pages in the large memory pool; performing alignment and regularity processing on the first target memory page, recycling the first target memory page after regularity processing as a whole, and eliminating the memory hole in the large memory pool. The present application is suitable for recycling memory pages.

Description

Technical Field

[0001] This application relates to the field of memory management technology, and in particular to a method, apparatus, electronic device, and readable storage medium for memory page reclamation under a large memory pool. Background Technology

[0002] In computer systems, memory management techniques pre-allocate a large contiguous memory region from the operating system. The application itself is responsible for allocating and releasing memory blocks within this region, thereby reducing the performance overhead caused by frequent calls to system memory allocation interfaces (such as malloc / free, new / delete). Multiple allocations and releases can lead to memory fragmentation. Furthermore, because physical memory pages contain some active data (data that is being used), even if the memory page has a large number of free areas (holes), the entire page cannot be reclaimed to the operating system, resulting in low resource utilization. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method, apparatus, electronic device, and readable storage medium for memory page reclamation under a large memory pool, which facilitates the improvement of memory utilization.

[0004] In a first aspect, embodiments of this application provide a memory page reclamation method for a large memory pool, applicable to memory management scenarios with a large memory pool capacity of 2MB or more. The method includes: receiving a memory allocation request initiated by an application, allocating virtual memory space only for the application and immediately returning the virtual memory address; asynchronously performing physical memory allocation and page table mapping construction on the allocated virtual memory range using a preset independent asynchronous thread; traversing the mapping relationship between allocated virtual memory and physical memory in the large memory pool in real time, actively pre-detecting empty memory page regions, marking discrete and unaligned invalid empty memory pages, and generating a list of pages to be reclaimed; obtaining the page address carrying the virtual address to be reclaimed. The system executes a page reclamation instruction; based on the virtual address to be reclaimed, it locates the first target memory page in the large memory pool that generates the memory hole; it determines whether there is a valid page region inside the first target memory page that is occupied by other virtual addresses different from the virtual address to be reclaimed; if there is a valid page region, it migrates the complete data within the valid page region to the free area of ​​other free memory pages in the large memory pool, and synchronously updates the page table mapping relationship of the corresponding virtual address; using the system standard physical page size as the alignment benchmark, it performs alignment and regularization processing on the first target memory page, and reclaims the regularized first target memory page as a whole to eliminate the memory hole inside the large memory pool.

[0005] According to a specific implementation of an embodiment of this application, the step of migrating complete data within the effective page area to the free area of ​​other free memory pages in a large memory pool includes: obtaining the actual storage capacity of the effective page area in the first target memory page; selecting and matching a second target memory page from the free memory pages in the large memory pool according to the storage capacity of the effective page area, wherein the free area capacity of the second target memory page is not less than the storage capacity of the effective page area; and completely copying all data within the effective page area to the free area of ​​the second target memory page.

[0006] According to a specific implementation of an embodiment of this application, after the second target memory page is obtained through screening, the method further includes: establishing a second page table mapping relationship between the other virtual addresses and the physical addresses corresponding to the free areas of the second target memory page; replacing the original first page table mapping with the second page table mapping to complete the dynamic iterative update of the memory mapping relationship.

[0007] According to a specific implementation of an embodiment of this application, the step of asynchronously performing physical memory allocation and page table mapping construction on the allocated virtual memory regions through a preset independent asynchronous thread includes: the asynchronous thread continuously polls and traverses the virtual memory regions that have been allocated virtual memory and are not bound to physical memory in the large memory pool, and dynamically classifies each virtual memory region according to business priority, memory access frequency, and virtual memory capacity; according to the scheduling rule of prioritizing high-frequency regions and placing large-capacity regions later, the corresponding physical memory pages are allocated to each virtual memory region in sequence, and a one-to-one page table mapping between virtual memory and physical memory is constructed; the priority level of each virtual memory region is dynamically updated in real time to adapt to the dynamic changes in system business load.

[0008] According to a specific implementation of an embodiment of this application, the step of asynchronously performing physical memory allocation and page table mapping construction on a virtual memory range that has been allocated through a preset independent asynchronous thread includes: if the application accesses the virtual memory range before the corresponding physical memory mapping is completed, a hardware page fault is triggered and the application enters a blocked waiting state. Differentiated fault tolerance processing is performed through a hierarchical blocking waiting mechanism. After the asynchronous thread completes the corresponding physical memory allocation and page table mapping construction, the blocking is lifted and the application's memory access operation is resumed.

[0009] According to a specific implementation of this application, the first target memory page is a plurality of consecutive or discrete physical pages in a large memory pool that have memory gaps; the step of determining whether there is a valid page region inside the first target memory page that is occupied by other virtual addresses different from the virtual address to be reclaimed includes: performing a full-domain detection on each of the plurality of first target memory pages to screen out target reclaimed pages that contain valid page regions; the step of migrating the complete data within the valid page region to the free area of ​​other free memory pages in the large memory pool includes: batch integrating and migrating the valid page data of the plurality of first target memory pages and uniformly storing it in the free area of ​​the second target memory page.

[0010] According to a specific implementation of an embodiment of this application, the real-time traversal of the mapping relationship between allocated virtual memory and physical memory in the large memory pool, proactive pre-detection of memory page hole regions, marking discrete and unaligned invalid memory hole pages, and generating a list of pages to be reclaimed includes: real-time inspection of the mapping relationship between virtual and physical memory in the large memory pool according to a preset period, comparing the matching degree between the virtual memory allocation range and the physical memory occupation range, identifying unoccupied or partially released unaligned memory holes; performing area statistics and dispersion classification on the detected memory holes, prioritizing marking highly discrete and large-area holes as pre-reclaimed objects, and generating a dynamic list of pages to be reclaimed; and performing preprocessing and regularization on the marked hole pages in advance during periods of low system business load.

[0011] According to a specific implementation of an embodiment of this application, the alignment and regularization process of the first target memory page based on the system standard physical page size includes: trimming and uniformly regularizing the non-aligned void areas of the first target memory page according to the system preset standard physical page size to form a complete standard physical page; and recycling the regularized complete standard physical page to the free memory queue of the large memory pool to complete the clearing of memory voids and fragments.

[0012] Secondly, embodiments of this application provide a memory page reclamation device for large memory pools, applicable to memory management scenarios with large memory pools of 2MB or more. The device includes: a receiving module, used to receive memory allocation requests initiated by applications, allocate virtual memory space only for the applications, and immediately return the virtual memory address; through a preset independent asynchronous thread, asynchronously perform physical memory allocation and page table mapping construction on the allocated virtual memory regions; a traversal module, used to traverse the mapping relationship between allocated virtual memory and physical memory in the large memory pool in real time, actively pre-detect empty memory page regions, mark discrete and unaligned invalid empty memory pages, and generate a list of pages to be reclaimed; and an acquisition module, used to acquire memory pages carrying virtual addresses to be reclaimed. The system comprises: a receiving instruction module; a positioning module, used to locate the first target memory page generating the memory hole in the large memory pool based on the virtual address to be reclaimed; a judgment module, used to determine whether there is a valid page region inside the first target memory page that is occupied by other virtual addresses different from the virtual address to be reclaimed; a migration module, used to migrate the complete data within the valid page region to the free area of ​​other free memory pages in the large memory pool if the valid page region exists, and synchronously update the page table mapping relationship of the corresponding virtual address; and an elimination module, used to perform alignment and regularization processing on the first target memory page based on the system standard physical page size, and reclaim the regularized first target memory page as a whole to eliminate the memory hole inside the large memory pool.

[0013] According to a specific implementation of an embodiment of this application, the migration module is specifically configured to: obtain the actual storage capacity of the effective page region in the first target memory page; filter and match a second target memory page in the free memory pages of a large memory pool according to the storage capacity of the effective page region, wherein the free area capacity of the second target memory page is not less than the storage capacity of the effective page region; and completely copy all data in the effective page region to the free area of ​​the second target memory page.

[0014] According to a specific implementation of an embodiment of this application, the apparatus is further configured to: after filtering and obtaining the second target memory page, establish a second page table mapping relationship between the other virtual addresses and the physical addresses corresponding to the free areas of the second target memory page; replace the original first page table mapping with the second page table mapping to complete the dynamic iterative update of the memory mapping relationship.

[0015] According to a specific implementation of an embodiment of this application, the receiving module is specifically used to continuously poll and traverse the virtual memory intervals allocated virtual memory and unbound physical memory in the large memory pool through an asynchronous thread, and dynamically classify each virtual memory interval according to business priority, memory access frequency, and virtual memory capacity; according to the scheduling rule of prioritizing high-frequency intervals and placing large-capacity intervals later, allocate corresponding physical memory pages to each virtual memory interval in sequence, and construct a one-to-one page table mapping between virtual memory and physical memory; and dynamically update the priority level of each virtual memory interval in real time to adapt to dynamic changes in system business load.

[0016] According to a specific implementation of an embodiment of this application, the receiving module is specifically used to trigger a hardware page fault and enter a blocked waiting state if the application accesses the virtual memory range before the corresponding physical memory mapping is completed. Differentiated fault tolerance processing is performed through a hierarchical blocking waiting mechanism. After the asynchronous thread completes the corresponding physical memory allocation and page table mapping construction, the blocking is released and the application's memory access operation is resumed.

[0017] According to a specific implementation of an embodiment of this application, the first target memory page is a plurality of continuous or discrete physical pages in a large memory pool that have memory gaps; the judgment module is specifically used to perform full-domain detection on the plurality of first target memory pages one by one, and screen out target reclaimed pages containing valid page regions; the migration module is specifically used to batch integrate and migrate the valid page data of the plurality of first target memory pages, and uniformly store them in the free area of ​​the second target memory page.

[0018] According to a specific implementation of an embodiment of this application, the traversal module is specifically used to inspect the mapping relationship between the virtual and physical memory pools in real time according to a preset period, compare the matching degree between the virtual memory allocation range and the physical memory occupation range, identify unoccupied or partially released unaligned memory holes; perform area statistics and dispersion classification on the detected memory holes, prioritize marking high dispersion and large area holes as pre-reclaimable objects, and generate a dynamic list of pages to be reclaimed; and perform preprocessing and regularization on the marked hole pages in advance during periods of low system business load.

[0019] According to a specific implementation of an embodiment of this application, the elimination module is specifically used to trim and uniformly organize the unaligned hole areas of the first target memory page according to the standard physical page size preset by the system, forming a complete standard physical page; and to recycle the organized complete standard physical page to the free memory queue of the large memory pool, thereby completing the removal of memory holes and fragments.

[0020] Thirdly, embodiments of this application provide an electronic device, the electronic device comprising: a processor and a memory, wherein the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the memory page reclamation method under a large memory pool as described in any of the foregoing implementations.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the memory page reclamation method under a large memory pool as described in any of the foregoing implementations.

[0022] This embodiment of the memory page reclamation method, apparatus, electronic device, and readable storage medium for large memory pools, targeting scenarios with 2MB or more of memory pools, abandons the existing synchronous virtual-physical memory binding allocation mode. Through a mechanism of virtual memory priority allocation and asynchronous physical mapping, it significantly reduces the application memory request response latency, solves the industry pain points of rigid asynchronous memory allocation scheduling, passive memory fragmentation reclamation, and severe fragmentation accumulation. By aligning and regularizing physical pages during reclamation, it eliminates memory holes and fragments generated during the operation of large memory pools, effectively solving the problems of large memory fragmentation and low memory utilization in existing large memory pools, and significantly improving the memory utilization of large memory pools. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a memory page reclamation method under a large memory pool, provided as an embodiment of this application; Figure 2 A schematic diagram of a memory page reclamation device under a large memory pool provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application will now be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are merely some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0026] To enable those skilled in the art to better understand the technical concept, implementation scheme and beneficial effects of the embodiments of this application, detailed descriptions are provided below through specific embodiments.

[0027] Figure 1 A flowchart illustrating a memory page reclamation method under a large memory pool, as provided in an embodiment of this application, is shown below. Figure 1 As shown, the memory page reclamation method under a large memory pool in this embodiment is applied to memory management scenarios with a large memory pool capacity of 2MB or more, and may include: S101. Receive a memory allocation request initiated by the application, allocate virtual memory space only for the application and immediately return the virtual memory address; through a preset independent asynchronous thread, asynchronously perform physical memory allocation and page table mapping construction on the virtual memory range that has been allocated.

[0028] In this embodiment, only virtual memory space is allocated to the application and the virtual memory address is returned immediately; no synchronous physical memory allocation operation is performed.

[0029] In some examples, by pre-setting an independent asynchronous thread, physical memory allocation and page table mapping construction are performed asynchronously on the virtual memory range that has been allocated. This includes: if the application accesses the virtual memory range before the corresponding physical memory mapping is completed, a hardware page fault is triggered and the application enters a blocked waiting state. Differentiated fault tolerance processing is performed through a hierarchical blocking waiting mechanism. After the asynchronous thread completes the corresponding physical memory allocation and page table mapping construction, the blocking is lifted and the application's memory access operation is resumed.

[0030] The new system introduces three core innovations: dynamic hierarchical scheduling, hierarchical blocking for page faults, and pre-scanning of memory holes. These innovations address industry pain points such as rigid asynchronous memory allocation and scheduling, one-size-fits-all page fault handling, passive memory fragmentation reclamation, and severe fragmentation accumulation. In addition, the system combines page fault tolerance mechanisms to ensure memory access stability and eliminates memory holes and fragments generated during the operation of large memory pools through physical page alignment and regularization reclamation. This effectively solves the problems of large memory fragmentation and low memory utilization in existing large memory pools, and significantly improves the memory utilization of large memory pools.

[0031] A large memory pool refers to a contiguous virtual address space that is pre-allocated to the operating system, and the size of this space is no less than 2MB.

[0032] Provide the application with a memory allocation function interface (e.g., pool_alloc(size)). When this interface is called by the application, it is considered that a memory allocation request initiated by the application has been received. The memory allocation request includes at least the parameter of the requested memory capacity.

[0033] Upon receiving a memory allocation request, a contiguous virtual memory region is allocated from the pre-allocated large memory pool virtual address space according to the requested capacity. The starting address (i.e., virtual memory address), size, and current state (marked as "allocated, unmapped") of this virtual memory region are recorded in the memory pool's metadata. The starting address of this virtual memory region is then immediately returned to the application as the allocation result. Throughout the allocation process, no physical memory allocation functions (such as `alloc_pages`, `get_free_pages`, etc.) are called, no physical memory pages are requested from the operating system, and the valid bits of the page table entry corresponding to this virtual memory region are modified; that is, no form of synchronous physical memory allocation operation is performed.

[0034] Asynchronous threads are independent of the application.

[0035] This asynchronous thread is created and started during memory pool initialization. It operates independently of the application's business threads and does not block them. The asynchronous thread uses a dynamic hierarchical scheduling mechanism to process virtual memory regions that have been allocated but not yet mapped to physical memory.

[0036] The dynamic hierarchical scheduling mechanism refers to the dynamic adjustment of the order and processing weight of each virtual memory interval in the asynchronous processing queue based on parameters such as the business priority, memory access frequency, and interval capacity associated with the virtual memory interval.

[0037] The specific processing flow of the asynchronous thread includes: retrieving virtual memory regions sequentially from the queue to be mapped; using the standard physical page size of the operating system (usually 4KB) as the basic unit, calling the operating system kernel interface (such as alloc_pages in Linux or MAP_ANONYMOUS mode of mmap) to allocate physical memory pages for the virtual memory regions; and then calling the operating system page table modification interface to establish the mapping relationship between each virtual page in the virtual memory region and the corresponding physical page frame in the page table, thus completing the page table mapping construction.

[0038] Since asynchronous threads require a certain amount of time to perform physical memory allocation and page table mapping construction, if the application performs read or write access operations on the virtual memory range before the asynchronous thread has completed the above operations, the CPU's memory management unit (MMU) will find that the present bit in the page table entry corresponding to the virtual address is 0 or invalid when querying the page table. At this time, the hardware will automatically trigger a page fault, and the CPU will switch from user mode to operating system kernel mode and enter the page fault handling process.

[0039] A custom processing logic is registered in the operating system's page fault handler. When it is identified that the page fault is caused by an allocated but unmapped virtual memory region in the large memory pool, the current access thread of the application is placed into a blocked waiting state. During the blocked waiting period, this embodiment performs differentiated fault tolerance processing for services of different priorities through a hierarchical blocked waiting mechanism. The hierarchical blocked waiting mechanism selects different waiting methods according to the priority level of the service to which the memory access operation that caused the page fault belongs: high-priority services use a busy-wait (spin-wait) method to wait for the asynchronous thread to complete the mapping with the shortest delay; medium-priority services use a sleep waiting method that can be interrupted by a signal; low-priority services use a sleep waiting method that can be interrupted by a signal combined with a timeout timer. After the asynchronous thread completes the physical memory allocation and page table mapping construction of the virtual memory region, the blocked state of the corresponding access thread is released through the kernel wake-up mechanism (such as setting a completion flag and calling the wake_up_process function), and the execution of the application's memory access instructions is resumed, enabling the application to read and write the virtual memory region normally.

[0040] S102. Real-time traversal of the mapping relationship between allocated virtual memory and physical memory in the large memory pool, proactive pre-detection of empty memory page regions, marking discrete and unaligned invalid empty memory pages, and generating a list of pages to be reclaimed.

[0041] An asynchronous thread or an additional pre-defined independent inspection thread traverses all allocated virtual memory regions and their corresponding physical memory mapping relationships in the large memory pool in real time according to a preset time period (e.g., every 100 milliseconds or every 500 milliseconds).

[0042] The traversal refers to scanning the virtual-physical mapping table recorded in the memory pool metadata page by page to obtain the virtual address range of each mapped physical page and the physical page frame number it is mapped to.

[0043] Based on the mapping information obtained through traversal, this embodiment actively pre-detects whether there are empty regions in the memory pages. The empty region refers to: within a physical page, because the corresponding virtual memory range has been released or partially released, a portion of the bytes of the physical page is no longer mapped by any valid virtual address, and both sides of the free region (i.e., the lower address side and the higher address side of the physical page) contain data regions that are still validly mapped.

[0044] Furthermore, this embodiment performs discreteness analysis on the detected void regions to identify discretely distributed, unaligned invalid memory void pages. "Discrete" means that the void region is fragmented within a physical page, rather than concentrated in a continuous area at the beginning or end of the page; "unaligned" means that the start or end address of the void region is not aligned with the boundary of the system's standard physical page size (e.g., 4KB). For void pages that meet the above discrete and unaligned characteristics, this embodiment marks them as objects to be reclaimed and generates a list of pages to be reclaimed. The list of pages to be reclaimed is stored in the form of a linked list, array, or queue, and each entry records at least the physical address, page size, and location information of the void region within the physical page to be reclaimed.

[0045] S103. Obtain a memory page reclamation instruction carrying the virtual address to be reclaimed.

[0046] The system receives memory page reclamation instructions triggered by upper-layer management interfaces or automated memory compaction strategies. Each memory page reclamation instruction carries at least the virtual address to be reclaimed, which points to any valid address within the virtual memory region corresponding to the physical page to be reclaimed, or to the starting address of that virtual memory region. The triggering methods for these reclamation instructions include, but are not limited to: internal instructions actively generated by an asynchronous thread upon detecting a free page, external instructions issued by external management tools (such as command-line tools or management APIs), or reclamation instructions automatically triggered by a memory pool preset strategy (such as when the number of free physical pages falls below a threshold).

[0047] S104. Based on the virtual address to be reclaimed, locate the first target memory page that generates the memory hole in the large memory pool.

[0048] The asynchronous thread, based on the virtual address to be reclaimed carried in the reclamation instruction, performs the following location operation: Using the virtual address to be reclaimed as input, it queries the virtual-to-physical mapping table maintained in the memory pool metadata to obtain the physical page frame number corresponding to the virtual address; based on the physical page frame number and combined with the operating system's physical memory management information, it determines the starting physical address of the physical memory page corresponding to that physical page frame number, thereby locating the specific physical page in the large memory pool that caused the memory hole, and using this physical page as the first target memory page. The first target memory page refers to a physical page containing at least one memory hole region; it can be a single physical page or a collection of multiple physical pages.

[0049] S105. Determine whether there is a valid page region inside the first target memory page that is occupied by other virtual addresses that are different from the virtual address to be reclaimed.

[0050] After locating the first target memory page, this embodiment further queries the memory pool metadata to obtain a complete list of all virtual address ranges currently mapped to the first target memory page.

[0051] The determination process specifically involves: traversing the complete list and checking whether there are other virtual address ranges different from the virtual address to be reclaimed that still map to certain byte ranges within the first target memory page. If one or more virtual address ranges other than the virtual address to be reclaimed still occupy a portion of the byte range within the first target memory page, then it is determined that a valid page region exists within the first target memory page.

[0052] The effective page region refers to a contiguous byte range within the first target memory page that is still mapped by a valid virtual address and stores valid application data. If no other virtual address is occupied, the first target memory page is determined to be a completely free page and can be directly reclaimed.

[0053] S106. If the effective page region exists, migrate the complete data within the effective page region to the free area of ​​other free memory pages in the large memory pool, and synchronously update the page table mapping relationship of the corresponding virtual address.

[0054] When it is determined that there is a valid page region inside the first target memory page, this embodiment performs a data migration operation.

[0055] The data migration refers to copying all valid data stored within the valid page region from the first target memory page to the free area of ​​other free memory pages within a large memory pool. Specifically, firstly, the starting offset position and data length of the valid page region within the first target memory page are determined; then, a target page with sufficient free capacity is found among the free memory pages in the large memory pool; and all data from the valid page region is completely copied to the free area of ​​the target page. After the data copying is completed, this embodiment synchronously updates the mapping relationship between the virtual address in the page table and the virtual address corresponding to the valid page region, that is, updating the virtual address that originally pointed to the valid page region within the first target memory page to point to the new physical address where the migrated data is located within the target page, ensuring that the application can correctly locate the migrated data when accessing it subsequently.

[0056] S107. Using the system standard physical page size as the alignment benchmark, perform alignment and regularization processing on the first target memory page, and reclaim the regularized first target memory page as a whole to eliminate memory holes inside the large memory pool.

[0057] After completing the data migration and page table mapping update of the valid page area, the first target memory page no longer contains any valid virtual address mappings, meaning the physical page is completely free. This embodiment uses the standard physical page size defined by the operating system (typically 4KB) as the alignment benchmark to perform alignment normalization processing on the first target memory page. This alignment normalization process ensures that the starting physical address and page size of the first target memory page are aligned with integer multiples of the system's standard physical page size, eliminating any misalignment that may occur due to data migration.

[0058] After the reclaiming process is complete, the reclaimed first target memory page is returned to the operating system or the free memory queue of the large memory pool by calling the operating system's physical memory release interface (such as the `free_pages` or `__free_page` function in Linux). At this point, the first target memory page has been completely reclaimed, and the empty areas within the original page no longer exist due to the reclamation of the entire page, thus completely eliminating memory holes and clearing memory fragmentation within the large memory pool.

[0059] This embodiment, targeting large memory pools of 2MB and above, abandons the existing synchronous virtual-physical memory binding allocation mode. Through the mechanism of virtual memory priority allocation and asynchronous physical mapping, it significantly reduces the application memory request response latency, solves the industry pain points of rigid asynchronous memory allocation scheduling, passive memory fragmentation reclamation, and severe fragmentation accumulation. By eliminating memory holes and fragments generated during the operation of large memory pools through physical page alignment and regularization reclamation, it effectively solves the problems of large memory fragmentation and low memory utilization in existing large memory pools, and significantly improves the memory utilization of large memory pools.

[0060] In some examples, step S106, which involves migrating the complete data within the effective page area to the free area of ​​other free memory pages within the large memory pool, may include: S106a. Obtain the actual storage capacity of the valid page region in the first target memory page.

[0061] Before performing data migration, the record information of the valid page region in the first target memory page is first read from the metadata to obtain the actual storage capacity of the valid page region. The actual storage capacity refers to the total number of bytes in all consecutive byte intervals still mapped by valid virtual addresses within the valid page region. Specifically, the list of valid virtual address intervals mapped to the first target memory page is traversed, and the length of each valid interval within the first target memory page is accumulated to obtain the actual storage capacity value of the valid page region. This actual storage capacity will be used as the basis for subsequent capacity matching in selecting target free pages.

[0062] S106b. Based on the storage capacity of the valid page region, select and match the second target memory page from the free memory pages in the large memory pool, wherein the free region capacity of the second target memory page is not less than the storage capacity of the valid page region.

[0063] After obtaining the actual storage capacity of the valid page region, the current list of free memory pages in the large memory pool is traversed. For each free memory page, the currently available free region capacity is queried. The free region capacity refers to the size of the contiguous byte space within the free memory page that has not yet been mapped by any virtual address and is available for writing new data.

[0064] Pages that meet the following criteria are selected from the list of free memory pages as the second target memory pages: the free area capacity of the page is greater than or equal to the actual storage capacity of the effective page area. If multiple free memory pages meet the criteria, the page whose free area capacity is closest to the storage capacity of the effective page area is further selected to reduce internal fragmentation.

[0065] S106c: Completely copy all data within the valid page area to the free area of ​​the second target memory page.

[0066] Call a memory copy function (such as the memcpy function in the standard C library or the memmove function optimized for large blocks of data), with the starting physical address of the effective page region as the source address, the starting physical address of the free area of ​​the second target memory page as the target address, and the actual storage capacity of the effective page region as the copy length, to perform a data copy operation, completely copying all bytes of data in the effective page region to the free area of ​​the second target memory page.

[0067] The data replication process ensures that the byte order and content integrity of the source data remain unchanged. After copying is complete, the free area capacity of the second target memory page is reduced accordingly, and the valid data has been safely migrated to the new location.

[0068] This embodiment filters target migration pages by accurately matching effective data capacity, avoiding data migration failures and data truncation caused by insufficient free space. At the same time, it ensures the complete migration of effective data, avoiding the defects of data loss and data disorder that are prone to occur in the existing page reclamation process. Without affecting the normal data storage and access of the application, it provides a prerequisite for the subsequent reclamation of empty pages and improves the reliability of memory fragmentation regularization.

[0069] To prevent the application from experiencing memory access failures and address mapping errors after data migration, in some examples, after selecting the second target memory page in step S106b, the method of this embodiment may further include: S108. Establish a second page table mapping relationship between other virtual addresses and the physical addresses corresponding to the free areas of the second target memory page.

[0070] Obtain the other virtual addresses that originally corresponded to the valid page region (i.e., the virtual address range that is different from the virtual address to be reclaimed and is still using the valid page region). For each of the other virtual addresses, calculate the new physical address corresponding to the virtual address in the free area of ​​the second target memory page (i.e., the new physical location where the data is copied), and call the operating system page table modification interface to establish a new mapping relationship between the virtual address and the new physical address in the page table. This new mapping relationship is the second page table mapping relationship. Specifically, the second page table mapping relationship includes at least the correspondence between the virtual page number and the new physical page frame number, as well as the correspondence between the page offsets.

[0071] S109. Replace the original first page table mapping with the second page table mapping to complete the dynamic iterative update of the memory mapping relationship.

[0072] The first page table mapping relationship, which originally pointed to the valid page area within the first target memory page, is replaced with the second page table mapping relationship. Specifically, the overwrite / replacement operation involves modifying the physical page frame number field in the page table entries corresponding to other virtual addresses from the physical page frame number of the first target memory page to the physical page frame number of the second target memory page, and simultaneously modifying the page offset from the original offset within the first target memory page to a new offset within the free area of ​​the second target memory page.

[0073] After the above-mentioned overwrite replacement is completed, the other virtual addresses will be directly located to the new physical location of the migrated data in the second target memory page via the second page table mapping relationship during subsequent accesses, thereby realizing dynamic iterative updates of the memory mapping relationship. To ensure data consistency during the mapping update process, this embodiment uses memory barriers or disables local interrupts during page table updates to ensure the atomicity of the page table update operation.

[0074] To improve the scheduling efficiency and system load adaptability of asynchronous mapping, in some examples, a pre-defined independent asynchronous thread is used to asynchronously perform physical memory allocation and page table mapping construction on the already allocated virtual memory region. This may include: S101a: The asynchronous thread continuously polls and traverses the virtual memory ranges in the large memory pool that have been allocated virtual memory and those that have not been bound to physical memory, and dynamically classifies each virtual memory range according to business priority, memory access frequency, and virtual memory capacity.

[0075] The asynchronous thread is configured to run continuously in the background. Its core processing logic is to continuously poll and traverse all virtual memory regions in the large memory pool that are in the state of "allocated, unmapped" using the unmapped queue of the large memory pool as the polling target.

[0076] The polling traversal refers to an asynchronous thread repeatedly scanning the queue to be mapped at a preset scanning period (e.g., every 10 milliseconds) to check if there are any virtual memory regions to be processed. For each virtual memory region to be processed that is scanned, the asynchronous thread obtains multi-dimensional parameters associated with that region, including but not limited to: the priority identifier of the business type to which the virtual memory region belongs (passed by the application when allocating requests or configured by the system by default), the memory access popularity of the virtual memory region (based on historical access frequency statistics or preset access popularity markers), and the capacity of the virtual memory region.

[0077] Based on the aforementioned multi-dimensional parameters, the asynchronous thread dynamically classifies each virtual memory region using a preset classification algorithm. This dynamic classification means that the classification result is not fixed but dynamically adjusted according to the time decay of business priorities, real-time changes in access frequency, and fluctuations in system load. For example, the same virtual memory region may be classified as low priority when first added to the mapping queue, but its level may be dynamically upgraded as its waiting time in the queue increases or its business priority increases, to prevent starvation.

[0078] S101b: Following the scheduling rule of prioritizing high-frequency and high-priority memory and placing large-capacity memory ranges later, allocate corresponding physical memory pages to each virtual memory range in sequence, and construct a one-to-one page table mapping between virtual memory and physical memory.

[0079] For virtual memory regions with high access frequency and high business priority, the highest processing priority is assigned, and physical memory pages are allocated and page table mappings are built for them in each scheduling cycle. For virtual memory regions with low access frequency and low business priority, lower processing priority is assigned. Under the same priority conditions, larger virtual memory regions are processed after smaller regions to avoid large regions occupying asynchronous threads for a long time, causing starvation in smaller regions. According to the order determined by the above scheduling rules, physical memory allocation operations (calling the operating system kernel interface to obtain free physical pages) and page table mapping construction operations (establishing the correspondence between virtual addresses and physical addresses in the page table) are performed for each virtual memory region in sequence, ensuring that each virtual memory region eventually has a one-to-one corresponding physical memory support.

[0080] This step can be performed by an asynchronous thread.

[0081] S101c dynamically updates the priority levels of each virtual memory region in real time to adapt to dynamic changes in system workload.

[0082] The priority levels of all virtual memory regions in the current unmapped queue are re-evaluated and updated at a preset update period (e.g., every 50 milliseconds).

[0083] The update process includes: monitoring the waiting time of each virtual memory region in the queue, and automatically increasing the priority of regions whose waiting time exceeds a preset starvation threshold; monitoring the real-time load changes of the business types corresponding to each virtual memory region, and dynamically increasing the priority of virtual memory regions corresponding to businesses with sudden load increases; monitoring the overall CPU utilization of the system, and uniformly reducing the asynchronous mapping speed of all virtual memory regions (i.e., extending the polling interval or reducing the number of processes per cycle) when the CPU utilization is below the preset low watermark threshold, and uniformly increasing the asynchronous mapping speed when the CPU utilization is below the preset low watermark threshold. Through the above dynamic updates, this embodiment enables the scheduling behavior of asynchronous threads to adaptively adapt to the dynamic changes in system business load, ensuring the response speed of high-priority businesses while avoiding excessive consumption of system resources by background tasks.

[0084] By pre-setting independent asynchronous threads and adopting a dynamic hierarchical scheduling mechanism, physical memory allocation and page table mapping construction are performed asynchronously for the virtual memory range that has been allocated.

[0085] To provide differentiated page fault handling strategies, in some examples, differentiated fault tolerance is performed through a tiered blocking and waiting mechanism. After the asynchronous thread completes the corresponding physical memory allocation and page table mapping construction, the blocking is lifted and the application's memory access operations resume. This can include: S101d, captures a hardware page fault exception triggered by an application memory access operation.

[0086] When an application thread executes a memory access instruction (read or write), if the present bit in the page table entry corresponding to the accessed virtual address is 0, the CPU hardware automatically triggers a page fault. The CPU immediately saves the context of the current instruction (including the instruction pointer register, program status word, etc.), switches from user mode to kernel mode, and jumps to the entry address of the page fault handler pre-configured by the operating system. The operating system then begins executing the page fault handling process. In this embodiment, a custom page fault capture logic is registered in the operating system's page fault handling framework. This capture logic is executed first when the system page fault handler is called, and is used to identify whether the page fault is caused by a virtual memory region that has been allocated but not mapped in the large memory pool. If the identification is confirmed, the differentiated fault-tolerant processing process of this embodiment is entered; if the page fault is identified as a non-large memory pool region, it is handed over to the operating system's default page fault handler for further processing.

[0087] S101e, Identify the priority and urgency of the currently accessed service.

[0088] Identify the priority level and urgency of the current access service that triggered the page fault. This identification specifically includes: reading the service priority identifier from the control block of the application thread where the page fault occurred. This identifier is specified by the application when submitting a memory allocation request via the `pool_alloc(size, priority)` interface, or automatically mapped by the system based on the process / thread scheduling priority. The urgency is further determined based on at least one of the following methods: whether the current access is an operation on a real-time critical path; whether the virtual memory region corresponding to the current access belongs to a frequently accessed hot data area; and whether the service type associated with the current access is configured as a high-sensitivity service. Based on the identified priority and urgency, determine the blocking wait strategy level used for the current page fault handling.

[0089] S101f performs short-term preemptive blocking wait for high-priority services and prioritizes scheduling asynchronous threads to complete the corresponding physical memory mapping.

[0090] Implement differentiated blocking and waiting strategies: For high-priority services (such as real-time transaction processing and high-frequency data acquisition), this embodiment implements short-term preemptive blocking and waiting. Preemptive blocking and waiting refers to: placing the current accessing thread into a busy-waiting state, continuously polling and checking the mapping completion flag of the target virtual memory region during the busy-waiting period; simultaneously, this embodiment sends a high-priority scheduling signal to the asynchronous thread, causing the asynchronous thread to immediately interrupt its currently processing task and prioritize processing the virtual memory region corresponding to the page fault, thus achieving preemptive scheduling of physical memory allocation. For medium-priority services, this embodiment implements a conventional sleep blocking and waiting strategy that can be interrupted by signals, placing the current accessing thread into a sleep state and adding it to the asynchronous thread's wake-up queue, waiting for mapping completion according to the asynchronous thread's normal processing order.

[0091] For low-priority services (such as background log recording, offline data analysis, etc.), this embodiment implements a sleep blocking wait that can be interrupted by a signal, and starts a timeout timer to allow low-priority services to return after waiting for more than a preset time threshold (such as 100 milliseconds), thus avoiding the blocking of low-priority services from affecting the overall availability of the system.

[0092] S101g: Once the physical memory allocation and page table mapping for the corresponding virtual memory region are completed, the current access instruction will be released from its blocking state, and the application's normal memory access process will resume.

[0093] For high-priority tasks in busy-wait mode, a completion flag is set (using atomic operations to ensure visibility) to ensure the detection conditions in the busy-wait loop are met, causing the CPU to automatically exit the busy-wait loop and resume instruction execution. For medium- and low-priority tasks in sleep-wait mode, a kernel wake-up function (such as `wake_up_process` or `complete_and_exit` in the Linux kernel) is called to switch the blocked thread's state from `TASK_INTERRUPTIBLE` (interruptible sleep) to `TASK_RUNNING` (ready state), allowing the operating system scheduler to reschedule the thread at an appropriate time. After the blocking is resolved, the application thread re-executes the memory access instructions that triggered the page fault. Since a valid mapping exists in the page table at this point, the MMU can successfully complete the address translation, allowing the application to access valid data in the virtual memory region normally.

[0094] Memory gaps are often not isolated in a single physical page, but are widely distributed across multiple physical pages. In one embodiment of this application, the first target memory page is a series of consecutive or discrete physical pages with memory gaps in a large memory pool.

[0095] Step S105, determining whether there is a valid page region within the first target memory page that is occupied by other virtual addresses different from the virtual address to be reclaimed, may include: S105a. Perform full-domain detection on each of the multiple first target memory pages to screen out target reclaimed pages that contain valid page regions.

[0096] For each physical page in the first target memory page set, a full-domain check is performed. This check involves querying the memory pool metadata for all virtual address ranges still mapped to that physical page across its entire byte range (from page start offset 0 to page end offset, i.e., page size minus one), determining whether each byte region within that physical page is still occupied by a valid virtual address. Through this full-domain check, physical pages containing at least one valid page region are identified from the multiple first target memory pages and marked as target reclaimed pages. Physical pages that contain no valid page regions (i.e., completely free) do not require data migration and can be directly reclaimed.

[0097] Step S106, which involves migrating the complete data within the effective page area to the free area of ​​other free memory pages within the large memory pool, may include: S106d: Perform batch integration and migration of valid page data from multiple first target memory pages, and uniformly store them in the free area of ​​the second target memory page.

[0098] This involves a unified batch integration and migration of data from the valid page regions contained in multiple target reclaimed pages. The batch integration and migration refers to sequentially reading the data from the valid page regions of each target reclaimed page and writing this valid data from different physical pages into the free areas of one or more second target memory pages according to a preset arrangement order (e.g., according to the original page address order or according to the offset order of the valid page regions). Specifically, this embodiment centrally caches or directly streams the valid page data from multiple target reclaimed pages and dynamically allocates free space in the second target memory pages according to the size of each valid page region. While meeting the capacity requirements of each valid page region, this ensures that data from different physical pages is stored as compactly as possible in a small number of second target memory pages, thereby reducing the number of physical pages required for data migration and improving migration efficiency.

[0099] To improve the accuracy of hole detection and the rationality of reclamation timing, in some examples, step S102, which involves real-time traversal of the mapping relationship between allocated virtual memory and physical memory in the large memory pool, actively pre-detects memory page hole regions, marks discrete and unaligned invalid memory hole pages, and generates a list of pages to be reclaimed, may include: S102a. In real time, inspect the mapping relationship between the virtual and physical memory pools according to the preset cycle, compare the matching degree between the virtual memory allocation range and the physical memory occupation range, and identify unoccupied and partially released non-aligned memory gaps.

[0100] The inspection task is executed periodically according to a preset time period (e.g., every 100 milliseconds or every 200 milliseconds). In each inspection, the inspection thread obtains a snapshot of the current complete virtual-physical mapping relationship of the large memory pool. The snapshot contains a complete list of all currently allocated virtual memory regions and their mapped physical page frame numbers.

[0101] Further iterate through each mapped physical page in the large memory pool, and for each physical page, obtain the set of all virtual memory regions it maps to.

[0102] By comparing the actual occupied range of each virtual memory region within the physical page with the complete byte range of the physical page, the byte range not covered by any virtual memory region is calculated. If an uncovered byte range exists, it is identified as a memory hole. The term "partial release" refers to a situation where, within a physical page, the original mapping of a portion of the byte range has been released (the virtual address has been released), but the physical page is still partially occupied by other virtual addresses, thus creating a hole within the physical page.

[0103] The term "non-alignment" refers to the fact that the starting or ending offset of the hole region is not aligned with the boundary of an integer multiple of the system's standard physical page size, meaning that the starting position of the hole is not an integer multiple of the page's starting address.

[0104] S102b: Perform area statistics and dispersion classification on the detected memory holes, prioritize marking high dispersion and large area holes as pre-reclaimable objects, and generate a dynamic list of pages to be reclaimed.

[0105] Area statistics refer to calculating the byte length of the hole region to determine its size level (e.g., small hole, medium hole, large hole). Dispersion grading refers to analyzing the distribution characteristics of the hole within the physical page: if the hole is concentrated in a continuous area at the beginning or end of the page, it is classified as low dispersion; if the hole is scattered across multiple discontinuous areas within the page, it is classified as high dispersion. Based on the area statistics and dispersion grading results, a recycling priority score is calculated for each hole page. Recycling priority is positively correlated with hole area and dispersion, meaning that high dispersion and large-area holes have higher recycling priority. According to the recycling priority from high to low, the corresponding physical pages are added to the list of pages to be recycled, with high dispersion and large-area holes being prioritized for pre-recycling. The dynamic list of pages to be recycled is dynamically updated with each inspection, ensuring that the list content always reflects the current fragmentation state of the large memory pool.

[0106] S102c. During periods of low system load, perform preprocessing and regularization on pages marked with holes in advance.

[0107] To prevent memory reclamation operations from impacting system performance during peak business periods, this embodiment further introduces a load-aware preprocessing mechanism. The system's current business load metrics are monitored in real time, including but not limited to CPU utilization, memory bandwidth utilization, and application request response latency. When these load metrics fall below a preset low-load threshold (e.g., CPU utilization below 30%), the system is determined to be in a low-load period. During this low-load period, preprocessing and smoothing operations are performed on physical pages marked as empty pages. These preprocessing and smoothing operations include, but are not limited to: pre-organizing and preparing for migration of active data within empty pages, pre-adjusting address alignment for empty pages, and pre-cleaning page table mappings for physical pages about to be reclaimed. Through this preprocessing, when the formal memory reclamation instruction arrives, the main workload of the reclamation process (data migration preparation and page smoothing) has already been completed during the low-load period, thereby significantly reducing the impact of instantaneous memory reclamation on system performance during peak business periods.

[0108] To ensure that the reclaimed physical pages fully comply with the operating system's physical memory management requirements in terms of size and address, in some examples, step S107, which uses the system's standard physical page size as an alignment benchmark, performs alignment and normalization processing on the first target memory page. This may include: S107a. Based on the system's preset standard physical page size, the unaligned empty areas of the first target memory page are trimmed and uniformly regulated to form a complete standard physical page.

[0109] The internal structure of the first target memory page may exhibit the following characteristics: valid data in the original page has been removed, but unaligned metadata markers or page table remnants may remain within the page; or the page originally contained multiple fragmented empty areas, and after data migration, the page is generally free, but the physical address boundaries are not fully aligned with the system's standard physical page size boundaries. This embodiment performs pruning and uniform normalization operations on the first target memory page based on the operating system's preset standard physical page size (typically 4KB, but 2MB for systems supporting large pages). Pruning refers to removing redundant marker information unrelated to the system's standard physical page size from the first target memory page, ensuring the purity of the page content. Uniform normalization refers to adjusting the starting physical address of the first target memory page to an address aligned with the system's standard physical page size boundary, ensuring the page size is precisely an integer multiple of the system's standard physical page size (e.g., 4KB); for any metadata remnants within the page, zeroing or normalization initialization is performed, ensuring that the first target memory page physically fully meets all the operating system's requirements for a standard physical page, forming a complete and pure standard physical page.

[0110] S107b: The normalized complete standard physical pages are recycled to the free memory queue of the large memory pool, completing the clearing of memory holes and fragments.

[0111] The entire page is reclaimed and added to the free memory queue of the large memory pool. The free memory queue is a data structure maintained within the large memory pool to record the set of currently reallocatable free physical pages. Specifically, the operating system's physical page release function (such as the Linux kernel's `free_pages` function) is called, passing the physical address and number of pages of the standardized physical page as parameters. The operating system marks the physical page as free and adds it to the free memory queue of the large memory pool for subsequent allocation requests. Through this reclamation operation, the original first target memory page and all its internal memory gaps (including areas already released by data migration and previously existing fragmented areas) completely disappear from the large memory pool along with the reclamation of the entire physical page. The gaps within the first target memory page no longer exist in the large memory pool in any form. Furthermore, because the fragments originally scattered across different physical pages are consolidated into a few second target memory pages after data migration and page reclamation, the free space released in the original physical pages returns to the free memory queue as complete pages, thus completing the systematic clearing of memory gaps and fragments within the large memory pool.

[0112] Figure 2 This is a schematic diagram of the structure of a memory page reclamation device under a large memory pool provided in an embodiment of this application, as shown below. Figure 2As shown, the memory page reclamation device of this embodiment is applied to memory management scenarios with a large memory pool of 2MB or more. The device includes: a receiving module 11, used to receive memory allocation requests initiated by applications, allocate virtual memory space only for the applications and immediately return the virtual memory address; through a preset independent asynchronous thread, asynchronously perform physical memory allocation and page table mapping construction for the allocated virtual memory range; a traversal module 12, used to traverse the mapping relationship between allocated virtual memory and physical memory in the large memory pool in real time, actively pre-detect memory page hole areas, mark discrete and unaligned invalid memory hole pages, and generate a list of pages to be reclaimed; an acquisition module 13, used to acquire memory page reclamation instructions carrying virtual addresses to be reclaimed; and a positioning module... Block 14 is used to locate the first target memory page that generates the memory hole in the large memory pool based on the virtual address to be reclaimed; the judgment module 15 is used to determine whether there is a valid page region inside the first target memory page that is occupied by other virtual addresses different from the virtual address to be reclaimed; the migration module 16 is used to migrate the complete data in the valid page region to the free area of ​​other free memory pages in the large memory pool if the valid page region exists, and synchronously update the page table mapping relationship of the corresponding virtual address; the elimination module 17 is used to perform alignment and regularization processing on the first target memory page based on the system standard physical page size, and reclaim the regularized first target memory page as a whole to eliminate the memory hole inside the large memory pool.

[0113] The apparatus of this embodiment can be used to perform Figure 1 The technical solutions of the method embodiments shown are similar in principle and in effect, and will not be described again here.

[0114] The device in this embodiment, designed for large memory pools of 2MB and above, abandons the existing synchronous virtual-physical memory binding allocation mode. Through a mechanism of virtual memory priority allocation and asynchronous physical mapping, it significantly reduces the application memory request response latency, solves the industry pain points of rigid asynchronous memory allocation scheduling, passive memory fragmentation reclamation, and severe fragmentation accumulation. By aligning and regularizing physical pages for reclamation, it eliminates memory holes and fragments generated during the operation of large memory pools, effectively solving the problems of large memory fragmentation and low memory utilization in existing large memory pools, and significantly improving the memory utilization of large memory pools.

[0115] As an optional implementation, the migration module is specifically used to: obtain the actual storage capacity of the effective page region in the first target memory page; filter and match a second target memory page in the free memory pages of the large memory pool according to the storage capacity of the effective page region, wherein the free area capacity of the second target memory page is not less than the storage capacity of the effective page region; and completely copy all data in the effective page region to the free area of ​​the second target memory page.

[0116] As an optional implementation, the apparatus is further configured to: after the second target memory page is obtained by filtering, establish a second page table mapping relationship between the other virtual addresses and the physical addresses corresponding to the free areas of the second target memory page; and replace the original first page table mapping with the second page table mapping to complete the dynamic iterative update of the memory mapping relationship.

[0117] As an optional implementation, the receiving module is specifically used to continuously poll and traverse the virtual memory ranges of allocated virtual memory and unbound physical memory in the large memory pool through an asynchronous thread, and dynamically classify each virtual memory range according to business priority, memory access frequency, and virtual memory capacity; according to the scheduling rule of prioritizing high-frequency and high-priority ranges and placing large-capacity ranges later, allocate corresponding physical memory pages to each virtual memory range in sequence, and construct a one-to-one page table mapping between virtual memory and physical memory; and dynamically update the priority level of each virtual memory range in real time to adapt to the dynamic changes in system business load.

[0118] As an optional implementation, the receiving module includes: a contact submodule for triggering a hardware page fault and entering a blocked waiting state if the application accesses the virtual memory range before the corresponding physical memory mapping is completed, performing differentiated fault tolerance processing through a hierarchical blocking waiting mechanism, and unblocking and resuming the application's memory access operation after the asynchronous thread completes the corresponding physical memory allocation and page table mapping construction.

[0119] As an optional implementation, the contact submodule is specifically used to capture hardware page faults triggered by application memory access operations; identify the priority and urgency of the current access service; perform short-term preemptive blocking wait for high-priority services, and prioritize the scheduling of asynchronous threads to complete the corresponding physical memory mapping; after the physical memory allocation and page table mapping of the corresponding virtual memory range are completed, the blocking state of the current access instruction is released, and the normal memory access process of the application is restored.

[0120] As an optional implementation, the first target memory page is a plurality of consecutive or discrete physical pages in a large memory pool that have memory gaps; the judgment module is specifically used to perform full-domain detection on the plurality of first target memory pages one by one, and screen out target reclaimed pages containing valid page regions; the migration module is specifically used to batch integrate and migrate the valid page data of the plurality of first target memory pages, and uniformly store them in the free area of ​​the second target memory page.

[0121] As an optional implementation, the traversal module is specifically used to inspect the mapping relationship between the virtual and physical memory pools in real time according to a preset period, compare the matching degree between the virtual memory allocation range and the physical memory occupation range, identify unoccupied or partially released unaligned memory holes; perform area statistics and dispersion classification on the detected memory holes, prioritize marking high dispersion and large area holes as pre-reclaimable objects, and generate a dynamic list of pages to be reclaimed; and perform preprocessing and regularization on the marked hole pages in advance during periods of low system business load.

[0122] As an optional implementation, the elimination module is specifically used to trim and uniformly organize the unaligned hole areas of the first target memory page according to the system's preset standard physical page size to form a complete standard physical page; and to recycle the organized complete standard physical page to the free memory queue of the large memory pool to complete the removal of memory holes and fragments.

[0123] The apparatus described in the above embodiments can be used to execute the technical solutions of the above method embodiments. The implementation principle and technical effects are similar, and will not be repeated here.

[0124] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, as shown below. Figure 3 As shown, it may include a processor 61 and a memory 62, wherein the memory 62 is used to store executable program code; the processor 61 runs the program corresponding to the executable program code by reading the executable program code stored in the memory 62, which is used to execute any of the memory page reclamation methods under a large memory pool provided in the foregoing embodiments, and thus can also achieve the corresponding beneficial technical effects, which have been described in detail above and will not be repeated here.

[0125] The aforementioned electronic devices exist in various forms, including but not limited to: (1) Personal computer devices: These devices have computing and processing functions and generally also have mobile Internet access capabilities.

[0126] (2) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0127] (3) Other electronic devices with data interaction functions.

[0128] Accordingly, embodiments of this application also provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement any of the memory page reclamation methods under a large memory pool provided in the foregoing embodiments. Therefore, it can also achieve the corresponding technical effects, as has been described in detail above and will not be repeated here.

[0129] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0130] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0131] In particular, the device embodiment is basically similar to the method embodiment, so the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0132] For ease of description, the above apparatus is described by dividing it into various functional units / modules. Of course, in implementing this application, the functions of each unit / module can be implemented in one or more software and / or hardware.

[0133] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0134] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for memory page reclamation under a large memory pool, characterized in that, For memory management scenarios involving large memory pools of 2MB or more, the method includes: Upon receiving a memory allocation request initiated by an application, the system allocates virtual memory space solely for the application and immediately returns the virtual memory address. Through a pre-defined independent asynchronous thread, the system asynchronously performs physical memory allocation and page table mapping construction for the allocated virtual memory region. The system iterates through the mapping relationship between allocated virtual memory and physical memory in the large memory pool in real time, actively pre-detects empty memory page areas, marks discrete and unaligned invalid empty memory pages, and generates a list of pages to be reclaimed. Retrieve memory page reclamation instructions carrying the virtual addresses to be reclaimed; Based on the virtual address to be reclaimed, locate the first target memory page that generates the memory hole in the large memory pool; Determine whether there is a valid page region within the first target memory page that is occupied by other virtual addresses that are different from the virtual address to be reclaimed; If the valid page region exists, the complete data within the valid page region is migrated to the free area of ​​other free memory pages in the large memory pool, and the page table mapping relationship of the corresponding virtual address is updated synchronously. Using the system's standard physical page size as the alignment benchmark, the first target memory page is aligned and regularized. The regularized first target memory page is then reclaimed as a whole, eliminating memory gaps within the large memory pool.

2. The memory page reclamation method under a large memory pool according to claim 1, characterized in that, The step of migrating complete data within the effective page area to the free area of ​​other free memory pages within the large memory pool includes: Obtain the actual storage capacity of the valid page region in the first target memory page; Based on the storage capacity of the effective page region, a second target memory page is selected and matched from the free memory pages in the large memory pool, wherein the free area capacity of the second target memory page is not less than the storage capacity of the effective page region. All data within the effective page area is completely copied to the free area of ​​the second target memory page.

3. The memory page reclamation method under a large memory pool according to claim 2, characterized in that, After obtaining the second target memory page through filtering, the method further includes: Establish a second page table mapping relationship between the other virtual addresses and the physical addresses corresponding to the free areas of the second target memory page; The original first page table mapping is replaced by the second page table mapping to complete the dynamic iterative update of the memory mapping relationship.

4. The memory page reclamation method under a large memory pool according to claim 1, characterized in that, The process of asynchronously allocating physical memory and constructing page table mappings for the already allocated virtual memory region via a pre-defined independent asynchronous thread includes: An asynchronous thread continuously polls and traverses the virtual memory regions in the large memory pool that have been allocated virtual memory and have not been bound to physical memory, and dynamically classifies each virtual memory region according to business priority, memory access frequency, and virtual memory capacity. Following the scheduling rule of prioritizing high-frequency and large-capacity intervals, physical memory pages are allocated to each virtual memory interval in sequence, and a one-to-one page table mapping between virtual memory and physical memory is constructed. The priority levels of each virtual memory region are updated dynamically in real time to adapt to dynamic changes in system workload.

5. The memory page reclamation method under a large memory pool according to claim 1, characterized in that, The process of asynchronously allocating physical memory and constructing page table mappings for the already allocated virtual memory region via a pre-set independent asynchronous thread includes: If the application accesses the virtual memory region before the corresponding physical memory mapping is completed, a hardware page fault is triggered and the application enters a blocked waiting state. Differentiated fault tolerance processing is performed through a hierarchical blocking waiting mechanism. After the asynchronous thread completes the corresponding physical memory allocation and page table mapping construction, the blocking is lifted and the application's memory access operation is resumed.

6. The memory page reclamation method under a large memory pool according to claim 2, characterized in that, The first target memory page is a series of consecutive or discrete physical pages with memory gaps in a large memory pool; The step of determining whether there is a valid page region within the first target memory page that is occupied by other virtual addresses different from the virtual address to be reclaimed includes: Perform a full-domain detection on each of the multiple first target memory pages to screen out the target reclaimed pages that contain valid page regions; The step of migrating complete data within the effective page area to the free area of ​​other free memory pages within the large memory pool includes: The valid page data of multiple first target memory pages are batch integrated and migrated, and uniformly stored in the free area of ​​the second target memory page.

7. The memory page reclamation method under a large memory pool according to claim 1, characterized in that, The real-time traversal of the large memory pool's allocated virtual memory and physical memory mapping relationship, proactive pre-detection of memory page hole regions, marking discrete and unaligned invalid memory hole pages, and generating a list of pages to be reclaimed, including: The mapping relationship between the virtual and physical memory pools is inspected in real time according to a preset cycle. The matching degree between the virtual memory allocation range and the physical memory occupation range is compared to identify unoccupied and partially released unaligned memory gaps. The detected memory holes are statistically analyzed for area and classified for dispersion. Holes with high dispersion and large area are prioritized as pre-reclaimable objects, and a dynamic list of pages to be reclaimed is generated. During periods of low system load, preprocessing and regularization are performed on pages marked with holes.

8. The memory page reclamation method under a large memory pool according to claim 1, characterized in that, The alignment and normalization process for the first target memory page, based on the system's standard physical page size, includes: Based on the system's preset standard physical page size, the unaligned empty areas of the first target memory page are trimmed and uniformly regulated to form a complete standard physical page; The normalized, complete standard physical pages are recycled to the free memory queue of the large memory pool, thus completing the clearing of memory holes and fragments.

9. A memory page reclamation device for a large memory pool, characterized in that, For memory management scenarios involving large memory pools of 2MB or more, the device includes: The receiving module is used to receive memory allocation requests initiated by applications, allocate virtual memory space only for the applications and immediately return the virtual memory address; through a preset independent asynchronous thread, it asynchronously performs physical memory allocation and page table mapping construction for the virtual memory range that has been allocated. The traversal module is used to traverse the mapping relationship between allocated virtual memory and physical memory in the large memory pool in real time, actively pre-detect empty memory page areas, mark discrete and unaligned invalid empty memory pages, and generate a list of pages to be reclaimed; The acquisition module is used to acquire memory page reclamation instructions carrying virtual addresses to be reclaimed; The positioning module is used to locate the first target memory page that generates the memory hole in the large memory pool based on the virtual address to be reclaimed; The judgment module is used to determine whether there is a valid page region inside the first target memory page that is occupied by other virtual addresses that are different from the virtual address to be reclaimed; The migration module is used to migrate the complete data within the effective page region to the free area of ​​other free memory pages in the large memory pool if the effective page region exists, and to synchronously update the page table mapping relationship of the corresponding virtual address. The elimination module is used to perform alignment and regularization processing on the first target memory page based on the system standard physical page size, and to reclaim the regularized first target memory page as a whole, thereby eliminating memory holes inside the large memory pool.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the memory page reclamation method under a large memory pool as described in any one of claims 1-8.

Images