A Linux page cache limiting method based on proactive page reclamation

Abstract

This invention proposes a Linux page cache limiting method based on active page cache reclamation, belonging to the field of memory control technology. This method configures a memory page allocation flag during the page cache allocation process; dynamically limits the maximum amount of physical memory used by the page cache; specifically, the maximum amount of physical memory used by the page cache is set by adding a kernel file, and the maximum amount of physical memory used by the page cache is dynamically and adaptively adjusted by adding a kernel thread; when the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; notably, only page caches with the memory page allocation flag, and not anonymous pages in Linux memory, are reclaimed. This invention dynamically limits the page cache, ensuring that page cache usage has a maximum limit; and automatically adjusts the page cache limit according to memory pressure and file read rate; when the page cache reaches the maximum limit, only the memory page cache pages are reclaimed.

Description

Technical Field

[0001] This invention belongs to the field of memory control technology, and in particular relates to a method for limiting Linux page cache based on active reclamation. Background Technology

[0002] The Linux operating system uses a page cache mechanism to cache files on block devices. The page cache can grow indefinitely until memory reclamation is triggered. In this situation, when the page cache is overused, some atomic memory allocations in the kernel will fail, causing delays or even failures in some business operations. Linux memory reclamation does not distinguish between page types, including both cached pages and anonymous pages, posing a risk of incorrect reclamation.

[0003] In existing technologies, the kswapd kernel thread is often used for background memory reclamation or direct kernel reclamation. The kswapd kernel thread performs memory reclamation based on a watermark threshold when memory is insufficient. However, if the watermark is set improperly, such as an excessively high low watermark, it may prematurely wake up the kswapd thread, causing unnecessary background reclamation overhead. Conversely, it may easily trigger direct reclamation, increasing the risk of blocking business processes. Memory reclamation includes both page cache pages and anonymous pages. When excessive page cache usage leads to memory reclamation, there is a risk of incorrect reclamation, impacting system performance. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a Linux page cache limitation scheme based on proactive memory reclamation, thereby achieving page cache limitation based on proactive memory reclamation.

[0005] The first aspect of this invention proposes a method for limiting Linux page cache based on proactive page reclamation, the method comprising:

[0006] Configure the memory page allocation flag during the page caching process;

[0007] Dynamically limit the maximum amount of physical memory used by the page cache; this is achieved by setting the maximum amount of physical memory used by the page cache through the kernel sysfs file, and by dynamically and adaptively adjusting the maximum amount of physical memory used by the page cache through the addition of kernel threads.

[0008] When the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; however, only page caches with memory page request flags are reclaimed, not anonymous pages in Linux memory.

[0009] According to the method of the first aspect of the present invention, a memory page allocation flag is configured during the page cache allocation process; wherein: a memory page allocation flag GFP_PAGECACHE is added for page cache pages, and all actions of allocating page cache are marked with GFP_PAGECACHE, and the page cache is managed uniformly.

[0010] According to a method of the first aspect of the present invention, the maximum amount of physical memory used by the page cache is dynamically limited; wherein:

[0011] Add the kernel sysfs file / proc / sys / pagecache_ratio to set the maximum amount of physical memory used by the page cache. This maximum usage is consistent with the low watermark WMARK_LOW during initialization.

[0012] Add a kernel thread to monitor memory usage and file read rate, and dynamically and adaptively adjust the maximum physical memory usage of the page cache based on memory usage and file read rate.

[0013] According to the method of the first aspect of the present invention, when the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; wherein:

[0014] When the kernel page cache management thread performs page cache reclamation, it calculates the activity level of the page cache pages and reclaims them first, prioritizing the reclamation of page cache pages with an activity level below the first threshold and marked with a memory page request flag.

[0015] At the same time, the increased kernel threads are used to monitor memory usage and file read rate; if the I / O operations in the system exceed the second threshold, the file read hit rate is further monitored; if the hit rate exceeds the third threshold, the maximum amount of physical memory used by the page cache is increased.

[0016] According to the method of the first aspect of the present invention, when physical memory is insufficient, kswapd is woken up to reclaim physical memory, and the kernel page cache management thread is woken up to reclaim the page cache. The increased kernel thread is used to reduce the maximum amount of physical memory occupied by the page cache based on the statistical memory usage and file read rate.

[0017] According to the method of the first aspect of the present invention, when requesting a page cache:

[0018] During the initialization phase of kernel memory management, the maximum amount of physical memory used by the page cache is configured by default to be equal to the low watermark WMARK_LOW.

[0019] When a received memory request is a page cache request, the GFP_PAGECACHE flag is marked, and it is further determined whether the requested page cache exceeds the maximum usage. If not, the page cache is allocated, and the memory request is successful.

[0020] According to the method of the first aspect of the present invention, it is determined whether the requested page cache exceeds the maximum usage; if so, the kernel page cache management thread is woken up to reclaim the page cache.

[0021] A second aspect of this invention proposes a Linux page cache limiting system based on proactive page reclamation, the system comprising a processing unit configured to:

[0022] Configure the memory page allocation flag during the page caching process;

[0023] Dynamically limit the maximum amount of physical memory used by the page cache; this is achieved by setting the maximum amount of physical memory used by the page cache through the kernel sysfs file, and by dynamically and adaptively adjusting the maximum amount of physical memory used by the page cache through the addition of kernel threads.

[0024] When the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; however, only page caches with memory page request flags are reclaimed, not anonymous pages in Linux memory.

[0025] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements a Linux page cache limiting method based on proactive cache reclamation, as disclosed in the first aspect of this invention.

[0026] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements a Linux page cache limitation method based on proactive cache reclamation, as described in the first aspect of this disclosure.

[0027] As can be seen, this invention sets a limit on the maximum usage of the page cache, including adding a kernel sysfs file ` / proc / sys / vm / pagecache_ratio` to set the maximum ratio of physical memory occupied by the page cache, and adding a memory page allocation flag `GFP_PAGECACHE`. This invention uses a page cache overload handling strategy, including adding a page cache management thread, adaptively setting the maximum ratio of memory that the page cache can occupy, and prioritizing the reclamation of inactive pages when reclaiming the page cache. The page cache management thread proposed in this invention includes functions for statistically analyzing memory usage, statistically analyzing file read rates, and adaptively adjusting the maximum page cache ratio based on memory usage and read rates. The page cache management thread proposed in this invention statistically analyzes the activity of page cache pages, divides page cache pages into gradients based on activity, and reclaims page cache pages in a gradient from inactive to active during page cache reclamation.

[0028] In summary, this invention provides a Linux page cache limiting scheme based on proactive reclamation, including setting page cache quotas; adding a new page request flag; and a memory reclamation strategy when the page cache is overloaded. Using this scheme, the page cache quota can be adaptively adjusted according to memory pressure and file read efficiency. When memory pressure is high, the page cache quota ratio is lowered, thus reducing page cache memory usage. When file read efficiency decreases and memory pressure is low, the page cache quota ratio is increased, thus reducing page swapping speed and improving file read hit rate. This invention improves the flexibility, accuracy, and stability of page cache management, and compensates for the shortcomings of existing mechanisms in some complex scenarios. Attached Figure Description

[0029] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating the Linux page cache limitation method based on active reclamation according to an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] The first aspect of this invention proposes a method for limiting Linux page cache based on proactive page reclamation, the method comprising:

[0033] Configure the memory page allocation flag during the page caching process;

[0034] Dynamically limit the maximum amount of physical memory used by the page cache; this is achieved by setting the maximum amount of physical memory used by the page cache through the kernel sysfs file, and by dynamically and adaptively adjusting the maximum amount of physical memory used by the page cache through the addition of kernel threads.

[0035] When the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; however, only page caches with memory page request flags are reclaimed, not anonymous pages in Linux memory.

[0036] According to the method of the first aspect of the present invention, a memory page allocation flag is configured during the page cache allocation process; wherein: a memory page allocation flag GFP_PAGECACHE is added for page cache pages, and all actions of allocating page cache are marked with GFP_PAGECACHE, and the page cache is managed uniformly.

[0037] According to a method of the first aspect of the present invention, the maximum amount of physical memory used by the page cache is dynamically limited; wherein:

[0038] Add the kernel sysfs file / proc / sys / pagecache_ratio to set the maximum amount of physical memory used by the page cache. This maximum usage is consistent with the low watermark WMARK_LOW during initialization.

[0039] Add a kernel thread to monitor memory usage and file read rate, and dynamically and adaptively adjust the maximum physical memory usage of the page cache based on memory usage and file read rate.

[0040] According to the method of the first aspect of the present invention, when the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; wherein:

[0041] When the kernel page cache management thread performs page cache reclamation, it calculates the activity level of the page cache pages and reclaims them first, prioritizing the reclamation of page cache pages with an activity level below the first threshold and marked with a memory page request flag.

[0042] At the same time, the increased kernel threads are used to monitor memory usage and file read rate; if the I / O operations in the system exceed the second threshold, the file read hit rate is further monitored; if the hit rate exceeds the third threshold, the maximum amount of physical memory used by the page cache is increased.

[0043] According to the method of the first aspect of the present invention, when physical memory is insufficient, kswapd is woken up to reclaim physical memory, and the kernel page cache management thread is woken up to reclaim the page cache. The increased kernel thread is used to reduce the maximum amount of physical memory occupied by the page cache based on the statistical memory usage and file read rate.

[0044] According to the method of the first aspect of the present invention, when requesting a page cache:

[0045] During the initialization phase of kernel memory management, the maximum amount of physical memory used by the page cache is configured by default to be equal to the low watermark WMARK_LOW.

[0046] When a received memory request is a page cache request, the GFP_PAGECACHE flag is marked, and it is further determined whether the requested page cache exceeds the maximum usage. If not, the page cache is allocated, and the memory request is successful.

[0047] According to the method of the first aspect of the present invention, it is determined whether the requested page cache exceeds the maximum usage; if so, the kernel page cache management thread is woken up to reclaim the page cache.

[0048] In some embodiments (such as) Figure 1 As shown in the figure, the page cache limiting method based on active memory reclamation provided by the present invention mainly includes: adding a memory page request flag; dynamically limiting the maximum amount of physical memory that the page cache can use; and a reclamation strategy when the page cache exceeds the limit.

[0049] One of the additions is a page request flag, namely GFP_PAGECACHE, which is used for requesting page caches. This flag is used for any page cache request, making it easier to manage page caches in a unified manner.

[0050] The dynamic limitation on the page cache's maximum page usage is achieved through two methods. First, a kernel sysfs file, ` / proc / sys / pagecache_ratio`, is added. This file sets a percentage representing the maximum percentage of total physical memory that the page cache can utilize. This value is initialized to match the low watermark, ensuring that the maximum available page cache level remains consistent with the low watermark. Second, a kernel thread is added to monitor memory usage and file read rates, dynamically and adaptively adjusting the maximum page cache limit based on memory usage.

[0051] The page cache reclamation strategy when it exceeds the limit includes:

[0052] When the page cache reaches its maximum limit, the kernel page cache management thread is woken up.

[0053] The page cache management thread reclaims the page cache, prioritizing inactive pages based on their activity level.

[0054] Meanwhile, the kernel thread tracks memory usage, including how much memory is used and the nature of memory usage (e.g., whether there are more I / O operations or other types of operations). If there are many I / O operations in the system, the kernel thread further tracks the file read hit rate. If the hit rate is low, the kernel thread appropriately increases the page cache memory usage ratio.

[0055] When memory is insufficient, kswapd is awakened to perform memory reclamation. At the same time, the page cache management thread is also awakened to collect memory usage data, perform page cache reclamation, and appropriately reduce the page cache memory usage ratio.

[0056] As can be seen, this invention sets a limit on the maximum usage of the page cache, including adding a kernel sysfs file ` / proc / sys / vm / pagecache_ratio` to set the maximum ratio of physical memory occupied by the page cache, and adding a memory page allocation flag `GFP_PAGECACHE`. This invention uses a page cache overload handling strategy, including adding a page cache management thread, adaptively setting the maximum ratio of memory that the page cache can occupy, and prioritizing the reclamation of inactive pages when reclaiming the page cache. The page cache management thread proposed in this invention includes functions for statistically analyzing memory usage, statistically analyzing file read rates, and adaptively adjusting the maximum page cache ratio based on memory usage and read rates. The page cache management thread proposed in this invention statistically analyzes the activity of page cache pages, divides page cache pages into gradients based on activity, and reclaims page cache pages in a gradient from inactive to active during page cache reclamation.

[0057] A second aspect of this invention proposes a Linux page cache limiting system based on proactive page reclamation, the system comprising a processing unit configured to:

[0058] Configure the memory page allocation flag during the page caching process;

[0059] Dynamically limit the maximum amount of physical memory used by the page cache; this is achieved by setting the maximum amount of physical memory used by the page cache through the kernel sysfs file, and by dynamically and adaptively adjusting the maximum amount of physical memory used by the page cache through the addition of kernel threads.

[0060] When the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; however, only page caches with memory page request flags are reclaimed, not anonymous pages in Linux memory.

[0061] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements a Linux page cache limiting method based on proactive cache reclamation, as disclosed in the first aspect of this invention.

[0062] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements a Linux page cache limitation method based on proactive cache reclamation, as described in the first aspect of this disclosure.

[0063] In summary, this invention provides a Linux page cache limiting scheme based on proactive reclamation, including setting page cache quotas; adding a new page request flag; and a memory reclamation strategy when the page cache is overloaded. Using this scheme, the page cache quota can be adaptively adjusted according to memory pressure and file read efficiency. When memory pressure is high, the page cache quota ratio is lowered, thus reducing page cache memory usage. When file read efficiency decreases and memory pressure is low, the page cache quota ratio is increased, thus reducing page swapping speed and improving file read hit rate. This invention improves the flexibility, accuracy, and stability of page cache management, and compensates for the shortcomings of existing mechanisms in some complex scenarios.

[0064] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A Linux page cache limiting method based on active page reclamation, characterized in that, The method includes: Configure the memory page allocation flag during the page caching process; Dynamically limit the maximum physical memory usage of the page cache; this is achieved by setting the maximum physical memory usage of the page cache through the kernel sysfs file, and by dynamically and adaptively adjusting the maximum physical memory usage of the page cache through the addition of kernel threads. When the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; however, only page caches with memory page request flags are reclaimed, not anonymous pages in Linux memory. In the method, when the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; wherein: When the kernel page cache management thread performs page cache reclamation, it calculates the activity level of the page cache pages and reclaims them first, prioritizing the reclamation of page cache pages with an activity level below the first threshold and marked with a memory page request flag. At the same time, the increased kernel threads are used to monitor memory usage and file read rate; if the I / O operations in the system exceed the second threshold, the file read hit rate is further monitored; if the hit rate is lower than the third threshold, the maximum amount of physical memory used by the page cache is increased. In the method, when physical memory is insufficient, kswapd is woken up to reclaim physical memory, and the kernel page cache management thread is woken up to reclaim the page cache. The added kernel thread is used to reduce the maximum amount of physical memory occupied by the page cache based on the statistical memory usage and file read rate.

2. The Linux page cache limiting method based on active reclamation according to claim 1, characterized in that, Configure memory page allocation flags during the page cache allocation process; specifically: add a memory page allocation flag GFP_PAGECACHE for page cache pages. All actions involving page cache allocation are marked with GFP_PAGECACHE, and page cache is managed uniformly.

3. The Linux page cache limiting method based on active reclamation according to claim 2, characterized in that, Dynamically limit the maximum amount of physical memory used by the page cache; where: Add the kernel sysfs file / proc / sys / pagecache_ratio to set the maximum amount of physical memory used by the page cache. This maximum usage is consistent with the low watermark WMARK_LOW during initialization. Add a kernel thread to monitor memory usage and file read rate, and dynamically and adaptively adjust the maximum physical memory usage of the page cache based on memory usage and file read rate.

4. The Linux page cache limiting method based on active reclamation according to claim 3, characterized in that, In the method described above, when requesting page caching: During the initialization phase of kernel memory management, the maximum amount of physical memory used by the page cache is configured by default to be equal to the low watermark WMARK_LOW. When a received memory request is a page cache request, the GFP_PAGECACHE flag is marked, and it is further determined whether the requested page cache exceeds the maximum usage. If not, the page cache is allocated, and the memory request is successful.

5. The Linux page cache limiting method based on active reclamation according to claim 4, characterized in that, In the method, it is determined whether the requested page cache exceeds the maximum usage. If so, the kernel page cache management thread is woken up to reclaim the page cache.

6. A Linux page cache limiting system based on active page reclamation, characterized in that, The system includes a processing unit, which is configured to: Configure the memory page allocation flag during the page caching process; Dynamically limit the maximum physical memory usage of the page cache; this is achieved by setting the maximum physical memory usage of the page cache through the kernel sysfs file, and by dynamically and adaptively adjusting the maximum physical memory usage of the page cache through the addition of kernel threads. When the page cache exceeds the maximum usage, the kernel page cache management thread is woken up to reclaim the page cache; however, only page caches with memory page request flags are reclaimed, not anonymous pages in Linux memory. The processing unit is further configured to, when the page cache exceeds the maximum usage, wake up the kernel page cache management thread and reclaim the page cache; wherein: When the kernel page cache management thread performs page cache reclamation, it calculates the activity level of the page cache pages and reclaims them first, prioritizing the reclamation of page cache pages with an activity level below the first threshold and marked with a memory page request flag. At the same time, the increased kernel threads are used to monitor memory usage and file read rate; if the I / O operations in the system exceed the second threshold, the file read hit rate is further monitored; if the hit rate is lower than the third threshold, the maximum amount of physical memory used by the page cache is increased. The processing unit is also configured to, when physical memory is insufficient, wake up kswapd to perform physical memory reclamation, and simultaneously wake up the kernel page cache management thread to perform page cache reclamation, and use the added kernel thread to reduce the maximum amount of physical memory occupied by the page cache based on the statistical memory usage and file read rate.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the Linux page cache limitation method based on active reclamation as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements a Linux page cache limitation method based on active reclamation as described in any one of claims 1-5.

Images