A three-level cache replacement method based on dirty page awareness and adaptive capacity

Abstract

The application discloses a three-level cache replacement method based on dirty page awareness and adaptive capacity, and belongs to the technical field of computer memory cache management. The method comprises the following steps: constructing short-time queues, medium-time queues and global queues on the side of a CXL extended memory device cache; performing object insertion, migration and metadata update according to the hit condition of each queue; when the cache space is insufficient, performing eviction determination in combination with access count, dirty page flag and dirty page revival flag; calculating the revisit ratio based on the global queue identification insertion frequency and revisit frequency, and dynamically adjusting the short-time queue capacity and the medium-time queue capacity. The application solves the problems of high dirty page write-back cost and inflexible cache space allocation in the CXL extended memory, optimizes the access characteristics, dirty page state and elimination revisit behavior in cooperation, reduces the redundant write-back overhead, improves the cache space utilization efficiency and reduces the access time delay.

Description

Technical Field

[0001] This invention relates to cache management technology in computer memory, specifically to a three-level cache replacement method based on dirty page awareness and adaptive capacity. Background Technology

[0002] With the increasing demands of data-intensive applications and large-scale model computations, traditional memory architectures based on local DRAM are gradually facing bottlenecks in terms of capacity and scalability. Compute Express Link (CXL) provides a feasible way to overcome memory capacity limitations by enabling processors to access extended memory devices in a near-memory manner by providing consistent access semantics. However, compared to local DRAM, CXL extended memory still has significant differences in access latency and bandwidth characteristics, especially when cache misses require access to slower backend storage media, which further exacerbates these differences. As the first line of defense against access latency, the design of caching strategies plays a crucial role in the overall system performance. Cache replacement strategies not only affect the hit rate but also directly determine the frequency of high-latency accesses. Especially in write-intensive workloads, a reasonable cache replacement strategy can reduce the frequency of dirty page write-backs, further reducing access latency.

[0003] Existing cache replacement research widely employs Least Recently Used (LRU), Least Frequently Used (LFU), and their variants, primarily using recentity or frequency as the core signal. While these strategies theoretically capture locality well, their implementation often relies on linked list maintenance, frequency counting, and reordering operations. The metadata maintenance and update paths become additional overhead sources, making it difficult to meet the low-overhead requirements of device-side caching. In contrast, First-In-First-Out (FIFO) replacement strategies, using sequential enqueueing and dequeueing as their main mechanism, have a simpler structure, lower maintenance costs, and greater engineering feasibility. However, a single FIFO strategy often lacks effective differentiation between hot and one-off data, easily leading to cache pollution under mixed access modes, thus limiting system performance.

[0004] To simultaneously achieve simplicity and hotspot identification, the S3-FIFO algorithm, proposed in recent years, uses a multi-level FIFO queue and a lightweight access counting mechanism to filter newly arriving data and protect frequently accessed data. While maintaining a simple implementation path, it achieves good cross-load performance and is therefore considered to have high engineering practical value. However, it should be noted that the original design of S3-FIFO was mainly aimed at general caching scenarios, and its replacement decision does not distinguish between clean and dirty pages. This can lead to frequent eviction of dirty pages during write hits, causing additional write-back overhead. Furthermore, its queue allocation is statically set, which can easily lead to insufficient queue capacity utilization and performance fluctuations when the load changes periodically or the access patterns are mixed.

[0005] This invention combines the features of current CXL extended memory cache replacement methods, introduces a dirty page awareness mechanism to reduce write-back overhead, and dynamically adjusts the capacity of each queue to adapt to different load characteristics. It proposes a three-level cache replacement method for CXL extended memory with dirty page awareness and adaptive capacity, which effectively solves the problem of additional overhead caused by dirty page write-back and improves system memory access performance. Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] The purpose of this invention is to provide a three-level cache replacement method based on dirty page awareness and adaptive capacity, which reduces the frequent write-back of dirty pages and balances capacity, thereby reducing cache pollution and average access latency.

[0008] (II) Technical Solution

[0009] This invention includes the following steps:

[0010] Step 1: Constructing a three-level cache queue structure.

[0011] The DRAM cache in the CXL extended memory is divided into cache data areas, which are further divided into three levels of cache queues: S (short-term), M (medium-term), and G (global). A status field is set for each accessed object.

[0012] Step 2: Cache hit handling.

[0013] When an object access hits queue S or M, its access count is incremented. When an access hits queue G, the revisit ratio in that queue is recorded, and the object data is inserted at the head of queue M; if the access does not hit queue G, it is considered a new object, and the object data is inserted at the head of queue S.

[0014] Step 3: Cache eviction determination.

[0015] When the queue capacity is full, cache eviction is triggered. During eviction on the S queue side, candidate objects are taken from the tail of the S queue. If the access count of a candidate object is greater than or equal to 1, it is moved to the head of the M queue for hotspot protection; otherwise, its identifier is written to the G queue and it is evicted from the cache data area. During eviction on the M queue side, to reduce bandwidth consumption caused by dirty page write-backs, a dirty page-aware eviction strategy is adopted. Candidate object t is taken from the tail of the M queue, and different eviction operations are performed based on the state information of t.

[0016] Step 4: Adaptive adjustment of queue capacity.

[0017] The revisit ratio of objects in the monitored G queue is constructed as a lightweight feedback signal, and this signal is used as parameter information for adjusting the capacity of the S and M queues. After obtaining the adjustment results, the queue capacity is adjusted online as required.

[0018] (III) Beneficial Effects

[0019] The advantages of this invention are as follows:

[0020] This invention innovatively utilizes dirty page awareness to limit the frequency of dirty page writebacks and achieves adaptive capacity adjustment by adjusting the revisit ratio of objects in the G queue. This reduces cache pollution, increases the proportion of fast paths, and effectively reduces the average access latency of memory access in the CXL extended memory system. Attached Figure Description

[0021] Figure 1 This is a flowchart of the three-level cache replacement method based on dirty page awareness and adaptive capacity proposed in this invention.

[0022] Figure 2 This is a schematic diagram of the three-level cache replacement method based on dirty page awareness and adaptive capacity proposed in this invention.

[0023] Figure 3 This is a schematic diagram illustrating the cache hit rate under different loads in the simulation system of this invention.

[0024] Figure 4 This is a schematic diagram showing the proportion of sub-microsecond delays under different loads in the simulation system of this invention.

[0025] Figure 5 This is a schematic diagram illustrating the average access latency under different loads in the simulation system of this invention. Detailed Implementation

[0026] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings:

[0027] Reference Figure 1 The specific implementation steps of the present invention are as follows:

[0028] Step 1: Constructing a three-level cache queue structure.

[0029] Two FIFO queues for storing data objects are maintained in the device-side DRAM: the S queue and the M queue, and a G queue that only stores metadata identifiers. The S queue is used to accept newly arriving objects and filter one-time accesses, the M queue is used to store hot objects that are more likely to be reused, and the G queue is used to record the identifiers of recently evicted objects to capture revisit behavior after evicting.

[0030] Each object's metadata includes: key (address / page number), access count freq (2-bit saturation, range 0~3), dirty page flag (1 bit), dirty page rescued flag (1 bit), and a queue pointer used to maintain the FIFO queue.

[0031] Configure system monitoring counters: the number of times objects are inserted into queue G (gins), the number of times objects from queue G are added to queue M (ghit), the scan window (scan), and the trend status, including UP, DOWN, and NONE.

[0032] Step 2: Cache hit handling.

[0033] Figure 2 The three-level queue caching strategy embodies the cache hit processing process.

[0034] When an object access request is received, the system first searches the S and M queues based on the object identifier key. If the target object is in the S queue, it is considered a hit in the S queue; if the target object is in the M queue, it is considered a hit in the M queue. For a hit object, the FIFO relative order of the corresponding queue is not changed; only its metadata status is updated. Specifically, the access count (freq) of the object is incremented by one, and a 2-bit saturation counting method is used to ensure that its maximum value does not exceed 3. If the request is a write request, the dirty page flag (dirty) of the object is set to 1; otherwise, the original state remains unchanged.

[0035] When a requested object is not found in either the S or M queues, its identifier is further searched in the G queue. If the object identifier exists in the G queue, a G queue hit is considered to have occurred. A G queue hit indicates that although the object has been cached and evicted, it has been accessed again within a short period of time, suggesting it has some reuse value. At this point, the object's identifier is removed from the G queue, and the object is reloaded to the tail of the M queue to increase its probability of subsequent access hits. Simultaneously, the access count `freq` is initialized to a value that characterizes its re-access characteristics, and the `dirty` flag is updated according to the request type. If the object previously entered the G queue due to dirty page eviction, or its corresponding metadata carries dirty page revival information, its dirty page revival flag `rescued` is set to 1, indicating a preference for retaining the object in subsequent eviction decisions. At the same time, the system monitoring counter `ghit` is incremented for adaptive adjustment of subsequent queue capacity.

[0036] If the requested object is not found in the S, M, and G queues, it is considered a completely invalid access. In this case, the object is inserted as a new object at the tail of the S queue, and its metadata is initialized. Specifically, `freq` is set to its initial value, `dirty` is set according to the access type, and `rescued` is set to 0. If the insertion operation causes insufficient space in the S queue, a subsequent cache eviction process is triggered.

[0037] Step 3: Cache eviction determination.

[0038] When queue S or queue M reaches its capacity limit due to the insertion of new objects, a cache eviction process is performed to determine the objects to be evicted and their subsequent processing. Cache eviction follows the basic dequeue order of FIFO queues, prioritizing the selection of candidate objects from the head of the target queue where space is insufficient, and combining the object's access count (freq), dirty page flag (dirty), and dirty page resurrection flag (rescued) for a comprehensive determination.

[0039] When queue S needs to free up space, it first checks the object at the head of the queue. If the object's access count (freq) is low, indicating that it has not been significantly reused after entering the cache, it is treated as an object to be evicted. If the object's access count (freq) reaches a preset threshold, indicating that it has shown certain hotspot characteristics, it can be moved to the tail of queue M to avoid prematurely evicting objects with reuse potential. If queue M is insufficient when moving to queue M, it further triggers the eviction decision in queue M. For objects that are eventually evicted, if their dirty value is 0, only their identifier is written to the tail of queue G to record recent eviction history; if their dirty value is 1, a write-back operation is performed first, then their identifier is written to the tail of queue G, and the Gins counter is incremented. If queue G is full, the old identifier at the head of queue G is evicted in a FIFO manner to maintain the limited recording window of queue G.

[0040] When queue M needs to free up space, candidate objects are selected from the head of queue M, prioritizing the dirty page attribute and resurrection status of the objects. If the dirty value of a candidate object is 0, it can be directly evicted, and its identifier is inserted at the tail of queue G. If the dirty value is 1 and the rescued value is 0, it means that although the object is a dirty page, it does not show obvious characteristics of being accessed again after being evicted, and it can be evicted as a normal dirty page after being written back, and its identifier is inserted at the tail of queue G. If the dirty value is 1 and the rescued value is 1, it means that the object has been accessed again after being evicted and has high reuse value. To reduce the overhead caused by repeated writing back of dirty pages, it can be given delayed eviction processing, that is, continue scanning the subsequent objects in queue M, and within the predetermined scanning range, prioritize the selection of objects without the rescued flag or clean pages as alternative eviction targets.

[0041] If a more suitable object for eviction is found within the predetermined scan range, the replacement object is evicted; otherwise, the original candidate object is still evicted, ensuring that the queue eviction process is bounded and controllable. Through this method, dirty pages, especially those that have been accessed again, are protected during eviction without compromising the main FIFO structure and with low implementation complexity, thereby reducing performance losses caused by redundant write-backs and repeated loading.

[0042] Step 4: Adaptive adjustment of queue capacity.

[0043] To ensure that cache space allocation adapts to different access hotspot patterns, the capacity ratio of the S queue and M queue is dynamically adjusted based on G queue behavior statistics during system operation. Specifically, using a preset scan window (scan) as an adjustment period, the number of insertions into the G queue (gins) and the number of times G queue objects re-enter the M queue (ghit) are continuously counted within each adjustment period. The revisit intensity of currently evicted objects is calculated based on the relationship between these two counts. The revisit ratio of the G queue is calculated as follows:

[0044] ,

[0045] Where ghit represents the number of times a G queue object re-enters the M queue; gins represents the number of times an object identifier is inserted into the G queue. To prevent extremely small constants with a denominator of zero.

[0046] A high return visit rate indicates that recently evicted objects are frequently accessed again, and the current cache is not capable of retaining objects with reuse potential. When ghit is relatively lower than gins, it indicates that most evicted objects are not accessed again, and the current cache may be compressing the space for accepting new objects by retaining too many historical objects.

[0047] Based on the above statistical results, the queue capacity is adjusted by combining the trend status (trend). When ghit increases or remains at a high level for several consecutive adjustment periods, the trend is set to UP, and the capacity of queue M is appropriately increased while the capacity of queue S is decreased to enhance the retention capacity of reused objects. When ghit decreases or remains at a low level for a long period for several consecutive adjustment periods, the trend is set to DOWN, and the capacity of queue S is appropriately increased while the capacity of queue M is decreased to improve the acceptance and filtering capacity of newly entered objects. When the statistical results do not show significant changes, the trend is set to NONE, and the current capacity division remains unchanged. Upper and lower limits are set during the capacity adjustment process to ensure that the capacities of queue S and queue M are always within the preset range, thereby avoiding system oscillations caused by frequent adjustments.

[0048] After an adjustment cycle ends, the gins and ghit counters are cleared or reset to enter the next statistical cycle. Through this capacity adaptive mechanism based on the return behavior and changing trends of the G queue, the caching system can dynamically coordinate the filtering function of the S queue and the retention function of the M queue under different load conditions such as a high proportion of one-time accesses, frequent switching of short-term hot spots, and stable reuse as the main feature. This improves cache space utilization and reduces access latency caused by unreasonable eviction and repeated write-back of dirty pages.

[0049] Example

[0050] The effects of the present invention will be further illustrated by the following data:

[0051] The cache replacement strategy is based on an experimental environment, the CXL-SSD, which effectively simulates the behavior of an SSD connected via a CXL. The CXL-SSD has a DRAM latency of 46ns, a cache size of 64MB, a page size of 16KB, and a backend storage medium capacity of 1TB, using SLC flash memory. Seven different workloads were used in the experiment to evaluate the caching algorithm: hash map (Hash), matrix multiplication (MM), min-heap (MM), Random, Stride, Bert, and XZ. These workloads were generated by memory tracing tools. To ensure the stability of the experiment, the reliability of the results, and the performance changes of the algorithm over long periods, the total number of requests for each workload was set to 20 million. Figure 3The figure compares the cache hit rates of different caching strategies under various workloads. As can be seen, the present invention achieves the highest or tied-highest hit rates across most workloads, especially under workloads such as MM, Stride, and XZ, where its hit advantage is more stable. This indicates that the present invention, through more reasonable replacement decisions, can more effectively retain data objects that contribute more to access and reduce the frequency of access to slow paths. For workloads with strong locality, such as Hash, Heap, and Stride, the overall hit rates of each strategy are high and the differences are narrowing, but the present invention still maintains its leading hit performance, demonstrating its robustness under different levels of locality. Figure 4 Further analysis of the sub-microsecond latency request ratio characterizes whether requests are effectively accelerated to the fast path of the device-side DRAM. Experimental results show that this invention significantly improves the sub-microsecond latency ratio in most workloads except Random. For example, under workloads such as Hash, Heap, Stride, BERT, and XZ, the ratio corresponding to this invention is generally higher than other strategies, indicating that it not only increases the hit count, but more importantly, enables a higher proportion of accesses to be completed with lower latency, thereby migrating request distribution to the low-latency range. Figure 5 The average access latency of the system under different caching strategies is shown, which can more directly reflect the ability of cache replacement strategies to improve access requests in expanded memory. It can be seen that the present invention achieves the lowest or near-lowest average latency under most workloads.

[0052] The above embodiments of the present invention are merely examples for clearly illustrating the present invention and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively describe all possible implementations here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

[0053] The innovation of this invention is reflected in the following aspects:

[0054] This invention addresses the issues of high slow-path access costs and write-back overhead in CXL extended memory scenarios. Starting from the runtime metadata of the device-side DRAM cache, it constructs a lightweight feedback signal using the return ratio of the G queue, enabling online discrimination and adaptive adjustment of cache filtering strength and hotspot protection strength. Based on this discrimination result, it further utilizes the relationship between the target ratio and queue occupancy to dynamically select the S queue and Main queue for eviction, thereby maintaining a stable fast-path ratio under different load locality and phase change conditions. Simultaneously, this invention introduces a dirty page awareness mechanism during Main queue eviction, leveraging the cost difference between clean pages requiring no write-back and dirty pages triggering write-back. Combined with scan window constraints and a one-time dirty page revival strategy, it achieves clean-priority eviction and write-back jitter suppression, thereby reducing device-side queuing and tail latency. In summary, the cache replacement proposed in this invention, based on the runtime statistical characteristics and write-back cost characteristics available on the device side, overcomes the shortcomings of traditional replacement strategies such as fixed queue ratios and neglecting dirty page write-back amplification. It exhibits high engineering practicality and cross-load robustness while maintaining constant-level metadata maintenance overhead.

Claims

1. A three-level cache replacement method based on dirty page awareness and adaptive capacity, characterized in that, include: (1) Construction of a three-level cache queue structure: S queue, M queue and G queue are maintained in the CXL extended memory device-side cache; wherein, the S queue is used to accept newly entered objects and filter one-time access, the M queue is used to store objects that are more likely to be accessed again, and the G queue is used to record the identifiers of recently evicted objects to represent revisit behavior; at the same time, metadata is maintained for each cache object, the metadata including at least object identifier key, access count freq, dirty page flag dirty, dirty page resurrection flag rescued and pointer information for maintaining the queue; (2) Cache hit handling: When an object access request is received, the object is searched in the S queue and M queue according to the object identifier key; if the target object is hit in the S queue or M queue, the access count freq and dirty page flag of the object are updated; if the target object is not hit in the S queue and M queue but hit in the G queue, the object is reloaded from the backend storage to the M queue, and its access status information and dirty page resurrection flag are updated; if the target object is not hit in the S queue, M queue and G queue, the object is inserted into the S queue. (3) Eviction determination: When the S queue or M queue reaches the corresponding capacity limit, candidate objects are selected from the head of the corresponding queue for eviction determination; the eviction determination is combined with the object's access count freq, dirty page flag dirty and dirty page resurrection flag rescued; for objects that are finally evicted, their flags are inserted into the G queue to record recent eviction history; for dirty page objects, a write-back operation is performed before eviction. (4) Adaptive adjustment of queue capacity: The number of times the object identifier is inserted into the G queue (gins) and the number of times the object in the G queue re-enters the M queue (ghit) are counted. The return visit ratio is calculated based on ghit and gins. The return visit ratio is used as the statistical basis to characterize the return visit intensity of recently eliminated objects. The capacity allocation of the S queue and the M queue is dynamically adjusted based on the return visit intensity to adapt to the caching needs under different access hotspot modes.

2. The three-level cache replacement method based on dirty page awareness and adaptive capacity as described in claim 1, characterized in that, The three-level cache queue structure is constructed, including: In the device-side DRAM, maintain S queue and M queue as actual data object storage queues, and maintain G queue as a historical record queue that only stores object identifiers; For each cached object, maintain an object identifier key, access count freq, dirty page flag dirty, dirty page rescued flag rescued, and a queue pointer; among them, the access count freq uses a 2-bit saturation count, with a value range of 0~3; Configure system monitoring counters, including the number of times an object identifier is inserted into queue G (gins), the number of times an object in queue G re-enters queue M (ghit), the statistics window (scan), and the trend status (trend).

3. The three-level cache replacement method based on dirty page awareness and adaptive capacity as described in claim 1, characterized in that, Cache hit handling includes: When a target object hits either queue S or queue M, the access count freq of the target object is updated according to the following rules: ， If this access request is a write request, then the dirty page flag of the target object is set to 1; When a target object misses queues S and M but hits queue G, the identifier corresponding to the target object is removed from queue G, and the target object is reloaded to the tail of queue M; at the same time, the ghit counter is updated, and the dirty page flag of the target object is updated according to the access type; if the target object is a dirty page object that has been evicted and accessed again, its dirty page revival flag is set to 1. If the target object is not found in the S queue, M queue, and G queue, the target object is inserted as a new object at the tail of the S queue, and its access status information is initialized.

4. The three-level cache replacement method based on dirty page awareness and adaptive capacity as described in claim 1, characterized in that, Cache eviction determination includes: When the space in queue S is insufficient, a candidate object is selected from the head of queue S; if the access count of the candidate object reaches a preset threshold, the candidate object is moved to the tail of queue M; if the access count of the candidate object does not reach the preset threshold, it is treated as a discarded object and its identifier is inserted into queue G. When the space in queue M is insufficient, a candidate object is selected from the head of queue M, and the candidate object's dirty page flag and dirty page resurrection flag are used to perform an eviction decision. For clean page objects, they are directly evicted and their flags are inserted into queue G. For dirty page objects, a write-back operation is performed first, and then their flags are inserted into queue G. When the object to be eliminated is a dirty page revival object, delayed elimination processing is performed on it, and clean page objects or objects without dirty page revival flags are selected as replacement elimination objects within the predetermined scanning range. Whenever an object identifier is inserted into the G queue, the gins counter is updated; when the G queue reaches its capacity limit, the oldest object identifier at the head of the G queue is removed in a first-in, first-out manner.

5. The three-level cache replacement method based on dirty page awareness and adaptive capacity as described in claim 4, characterized in that, The delayed elimination process includes: When a candidate eviction object satisfies both dirty and rescued values ​​of 1, a write-back eviction is not immediately performed on the candidate eviction object. Instead, a replacement eviction object is searched within a predetermined scan range. If a clean page object or an object with rescued value of 0 is found, the replacement eviction object is prioritized for eviction. If no replacement eviction object is found, the original candidate eviction object is evictioned.

6. The three-level cache replacement method based on dirty page awareness and adaptive capacity as described in claim 1, characterized in that, Queue capacity adaptive adjustment, including: Using the statistical window scan as the adjustment period, count the number of times an object identifier is inserted into queue G and the number of times an object in queue G re-enters queue M within the adjustment period (gins). The G-queue revisit ratio of recently evicted objects is calculated based on gins and ghit, using the following formula: ， in, To prevent constants with a denominator of zero; Set a trend state and adjust the capacity of the S and M queues based on the G queue return ratio and the trend state. When the return visit rate of queue G is higher than a preset range, the capacity of queue M is increased and the capacity of queue S is decreased; when the return visit rate of queue G is lower than a preset range, the capacity of queue S is increased and the capacity of queue M is decreased.

7. A three-level cache replacement method based on dirty page awareness and adaptive capacity as described in claim 6, characterized in that, The trend state is used to characterize the changing trend of the G queue return rate within multiple consecutive statistical windows. When the G queue return rate increases continuously, the trend is set to UP; when the G queue return rate decreases continuously, the trend is set to DOWN; when the G queue return rate does not change significantly, the trend is set to NONE. The trend state is used in conjunction with the trend state to constrain the capacity adjustment direction of the S queue and the M queue.

Images