Separated memory architecture-oriented swap space cache optimization system and method

Abstract

The invention relates to a separate memory architecture-oriented swap space cache optimization system and method, and the method comprises the steps: receiving a page swap request sent by a host memory space, and converting the page swap request into a swap space memory access request; a cardinal number tree cache index is generated according to the logic block number in the exchange space memory access request, searching is conducted according to the index, and if searching succeeds, a data block is obtained from a cache space and sent to a host memory space; if the search is not successful, adopting an asynchronous active cache replacement strategy to select a data block from the original cache region, writing the data block back to the remote memory pool, and meanwhile, adopting a dynamic pre-fetching strategy based on historical access to read pre-fetching data from the remote memory, and storing the pre-fetching data into a pre-fetching cache region. On the premise that an existing switching mechanism of an operating system is not changed, the access efficiency of the switching space of the separated memory is improved, the access frequency of the remote memory is reduced, and therefore the memory access time delay of the whole system is reduced, and the I / O performance of the system is improved.

Description

Technical Field

[0001] This application relates to the field of wireless communication, and more specifically, to a swap space cache optimization system and method for discrete memory architectures. Background Technology

[0002] In current discrete memory architectures, remote memory resources are typically provided to compute nodes via high-speed networks, such as RDMA (remote direct memory access), and are often configured as the operating system's swap space to alleviate the problem of limited local physical memory capacity. However, due to the existence of cross-node network communication, the access latency of remote memory is usually significantly higher than that of local memory.

[0003] In existing technologies, cache management strategies for swap space mostly follow general cache management mechanisms, typically employing passive replacement algorithms such as Least Recently Used (LRU) or Least Frequently Used (LFU) to manage cached data.

[0004] However, in the context of split memory swapping, swap space access requests exhibit characteristics distinct from ordinary data access in terms of access patterns, granularity, and timing. Existing cache management strategies fail to differentiate between these differences, potentially leading to inefficient cache resource allocation and reduced cache utilization.

[0005] Furthermore, traditional cache replacement strategies primarily rely on access history for passive eviction. In high-concurrency swap request scenarios, this may introduce additional replacement latency, thereby increasing overall memory access overhead. Meanwhile, existing prefetching mechanisms often employ fixed strategies, making it difficult to adjust according to dynamic changes in swap space access patterns, which can easily lead to poor prefetching performance or resource waste. Summary of the Invention

[0006] To overcome at least one deficiency in the prior art, this application provides a swap space cache optimization system and method for discrete memory architectures.

[0007] Firstly, a swap space cache optimization method for discrete memory architectures is provided, including:

[0008] The cache space is logically divided into the raw cache area and the prefetch cache area; It receives page swapping requests sent from the host memory space and converts them into swap space memory access requests. It generates a radix tree cache index based on the logical block number in the swap space memory access request, performs a lookup based on the index, and if the lookup is successful, it retrieves the data block from the cache space and sends it to the host memory space. If the lookup is unsuccessful, it adopts an asynchronous proactive cache replacement strategy to select the data block from the original cache area and write it back to the remote memory pool. At the same time, it adopts a dynamic prefetch strategy based on historical access to read prefetched data from the remote memory and store it in the prefetch cache area.

[0009] In one embodiment, the method further includes: Continuously monitor the access hit rate of swap space memory access requests in the cache space, and count the number of hits in the original cache region and the number of hits in the prefetch cache region; Calculate the adjustment value of the cache space allocation ratio based on the number of hits in the original cache region and the number of hits in the prefetch cache region; The space ratio of the original cache area and the prefetch cache area is dynamically adjusted based on the adjustment value of the cache space allocation ratio.

[0010] In one embodiment, the adjustment value for the cache space allocation ratio is calculated using the following formula:

[0011] in, Adjustment value for the proportion of cache space allocation. To prefetch the number of cache hits, This represents the number of hits in the original cache region. This represents the number of requests.

[0012] In one embodiment, the asynchronous proactive cache replacement strategy is as follows: If the search fails, and if the cache space is full and a passive replacement is triggered, the counter value is incremented by 1. If the search is successful, the calculator will be reset to zero and the counting will restart. When the counter accumulates to a preset empirical threshold N, an asynchronous active eviction task is immediately initiated. The asynchronous active eviction task includes: selecting multiple data blocks to be replaced from the active eviction range of the original cache area, compressing the data blocks, and then writing them back to the remote memory pool in batches using RDMA technology.

[0013] In one embodiment, a dynamic prefetching strategy based on historical access includes: Analyze the logical block numbers in the historical swap space access request sequence to determine the dominant trend in the access pattern; Calculate the sequence of target addresses to be prefetched based on the requested addresses, prefetch granularity, and dominant trends in the current swap space memory access requests. Based on the target address sequence to be prefetched, RDMA technology is used to read the prefetched data from the remote memory pool and store it in the prefetch cache area.

[0014] In one embodiment, analyzing the logical block numbers in a historical swap space access request sequence to determine the dominant trend in access patterns includes: Maintain a fixed-size FIFO (First Input First Output) queue to store the difference in logical block number between adjacent requests in the historical swap space memory access request sequence; A variable window mechanism is used to analyze the difference sequence in the FIFO queue. If a certain difference appears more than half the number of times in the window, that difference is the dominant trend.

[0015] In one embodiment, the dynamic prefetching strategy based on historical access further includes: Prefetch accuracy and prefetch coverage are calculated in real time based on the number of prefetch cache hits, using the following formula:

[0016]

[0017] in, To improve the accuracy of pre-fetching, For prefetch coverage, To prefetch the number of cache hits, To prefetch the number of cache misses, To request the number of misses; If prefetch accuracy improves, increase the prefetch granularity; if prefetch coverage decreases, decrease the prefetch granularity.

[0018] Secondly, a swap space cache optimization system for discrete memory architectures is provided to implement the above-mentioned method.

[0019] Compared with the prior art, this application has the following beneficial effects: The swap space cache optimization system and method for split memory architecture of this application can improve the access efficiency of split memory swap space and reduce the number of accesses to remote memory without changing the existing swap mechanism of the operating system, thereby reducing the overall system memory access latency and improving system I / O performance. Attached Figure Description

[0020] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings: Figure 1The overall architecture diagram of a swap space cache optimization system for a discrete memory architecture is shown. Figure 2 A schematic diagram of the prefetch cache adaptive allocation strategy is shown. Figure 3 A diagram comparing active and passive cache replacement strategies is shown. Figure 4 A schematic diagram of the prefetch module workflow is shown. Detailed Implementation

[0021] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0022] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution of this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0023] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0024] This application aims to address the problem that existing cache management strategies are unable to effectively distinguish and utilize the characteristics of swap space access in a split memory architecture, resulting in low cache utilization efficiency and high access latency. It provides a swap space cache optimization system and method for split memory architectures to improve overall access efficiency and system performance in split memory swap space scenarios.

[0025] Figure 1 The overall architecture diagram of a swap space cache optimization system for a discrete memory architecture is shown. This cache system is deployed on the Data Processing Unit (DPU) side to cache swap space access data from compute nodes and to perform feature-aware and optimized management of swap space memory access behavior. The cache system is referred to as DPUCache in this application.

[0026] In a split-memory architecture, when compute nodes run out of local physical memory, they swap some memory pages out to a swap space comprised of remote memory. When a swap-in request occurs, the relevant data pages need to be reloaded via remote access. Since remote memory access latency is significantly higher than local memory access latency, frequent swapping operations will adversely affect the overall system performance.

[0027] To address the aforementioned issues, this application introduces DPUCache on the DPU side as an intermediate cache layer for swap space data, used to temporarily store and manage frequently accessed data pages in the swap space. Unlike traditional caching systems, DPUCache does not simply cache data pages based on recent access patterns, but rather combines the access sequence characteristics exhibited by swap space memory access behavior to jointly optimize cache space allocation, data replacement, and timing prefetching processes.

[0028] DPUCache's cache space is logically divided into a raw cache and a prefetch cache. The raw cache stores swap space data pages that have already been accessed, while the prefetch cache stores potentially accessed data pages predicted based on historical access sequences. Both cache regions share the same physical storage resources, and their space allocation is not fixed but dynamically adjusted according to changes in swap space access behavior.

[0029] During operation, DPUCache continuously monitors the access order, frequency, and cache hit rate of swap space memory access requests. When a clear sequential access trend is detected in swap space memory access, DPUCache will combine historical access sequence information to perform prefetching operations on data pages that may be accessed subsequently. At the same time, through an asynchronous proactive cache replacement strategy, cache space is released in advance to avoid introducing additional replacement overhead on critical memory access paths.

[0030] Through the above overall design, this application can effectively perceive the memory access characteristics of the swap space without changing the existing swap mechanism of the operating system, and reduce the number of remote memory accesses and improve the overall memory access efficiency in the scenario of separate memory swapping by dynamically managing the cache space and coordinating the optimization of the prefetch strategy.

[0031] This application provides a swap space cache optimization method for discrete memory architectures, including: The cache space is logically divided into the raw cache area and the prefetch cache area; It receives page swapping requests sent from the host memory space and converts them into swap space memory access requests. It generates a radix tree cache index based on the logical block number in the swap space memory access request, and performs a lookup based on the index. If the lookup is successful, it retrieves the data block from the prefetch cache area and sends it to the host memory space. If the lookup is unsuccessful, it adopts an asynchronous proactive cache replacement strategy to select the data block from the original cache area and write it back to the remote memory pool. At the same time, it adopts a dynamic prefetch strategy based on historical access to read prefetched data from remote memory and store it in the prefetch cache area.

[0032] See Figure 1 The system's request conversion module converts received page swapping requests into swapping space access requests; the radix tree cache index module generates a radix tree cache index based on the logical block number, and performs a search based on the index. If the data block metadata is found, the search is successful; otherwise, the search is unsuccessful, and the cache prefetch module executes a dynamic prefetching strategy based on historical access.

[0033] In one embodiment, in order to improve the utilization efficiency of cache resources in swap space access scenarios, this embodiment further proposes an adaptive cache space allocation mechanism to dynamically adjust the space ratio of the original cache area and the prefetch cache area according to the swap space access characteristics. Figure 2 A schematic diagram of the prefetch cache adaptive allocation strategy is shown.

[0034] During system operation, DPUCache continuously monitors the cache hit rate of swap space access requests and counts the number of hits in the original cache region and the number of hits in the prefetch cache region. A hit means a successful lookup, the number of hits in the original cache region means that the data block found is located in the original cache region, and the number of hits in the prefetch cache region means that the data block found is located in the prefetch cache region. By analyzing the above statistics, DPUCache can determine the contribution of different cache regions to the overall cache hit rate under the current swap space access mode.

[0035] The adjustment value for the cache space allocation ratio is calculated based on the number of hits in the original cache region and the number of hits in the prefetch cache region; specifically, the following formula can be used:

[0036] in, Adjustment value for the proportion of cache space allocation. To prefetch the number of cache hits, This represents the number of hits in the original cache region. This represents the number of requests.

[0037] The space ratio of the original cache area and the prefetch cache area is dynamically adjusted based on the adjustment value of the cache space allocation ratio.

[0038] Here, if If the number of prefetch cache hits continues to increase over a period of time, it will expand the space ratio of the prefetch cache to accommodate more potentially accessed data pages. For example, if If the increase rate is 30%, the space ratio of the prefetch cache area will increase by 30%; conversely, the space size of the prefetch cache area will be reduced accordingly to avoid the prefetched data causing invalid occupation of cache resources.

[0039] It should be noted that the above cache space adjustment process is not triggered immediately after each memory access request, but is executed periodically based on a preset time window or access count threshold, thereby avoiding the impact of frequent adjustments to the cache space ratio on system stability.

[0040] By introducing the aforementioned adaptive cache space allocation mechanism, this application enables the DPUCache's cache resource configuration to dynamically match the swap space access mode, maintaining high cache utilization efficiency under different load conditions, and providing a stable cache foundation for subsequent proactive cache replacement and dynamic prefetching strategies.

[0041] In a split-memory swapping scenario, traditional Least Recently Used (LRU) cache replacement strategies typically employ a passive triggering approach, meaning that cache replacement is only performed when the cache space is full and a new memory access request arrives. This approach works well in ordinary caching scenarios, but in high-concurrency swapping scenarios, it can easily introduce additional replacement latency on critical memory access paths, thus significantly increasing the overall access latency of swapping operations.

[0042] In one embodiment, to address the additional replacement latency introduced by traditional passive replacement algorithms under high concurrency requests in a split memory swapping scenario, an asynchronous proactive cache replacement strategy is provided. This strategy optimizes memory access performance by sensing system swapping pressure and pre-releasing cache space in the background. Figure 3 A diagram comparing active and passive cache replacement strategies is shown.

[0043] First, the original cache area space uses a Least Recently Used (LRU) queue. The system logically divides the LRU queue, such as... Figure 3As shown, the LRU queue is divided into a front half and a back half. The front half serves as the regular cache area, while the back half is defined as the "active eviction range." By limiting the candidate data blocks in the active eviction range to the back half of the LRU queue, it is possible to effectively avoid falsely evicting active data that may still be frequently accessed in the near future, thus ensuring the overall cache hit efficiency.

[0044] During system operation, a triggering mechanism based on access behavior feedback is used to detect exchange pressure. The system uses a counter to monitor continuously occurring passive replacement behaviors in real time. The specific logical judgment process is as follows: If the lookup fails (i.e., the host initiates a page swap request but the lookup fails in the radix tree cache index module), and if the cache space is full and a passive replacement (immediate page swap operation) is triggered, the counter value is incremented by 1. If the search is successful, the calculator will be reset to zero and the counting will restart. When the counter accumulates to a preset empirical threshold N (i.e., the system detects N consecutive passive replacements), it is determined that the current workload is in a high-frequency switching state, and an asynchronous active eviction task is immediately started. This task is executed by an independent background thread and does not occupy the main logical path for processing host requests.

[0045] The asynchronous active eviction task includes: a background thread selects multiple data blocks to be replaced from the active eviction range of the original cache area, compresses the data blocks, and then writes them back to the remote memory pool in batches using RDMA technology.

[0046] While initiating the asynchronous write-back, the system synchronously updates the index entries in the radix tree cache index module, marks the corresponding cache physical blocks as idle, and returns them to the cache area management module.

[0047] In this way, this embodiment can remove time-consuming operations such as "selecting old blocks, compressing, and RDMA write-back" from the critical path. Before subsequent new exchange requests arrive, sufficient free space has been reserved in the cache, thereby significantly reducing the average memory access latency and improving the system's throughput in high-concurrency scenarios.

[0048] In one embodiment, in a split memory swapping scenario, traditional fixed prefetching strategies are difficult to adapt to dynamic changes in access patterns, and are prone to problems such as inaccurate prefetching, bandwidth waste, or cache pollution. Figure 4 A schematic diagram of the prefetch module workflow is shown, and a dynamic prefetch strategy based on historical access is proposed. By performing real-time analysis of the swap space access history, access trends are dynamically identified and the prefetch granularity is adjusted, thereby improving the prefetch hit rate and reducing invalid prefetches.

[0049] Dynamic prefetching strategies based on historical access include: First, analyze the logical block numbers in the historical swap space access request sequence to determine the dominant trend in the access pattern.

[0050] Specifically, a fixed-size FIFO queue is maintained to store the difference in logical block numbers between adjacent requests in the historical swap space access request sequence, reflecting the dynamic changes in the access order.

[0051] A variable window mechanism is used to analyze the difference sequence in the FIFO queue. If a certain difference appears more than half the number of times in the window, that difference is considered the dominant trend. The detection window starts from a small size and gradually increases. Once a dominant trend is detected, the system triggers a prefetch operation. In addition, when a data block in the prefetch buffer is hit, a new round of prefetching is also triggered, forming a positive feedback loop.

[0052] This mechanism can adapt to short-term access fluctuations while capturing long-term access patterns, avoiding blind prefetching during disordered access.

[0053] Then, based on the requested address, prefetch granularity, and dominant trend in the current swap space memory access requests, the target address sequence to be prefetched is calculated. This step is a general technique in this field and will not be described in detail.

[0054] Furthermore, the sequence of target addresses to be prefetched is stored in an ascending order in the prefetch auxiliary queue. A duplicate check is performed before inserting a new address to avoid repeated prefetching of the same data block, reducing waste of network bandwidth and cache space.

[0055] Then, based on the target address sequence to be prefetched, RDMA technology is used to read the prefetched data from the remote memory pool and store it in the prefetch cache area. The prefetch cache area adopts a FIFO management strategy; when the area is full, the earliest unaccessed prefetched data is evicted.

[0056] In one embodiment, to avoid cache pollution and bandwidth waste caused by excessive prefetching, a dynamic prefetching granularity adjustment mechanism based on hit feedback is introduced. The dynamic prefetching strategy based on historical access also includes: Prefetch accuracy and prefetch coverage are calculated in real time based on the number of prefetch cache hits, using the following formula:

[0057]

[0058] in, To improve the accuracy of pre-fetching, For prefetch coverage, To prefetch the number of cache hits, The number of prefetch cache misses refers to the number of requests minus the number of prefetch cache hits. , The number of request misses is the number of requests minus the number of hits.

[0059] If prefetch accuracy improves, increase the prefetch granularity; if prefetch coverage decreases, decrease the prefetch granularity. For example, if prefetch accuracy improves by 10%, the prefetch granularity should be increased by 10%.

[0060] This application analyzes the time and space overhead introduced by the aforementioned cache replacement strategy and dynamic prefetching strategy, demonstrating that they maintain extremely low system resource consumption while improving performance. The specific analysis is as follows: (1) Complexity analysis of asynchronous active cache replacement strategy Space complexity Since this strategy only triggers active eviction when multiple consecutive replacements occur, the system only needs to record the number of consecutive replacements and the state information of the current replacement block and the previous replacement block, so the introduced space complexity is O(1).

[0061] Time complexity The logic of this strategy in handling eviction and replacement requests is the same as that of the standard cache replacement strategy. The time complexity of its dequeueing process remains O(1), ensuring the real-time performance of cache replacement.

[0062] (2) Complexity analysis of dynamic prefetching strategy Space complexity The main space consumed by the dynamic prefetching scheme is used to maintain a fixed-size FIFO historical record queue. The space complexity is O(n), where n is the maximum size of the historical queue.

[0063] Time complexity Depending on the size of the historical queue and how the detection window grows, in the worst case, if it is necessary to traverse the entire historical queue to find the dominant trend, the time complexity is O(m). ln(m)), where m is the number of elements in the history queue. However, if the history queue is small or the dominant trend can be detected quickly, the time complexity will be significantly reduced.

[0064] This application also provides a swap space cache optimization system for discrete memory architectures, used to implement the swap space cache optimization method for discrete memory architectures described in the foregoing embodiments.

[0065] In summary, this application has the following technical effects: 1. This application is specifically designed for the access characteristics of swap space access scenarios under a separate memory architecture. Unlike the existing cache management method that treats swap space access and ordinary data access in the same way, it can avoid the invalid occupation of cache resources caused by swap space access, thereby improving the utilization efficiency of cache resources.

[0066] 2. This application introduces an active replacement decision mechanism in the cache management process by sensing the characteristics of memory access requests in the exchange space in terms of access sequence, access frequency and access locality. This overcomes the replacement lag problem caused by the traditional cache strategy relying solely on passive replacement. In high-concurrency exchange request scenarios, it can effectively reduce access latency on the critical path.

[0067] 3. This application introduces a dynamic prefetching mechanism based on historical access during cache management. The prefetching granularity can be adjusted according to the changes in the swap space access mode, thereby avoiding the problem of inaccurate prefetching or bandwidth waste caused by fixed prefetching strategies in complex load scenarios and improving the prefetch hit rate.

[0068] 4. This application divides the cache space into a raw cache area for caching swap space data and a prefetch cache area for storing prefetched data. It can adaptively adjust the space ratio of the above areas according to the memory access characteristics of swap space, so that the cache management strategy can maintain high adaptability and stability under different load conditions.

[0069] 5. Through the above technical means, this application can improve the access efficiency of the separate memory swap space and reduce the number of accesses to remote memory without changing the existing swap mechanism of the operating system, thereby reducing the overall system memory access latency and improving system I / O performance.

[0070] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for optimizing swap space caching in a split memory architecture, characterized in that, include: The cache space is logically divided into the raw cache area and the prefetch cache area; Receive page swapping requests sent from the host memory space and convert them into swap space access requests; A radix tree cache index is generated based on the logical block number in the swap space memory access request. A lookup is performed based on the index. If the lookup is successful, the data block is retrieved from the cache space and sent to the host memory space. If the lookup is unsuccessful, an asynchronous proactive cache replacement strategy is adopted to select the data block from the original cache area and write it back to the remote memory pool. At the same time, a dynamic prefetch strategy based on historical access is adopted to read prefetched data from the remote memory and store it in the prefetch cache area.

2. The method as described in claim 1, characterized in that, The method further includes: Continuously monitor the access hit rate of swap space memory access requests in the cache space, and count the number of hits in the original cache region and the number of hits in the prefetch cache region; The adjustment value of the cache space allocation ratio is calculated based on the number of hits in the original cache region and the number of hits in the prefetch cache region; The space ratio of the original cache region and the prefetch cache region is dynamically adjusted according to the adjustment value of the cache space allocation ratio.

3. The method as described in claim 2, characterized in that, The adjustment value for the cache space allocation ratio is calculated using the following formula: in, Adjustment value for the proportion of cache space allocation. To prefetch the number of cache hits, This represents the number of hits in the original cache region. This represents the number of requests.

4. The method as described in claim 1, characterized in that, The asynchronous, proactive cache replacement strategy is as follows: If the search fails, and if the cache space is full and a passive replacement is triggered, the counter value is incremented by 1. If the search is successful, the calculator will be reset to zero and start counting again. When the counter's accumulated value reaches the preset empirical threshold N, the asynchronous active eviction task is immediately initiated. The asynchronous active eviction task includes: selecting multiple data blocks to be replaced from the active eviction range of the original cache area, compressing the data blocks, and then writing them back to the remote memory pool in batches using RDMA technology.

5. The method as described in claim 1, characterized in that, The dynamic prefetching strategy based on historical access includes: Analyze the logical block numbers in the historical swap space access request sequence to determine the dominant trend in the access pattern; Calculate the target address sequence to be prefetched based on the requested address, prefetch granularity, and dominant trend in the current swap space memory access requests; Based on the target address sequence to be prefetched, RDMA technology is used to read the prefetched data from the remote memory pool and store it in the prefetch cache area.

6. The method as described in claim 5, characterized in that, in, Analyze the logical block numbers in the historical swap space access request sequence to determine the dominant trends in access patterns, including: Maintain a fixed-size FIFO queue to store the difference in logical block number between adjacent requests in the historical swap space access request sequence; A variable window mechanism is used to analyze the difference sequence in the FIFO queue. If a certain difference appears more than half the number of times in the window, that difference is the dominant trend.

7. The method as described in claim 5, characterized in that, The dynamic prefetching strategy based on historical access also includes: Prefetch accuracy and prefetch coverage are calculated in real time based on the number of prefetch cache hits, using the following formula: in, To improve accuracy, To prefetch coverage, To prefetch the number of cache hits, To prefetch the number of cache misses, To request the number of misses; If prefetch accuracy improves, increase the prefetch granularity; if prefetch coverage decreases, decrease the prefetch granularity.

8. A swap space cache optimization system for discrete memory architectures, characterized in that, Used to implement the method described in any one of claims 1-7.

Images