Remote memory exchange system with non-inductive cold and hot perception of user

Abstract

The invention discloses a user-noninductive cold and hot sensing remote memory exchange system, which belongs to the field of computer storage, and is characterized in that cluster nodes are divided into extensible remote memory service nodes and computing client nodes according to roles; the remote memory node comprises a remote memory exchange server and a memory resource registration module; the computing client node comprises a FrontSwap-based remote memory exchange client, and a remote memory exchange server side is registered as a memory resource which is non-inductive to a user; furthermore, popularity statistics based on a popularity histogram is realized in a kernel mode; the memory exchange behavior is guided through the dynamic decision-making module based on online reinforcement learning, the memory management efficiency and the system performance are remarkably improved, and efficient and non-inductive memory expansion service is provided for user application.

Description

Technical Field

[0001] The present invention belongs to the field of computer storage, and more specifically, relates to a remote memory swapping system with cold and hot awareness that is imperceptible to users. Background Art

[0002] With the continuous increase in the complexity of computer applications, the memory demand has risen sharply, and the memory management methods of traditional data centers face challenges in terms of scalability, flexibility, and resource utilization. To solve these problems, the Disaggregated Memory technology has emerged. Disaggregated Memory decouples memory resources from computing nodes to form an independent memory pool, enabling multiple computing nodes to share resources in the same memory pool. This architecture not only improves the utilization rate of memory resources, reduces hardware costs, but also enhances the scalability and flexibility of the system. The core idea of Disaggregated Memory is to connect a remote memory pool with local computing nodes through a high-speed network (such as RDMA), so that computing nodes can access remote memory resources just like accessing local memory.

[0003] Although the Disaggregated Memory technology has significant advantages in theory, it still faces many challenges in practical applications. First, the latency and bandwidth issues of remote memory access limit the performance of the system. Although the RDMA technology alleviates this problem to a certain extent, frequent remote memory access still introduces a large performance overhead. In the prior art, the user-space library solution is highly flexible but cannot achieve application imperceptibility, requires a large amount of application code modification, and has a large migration difficulty; the AutoNUMA solution has limited scalability and is difficult to adapt to large-scale distributed environments; the system based on the swap mechanism has performance bottlenecks during page swapping in and out, affecting the overall efficiency. Summary of the Invention

[0004] Aiming at the above defects or improvement requirements of the prior art, the present invention provides a remote memory swapping system with cold and hot awareness that is imperceptible to users, which can improve the data access speed, reduce unnecessary data migration, and thus enhance the overall performance of the system.

[0005] To achieve the above object, according to the first aspect of the present invention, there is provided a remote memory swapping system with cold and hot awareness that is imperceptible to users, including a scalable remote memory service node and a computing client node;

[0006] The remote memory service node includes:

[0007] A remote memory swapping server module, configured to receive and process remote memory read / write requests and swapping requests from the computing client node;

[0008] A memory resource registration module, which is used to pre-allocate the memory resources of the remote memory service node itself before the remote memory exchange server module starts, and authorize the computing client node to access the memory resources;

[0009] The computing client node includes:

[0010] A remote memory exchange client module, which is used to rewrite the kernel's FrontSwap system and page swap path, register the remote memory service node as an extended memory resource to replace the original disk swap path; expand the operation set of FrontSwap, and introduce asynchronous read and polling operations; allow multiple processes to initiate asynchronous repeated swap-ins simultaneously, and continue to execute other logics of the multiple processes until any one process successfully swaps in, and other processes all obtain page references and roll back the swap-in results; when there is memory pressure, start scheduling asynchronous memory recycling tasks in advance;

[0011] A hot and cold awareness module, which is used to sense memory behavior and count page hotness;

[0012] A behavior decision module, which is used to guide remote memory exchange according to the page hotness.

[0013] According to the fourth aspect of the present invention, there is provided a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the method described in the first aspect is implemented.

[0014] Generally speaking, compared with the prior art by the above technical solution conceived by the present invention, the following beneficial effects can be achieved:

[0015] 1. The memory exchange system provided by the present invention, which is insensitive to users in the kernel state, does not require modification of upper-layer applications and is easy to migrate; by implementing the memory exchange mechanism at the kernel level, it is completely transparent to upper-layer applications and does not require any modification or adaptation to existing applications. This design not only reduces the cost of system migration, but also improves the universality of the solution, and can be seamlessly integrated into various computing environments, especially suitable for complex scenarios such as cloud computing and edge computing; in addition, the system provided by the present invention designs an asynchronous repeated swap-in mechanism, which avoids performance bottlenecks caused by repeated page swap-in checks, and at the same time effectively prevents the occurrence of blocking recycling by swapping out cold pages in advance. This optimization not only reduces the latency of memory access, but also improves the throughput of the system, ensuring stable performance under high load.

[0016] 2. As a preferred solution, in the system provided by the present invention, the thermal perception module maintains a global thermal view through a heat histogram, reducing the storage and calculation pressure of thermal metadata. By introducing the heat histogram, it is possible to efficiently maintain the global memory access thermal view, significantly reducing the storage and calculation overhead of thermal metadata. Compared with traditional fine-grained thermal statistics, the heat histogram reduces the resource consumption of the system and improves the overall performance while ensuring the effectiveness of information by aggregating and compressing data. In addition, the behavior decision module adopts an online reinforcement learning algorithm, which can prevent concept drift in long-running programs, and thus can adaptively control memory behavior. By introducing the online reinforcement learning algorithm, the present invention can dynamically track changes in application behavior and prevent model degradation caused by concept drift. The online reinforcement learning algorithm updates the memory management policy in real time to ensure that the system always adapts to the needs of the application, improving the adaptive ability of the system and the stability of long-term operation.

[0017] In summary, the remote memory swapping system with user-insensitive thermal perception provided by the present invention optimizes the memory resource management strategy by introducing artificial intelligence technology, improves the data access speed, reduces unnecessary data migration, and thus enhances the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the structure of the remote memory swapping system with user-insensitive thermal perception provided by the embodiment of the present invention;

[0019] Figure 2 Flowchart of asynchronous page-in and early reclamation operations provided by the embodiment of the present invention;

[0020] Figure 3 Schematic diagram of the heat histogram logic provided by the embodiment of the present invention;

[0021] Figure 4 Schematic diagram of the memory behavior sampling and heat histogram update logic provided by the embodiment of the present invention;

[0022] Figure 5 Schematic diagram of the machine learning framework for kernel-user state collaboration provided by the embodiment of the present invention;

[0023] Figure 6 Schematic diagram of the communication interface between the kernel state and the user state provided by the embodiment of the present invention;

[0024] Figure 7 Schematic diagram of the multi-task reinforcement learning algorithm provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] To make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0026] Since most existing remote memory resource solutions require explicit modification of the memory path by applications, which is difficult to migrate, and the allocation and management strategies for applications with different memory modes are not intelligent enough, a remote memory swapping system that is imperceptible to users is implemented based on the FrontSwap interface. As a preferred solution, an online reinforcement learning technology is further introduced on this basis to optimize the management strategy of memory resources, improve the data access speed, reduce unnecessary data migration, and thus improve the overall performance of the system.

[0027] Based on this, an embodiment of the present invention provides a remote memory swapping system with cold and hot perception that is imperceptible to users, including a scalable remote memory service node and a computing client node;

[0028] The remote memory service node includes:

[0029] A remote memory swapping server module, which is used to receive and process remote memory read / write requests and swapping requests from the computing client node;

[0030] A memory resource registration module, which is used to pre-allocate the memory resources of the remote memory service node itself before the remote memory swapping server module starts, and authorize the computing client node to access the memory resources;

[0031] The computing client node includes:

[0032] A remote memory swapping client module, which is used to rewrite the kernel's FrontSwap system and page swapping path, register the remote memory service node as an extended memory resource to replace the original disk swapping path; expand the operation set of FrontSwap, and introduce asynchronous reading and polling operations; allow multiple processes to initiate asynchronous repeated swap-ins simultaneously, and continue to execute other logics of the multiple processes until any one process successfully swaps in, and other processes obtain page references and roll back the swap-in results; when there is memory pressure, start scheduling asynchronous memory recovery tasks in advance;

[0033] A cold and hot perception module, which is used to perceive memory behavior and count page hotness;

[0034] A behavior decision module, which is used to guide remote memory swapping according to the page hotness.

[0035] In the remote memory swapping system with user - imperceptible cold - and - heat perception provided by the embodiments of the present invention, the remote memory service node pre - allocates memory resources before the system starts, and through the dynamic resource pool management and access control mechanism, authorizes the computing nodes (i.e., computing client nodes) to access rights, ensuring the security and efficient utilization of memory resources; and starts a daemon process running in the user space, which is responsible for receiving and processing the remote memory read - write requests and swapping requests from the client. According to the request type (read or write), it directly accesses the remote memory resources through the RDMA unilateral operation, ensuring the high availability and scalability of the system. Correspondingly, the remote memory service node includes a remote memory swapping server module and a memory resource registration module;

[0036] Among them, the remote memory swapping server module is a daemon process running in the user space, which is used to receive and process the remote memory read - write requests and swapping requests from the client. This module realizes the processing of remote memory read - write requests through the unilateral operation of an efficient data transmission protocol (RDMA), completes the swapping, and supports the asynchronous read function to optimize the performance of remote page acquisition;

[0037] The memory resource registration module is used to pre - allocate memory resources before the remote memory swapping server starts, and authorizes the computing service node to access the memory resources. This module ensures the security and efficient utilization of memory resources through the dynamic resource pool management and access control mechanism.

[0038] In the remote memory swapping system with user - imperceptible cold - and - heat perception provided by the embodiments of the present invention, the computing client node includes a remote memory swapping client module, a cold - and - heat perception module, and a behavior decision - making module;

[0039] Among them, the remote memory swapping client module is used to rewrite the kernel's FrontSwap system and page swapping system, register the remote memory node as an extended memory resource, replace the original disk swapping path, so as to provide the user - imperceptible remote memory access ability and realize the transparent expansion of memory resources; expand the operation set of FrontSwap, introduce asynchronous read and polling operations; during the swap - in process, this module does not need to perform complex repeated swap - in detection and inter - process synchronization, allows multiple processes to initiate asynchronous repeated swap - ins simultaneously, and attempts to continue to execute other logics of the multiple processes until any one process successfully swaps in, and other processes all obtain page references and discard the swap - in results; in addition, this module supports the asynchronous early recycling function, that is, when there is memory pressure, it starts to schedule the memory recycling task in advance.

[0040] Therefore, the entire working process of the remote memory swapping client module includes asynchronous page swap - in and early recycling operations, as Figure 2 shown, the entire working process of this module is as follows:

[0041] S1, a page fault occurs in the process, triggering the page fault interrupt processing, the kernel takes over and starts the page fault processing process;

[0042] S2, check the local cache of the page. If the local cache of the page hits, the cache is read directly and the process goes to step S7; otherwise, a page change operation is triggered and the process goes to step S3;

[0043] S3: Regardless of whether other processes are attempting to swap in the same page, no duplication check is performed. Instead, an asynchronous swap-in request is initiated and placed in a waiting queue. When any process completes its swap-in, that process updates the page table and wakes up the processes in the waiting queue. All other processes then obtain a reference to this page and discard any outstanding swap-in requests. This strategy significantly reduces inter-process synchronization overhead by incurring a small amount of bandwidth overhead, improving the system's concurrent performance.

[0044] S4: Update the cgroup statistics and trigger an early recovery attempt in the background.

[0045] S5, the early recovery process checks whether the available local memory is below the watermark; if it is below the watermark, memory resources are released based on the page popularity; otherwise, it means that there is sufficient available memory and no operation is performed to avoid unnecessary performance overhead.

[0046] S6, the system pre-fetches pages that may be accessed soon based on the predicted memory access pattern.

[0047] S7, updates relevant metadata to further optimize memory access performance;

[0048] S8: Update the page table entry. If it is a repeated swap and the data has been swapped in by other processes, rollback the swap result.

[0049] The hot and cold perception module is used to perceive memory behavior and count page heat; the behavior decision module is used to guide the memory swap behavior of the remote swap client module according to page heat, thereby realizing the management of memory swap behavior.

[0050] The above-mentioned hot and cold perception module and behavior decision module can be implemented based on existing methods. For example, for the hot and cold perception module, the following three implementation methods can be adopted:

[0051] 1. Full page table scan: Periodically scan the entire page table to check whether the access bit of a page table entry has been updated. If so, it indicates that the page has been accessed, which can increase the popularity of the page. However, a full page table scan is time-consuming and only indicates whether a page has been accessed since the last page table scan, not the total number of accesses since the last scan. This is a post-hoc statistics, requiring access detection during the scan, which is not timely enough.

[0052] 2. Page table entry scanning based on regional sampling: For example, the DAMON mechanism of the Linux kernel is adopted. In the virtual memory space, several memory regions with variable sizes are divided, and the dynamic merging and decomposition of regions are performed according to the changes in memory access behavior. Only the page access bit of one page is tracked in each memory region, representing the access frequency of all pages in this region. However, the page access bit can only indicate that it has been accessed since the last page table scan and cannot indicate the number of accesses since the last scan. It is a post hoc statistic and requires knowing that it has been accessed during the scan, which is not timely enough.

[0053] 3. Lightweight interrupt based on page-hint-fault: For example, Thermostat. An additional bit is added to the page table, and accessing these pages will trigger an interrupt, and the corresponding handler runs actively to immediately record the page access. However, the additional TLB misses and interrupts bring relatively large overhead.

[0054] For the behavior decision-making module, the following two implementation methods can be adopted:

[0055] 1. Rule-based decision-making module: For example, the Linux watermark. Specific thresholds are set in advance, and pages are divided into hot and cold according to the thresholds. However, this implementation method can only determine the object to be swapped, not the frequency and method of swapping; it cannot update the behavior decision in a timely manner for complex application behaviors.

[0056] 2. Traditional machine learning classifiers: For example, Bayesian-based classifiers and decision tree-based classifiers. However, the training of traditional machine learning classifiers takes a long time and cannot adapt to the concept drift problem caused by changes in application behaviors.

[0057] Considering that the allocation and management strategy of memory resources needs to be more intelligent, the traditional memory management mechanism is difficult to meet the requirements of the disaggregated memory architecture, especially in dynamically adjusting the memory allocation strategy and optimizing the data migration efficiency. In addition, the identification and migration of hot and cold data are also a major difficulty in the disaggregated memory system. Due to the complex and dynamically changing memory access patterns, the existing hot and cold awareness mechanisms often have difficulty accurately identifying hot and cold data, resulting in unnecessary data migration or resource waste. In addition, the collection of heat information needs to balance accuracy, efficiency, and timeliness, and it is difficult for existing solutions to meet all at the same time. The management of heat metadata introduces relatively large storage and computing overhead, affecting system performance. The frequency and granularity of data migration are difficult to dynamically adjust, resulting in resource waste or performance degradation. The training of traditional machine learning models takes a long time and cannot adapt to the concept drift problem caused by changes in application behaviors. The memory system has high requirements for the efficiency and real-time performance of the model, and existing models are difficult to meet. Based on this, as a further preferred solution, the hot and cold awareness module runs in the kernel state and includes a memory behavior sampling module and a heat view statistics module;

[0058] Among them, the memory behavior sampling module is used to collect memory access behavior data and interact with the reinforcement learning decision-making module running in the user space.

[0059] Specifically, the memory behavior sampling module uses the PEBS system interrupt to monitor memory behavior and collect memory access behavior data to efficiently and accurately analyze memory access behavior. Specifically, it includes: starting the ksamplingd process in the kernel, periodically monitoring the PEBS_ALL_STORES / LOADS and PEBS_STLB_MISS_STORES / LOADS events, capturing memory read / write and page table entry miss behaviors, and collecting the corresponding addresses and page numbers of memory behaviors to count memory access information.

[0060] The entire workflow of the memory behavior sampling module is as follows:

[0061] S1`, when starting the remote exchange system, register the PEBS event, use the Perf tool to register the cores and events to be monitored, and specify the specific memory access information to be obtained;

[0062] S2`, before running the application, start the ksamplingd process, initialize the statistical data, including sampling count, read / write behavior count, page fault count, etc., configure metadata such as the sampling frequency and batch size of the initial monitoring behavior, and then ksamplingd starts continuous monitoring;

[0063] S3`, count the memory access events, sample this event when reaching the statistical frequency, temporarily store the collected data in the perf ring buffer, and transfer it to the heat view statistics module and the reinforcement learning module when the number of events reaches the batch size.

[0064] The heat view statistics module is used to maintain a heat histogram in the memcgroup structure of the kernel; the heat histogram divides different heat regions where each page is located through percentile boundaries, and centrally maintains the global heat view. This module efficiently manages and maintains the global memory access heat view through the heat histogram technology, reducing the storage and calculation overhead of heat metadata.

[0065] Specifically, the heat view statistics module maintains a heat histogram in the memcgroup structure of the kernel to count the global heat view, divides different heat regions through percentile boundaries, centrally maintains the global heat view, and reduces the storage and calculation overhead of separately performing metadata statistics for each page.

[0066] The heat histogram is used to efficiently maintain the global memory access heat view, reduce the storage and calculation overhead of metadata, and quickly complete the cold / hot division. Its logical structure is as Figure 3As shown, the number of page accesses is divided into several bins according to exponential growth. Each bin records the number of pages with accesses in the corresponding interval. For example, bin 1 records pages with 1 access, bin 2 records pages with 2-3 accesses, bin 3 records pages with 4-7 accesses, and so on. The upper limit of the last bin is set to infinity to cover all frequently accessed pages, ensuring the integrity and adaptability of the data structure. A heat histogram and hot / cold ratio are maintained in the control group (cgroup), using a compact data storage format to reduce memory usage and access overhead.

[0067] Using a histogram, we can calculate percentiles based on the number of pages in each cell and determine the hot and cold thresholds. For example, pages above the 20th percentile are classified as hot pages, and the cell containing this demarcation threshold contains pages that are warm pages. Pages above this cell are hot pages, and pages below this cell are cold pages. Using the heat histogram data structure, hot and cold classification can be completed with simple interval queries and counting operations, significantly reducing computational complexity. Hot pages are retained in local memory, reducing the overhead of remote memory access. Cold pages are preferentially swapped to remote memory, freeing up local memory resources. Warm pages dynamically adjust their retention or swapping strategies based on system load and memory pressure.

[0068] The entire process of the heat view statistics module includes the registration and update of the global heat view and the judgment of page heat, which includes the following steps:

[0069] S1``, when starting the upper-layer application, you need to specify the memgroup to which it belongs and specify enable htmm. This additional operation is only required when the application is started; all subsequent actions are not affected by the upper-layer application;

[0070] S2 receives the memory access behavior of the page from the memory behavior sampling module and updates the histogram cell to which this page belongs.

[0071] like Figure 3 As shown, each histogram cell contains the page number and access count within the threshold range of this histogram within this cycle. Therefore, the histogram cell to which this page belongs can be determined based on the corresponding address, page number, and operation type of the memory access behavior of the page. For example, the memory sampling module samples the access to page 100. The page 100 exists in the histogram [512, 1024). Its access count at this moment is 1023. At this time, its access count is updated to 1024, and it belongs to the next histogram cell [1024, 2048). At this time, this item is deleted from the histogram [512, 1024) and added to the histogram [1024, 2048).

[0072] S3``, based on the ratio of the histogram cells, mark the pages in the histogram cells above the heat threshold as hot pages, mark the pages in the histogram cells below the heat threshold as warm pages, and mark the pages in the histogram cells below the heat threshold as cold pages.

[0073] Memory behavior sampling and heat histogram update logic is as follows Figure 4 As shown, the steps are as follows:

[0074] A1. When starting the remote switching system, register PEBS events. Use the Perf tool to register the cores and events to be monitored, and specify the specific memory access information to be obtained.

[0075] A2: Before running the application, start the ksamplingd process and initialize statistical data, including sampling counts, read and write behavior counts, and page fault counts. Configure metadata such as the sampling frequency and batch size for the initial monitoring behavior. After that, ksamplingd begins continuous monitoring.

[0076] A3: When starting the upper-layer application, you need to specify the memgroup to which it belongs and enable htmm. This additional operation is only required when the application is started; all subsequent actions are not affected by the upper-layer application.

[0077] A4 counts memory access events. When the statistical frequency is reached, the event is sampled and the collected data is temporarily stored in the perf ring buffer. When the number of events reaches the batch size, it is passed to the heat statistics module.

[0078] A5 receives memory access behavior from the memory behavior sampling module and updates the histogram cell to which this page belongs.

[0079] A6: Based on the ratio of the histogram cells, the pages in the histogram cells above the heat threshold are marked as hot pages, the pages in the histogram cells below the heat threshold are marked as warm pages, and the pages in the histogram cells below the heat threshold are marked as cold pages.

[0080] The behavior decision module is a reinforcement learning decision module running in user mode;

[0081] Among them, the reinforcement learning decision module adopts an online reinforcement learning algorithm, uses memory access behavior data as a training set, conducts phased training and model updates, and optimizes the memory swap strategy; the memory swap strategy includes the migration frequency, migration timing and migration granularity of hot and cold data.

[0082] Specifically, if Figure 5As shown in the figure, in order to adapt to a variety of application scenarios and their different modes, the reinforcement learning decision-making module adopts an online reinforcement learning algorithm. Based on the memory access behavior data obtained from the cold and hot perception module as the training set, it conducts periodic training and model updates, dynamically adapts to the changes in the application memory behavior, optimizes the memory swapping strategy, and ensures that the system can maintain high-efficient memory management performance in different application scenarios and different stages of the same application.

[0083] The advantage of online learning is that it can update the model in real time according to the memory access behavior data, dynamically adapt to the changes in the memory access mode of the application, and ensure that the system can maintain high-efficient memory management performance in different scenarios and stages. Through online learning, the system can continuously optimize the memory swapping strategy during operation, avoiding the limitations of traditional offline learning that requires retraining the model. To further benefit from parameter sharing and cross-boosting of related tasks, the embodiments of the present invention adopt multi-task reinforcement learning. By sharing some model parameters, it reduces the computational overhead and storage requirements for model training, and at the same time improves the generalization ability of the model. In addition, the information sharing and cross-learning between related tasks significantly improve the accuracy and robustness of the model.

[0084] That is, the aforementioned ksamplingd process collects memory access behaviors, uses the eBPF kernel ring buffer to batch transfer them to the user space. The user space continuously receives the latest memory access behaviors, constructs a training set, evenly distributes the training process to the latest small number of samples, dynamically updates the parameters, decides the frequency of cold and hot data collection to balance the information collection overhead and accuracy, decides the timing of cold and hot data migration to balance the hardware utilization rate and the risk of direct recycling, and decides the granularity of cold and hot data migration to prevent write amplification; and adopts a multi-task reinforcement learning framework. Multiple related tasks share some parameters and network structures. On the one hand, it reduces the requirement for the quantity of the training data set and accelerates the training; on the other hand, the related tasks obtain knowledge transfer and verification, improving the accuracy. After making a decision, the configuration update is passed back to the kernel space through the eBPF map to achieve dynamic adjustment of the system policy.

[0085] The online reinforcement learning algorithm adopted by the reinforcement learning decision-making module can be any reinforcement learning algorithm. Considering the computational efficiency, as a further preferred solution, the reinforcement learning decision-making module adopts the online PPO reinforcement learning algorithm. The entire workflow of the reinforcement learning decision-making module includes memory behavior collection, model update, and decision update. The steps are as follows:

[0086] S1``` receives the information such as the address, operation mode, and size of the memory behavior collected by the Profiling process monitoring the PEBS event and forwarded by the eBPF kernel ring buffer, and organizes it into a mini-batch; makes decisions on the migration frequency, migration timing, and migration granularity of hot and cold data according to the past model weights, passes them back to the kernel state through the eBPF map, and updates the configuration of the remote memory interaction client node;

[0087] S2``` collects new memory statistics, calculates rewards, and dynamically updates model parameters.

[0088] It can be understood that correspondingly, in step S5 of the working process of the remote memory exchange client module, the hot and cold pages are divided according to the hot percentile of the heat histogram, the cold pages are recycled, and the asynchronous recycling frequency is dynamically adjusted according to the online reinforcement learning algorithm to balance the allocation efficiency and memory resource utilization rate.

[0089] The hot and cold perception module can communicate with the behavior decision-making module in the user state through the procfs function or the perf buffer function. However, considering the low communication efficiency of the procfs function and the fact that the perf buffer function only supports the monitoring of memory behavior of a single core, based on this, as a further preferred solution, the hot and cold perception module uses the eBPF function to communicate with the behavior decision-making module in the user state, which can improve communication efficiency and save system resources;

[0090] As Figure 6 shown, the eBPF source code is pre-written, the kernel ring buffer and the eBPF map are registered, the bytecode is generated via libbpf, and after security verification, the corresponding kernel function is captured and overloaded through JIT counting. The profiling process in the kernel state collects memory access behaviors and uses the eBPF kernel ring buffer to batch-transfer them to the reinforcement learning decision-making module in the user state; the reinforcement learning module dynamically updates the online model according to the received data and passes the configuration update back to the kernel state through the eBPF map to achieve dynamic adjustment of the system policy

[0091] As Figure 7As shown in the figure, the embodiment of the present invention adopts a design architecture that separates the training network from the inference network. The inference network receives the input information collected by the profiling process, makes behavioral decisions, and calculates the reward value. At the same time, the system introduces an experience replay mechanism to store the memory access behavior data and the reward value in the experience pool, which serves as the main source of training data. These data are randomly sampled and batch-processed to generate a training dataset for updating the weights of the training network. The training network is responsible for optimizing key decision items such as migration frequency, sampling configuration, and data path, etc., and synchronizes the updated parameters to the inference network in a periodic manner. This design not only accelerates the training process but also ensures the stability of the inference network, avoiding interference with its inference process caused by frequent updates. Through the separation of training and inference and the periodic synchronization of parameters, the system can continuously optimize the memory management strategy in a dynamically changing environment and improve the overall performance.

[0092] In summary, the present invention divides the nodes in the system into remote memory service nodes and computing client nodes. The remote memory service nodes pre-register memory areas and use the RDMA network to provide memory read / write and exchange services; the computing client nodes maintain the system heat view using a heat histogram in the kernel state, reducing the storage and computing overhead of metadata statistics; use the PEBs system terminal based on CPU hardware events to monitor memory behavior and dynamically adjust the sampling frequency; achieve efficient communication between the kernel state and the user state through eBPF technology, supporting dynamic configuration updates and batch data transmission; adopt an online reinforcement learning model to dynamically adjust system configuration parameters in real time to adapt to changes in application behavior. By providing a standardized interface through FrontSwap, remote memory can be accessed like local swap space without complex underlying implementation, reducing development and maintenance costs. By registering remote memory as swap space, the system can dynamically adjust the memory resource usage strategy to meet the needs of different application scenarios, improving the flexibility and scalability of the system; the eBPF function adopted supports dynamic updating of kernel configuration without restarting the system, can adjust the memory management strategy in real time according to application requirements, and can batch-transfer information to the user state through the eBPF kernel ringbuffer; online reinforcement learning can dynamically adjust system configuration parameters as application behavior changes, solving the problems of long training time and inability to handle concept drift of traditional machine learning models.

[0093] Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A remote memory swapping system with cold and hot perception that is imperceptible to users, characterized in that, It includes an extensible remote memory service node and a computing client node; The remote memory service node includes: A remote memory swapping server module for receiving and processing remote memory read / write requests and swapping requests from the computing client node; A memory resource registration module for pre-allocating the memory resources of the remote memory service node itself before the remote memory swapping server module starts, and authorizing the computing client node to access the memory resources; The computing client node includes: A remote memory swapping client module for rewriting the kernel's FrontSwap system and page swapping path, registering the remote memory service node as an extended memory resource to replace the original disk swapping path; expanding the operation set of FrontSwap, introducing asynchronous reading and polling operations; allowing multiple processes to simultaneously initiate asynchronous repeated swap-ins and continue to execute other logics of the multiple processes until any one process successfully swaps in, and other processes obtain page references and roll back the swap-in results; when there is memory pressure, scheduling asynchronous memory recycling tasks in advance; A hot and cold awareness module for perceiving memory behavior and counting page hotness; A behavior decision module for guiding remote memory swapping according to the page hotness.

2. The system according to claim 1, wherein The working process of the remote memory swapping client module includes the following steps: S1. A process has a page fault, triggering a page fault interrupt handling, and the kernel takes over and starts the page fault handling process; S2. Check the page local cache. If the page local cache hits, directly read the cache and enter S7; otherwise, trigger a page swapping operation and enter S3; S3. Regardless of whether there are other processes attempting to swap in the same page, do not perform repetitive checks, directly initiate an asynchronous swap-in request and add it to the waiting queue. If any process finishes swapping in, this process updates the page table and wakes up the processes in the waiting queue, and all other processes obtain the page reference and discard the unfinished swap-in requests; S4. Update the cgroup statistical count and trigger a background attempt at early recycling; S5. The early recycling process checks whether the available local memory is below the water level. If so, release the corresponding memory resources according to the page hotness and enter S6; otherwise, directly enter S6; S6. Prefetch the pages that may be about to be accessed according to the predicted memory access pattern; S7. Update relevant metadata to optimize memory access performance; S8. Update the page table entry. If it is a repeated swap-in and the data has been swapped in by other processes, roll back the swap-in result of this time.

3. The system according to claim 1 or 2, characterized in that, The hot and cold awareness module runs in the kernel state and includes a memory behavior sampling module and a hotness view statistical module; Among them, the memory behavior sampling module uses the PEBS system interrupt to monitor memory behavior, collects memory access behavior data, and conducts information interaction with the reinforcement learning decision module running in the user state; The hotness view statistical module is used to maintain a hotness histogram in the kernel's memcgroup structure; the hotness histogram divides different hotness regions where each page is located through percentile boundaries to centrally maintain the global hotness view; The behavior decision module is a reinforcement learning decision module running in the user state; Among them, the reinforcement learning decision-making module adopts an online reinforcement learning algorithm, uses the memory access behavior data as the training set, conducts stage training and model update, and optimizes the memory swapping policy; the memory swapping policy includes the migration frequency, migration timing, and migration granularity of hot and cold data.

4. The system according to claim 3, wherein The working process of the memory behavior sampling module includes the following steps: S1`, When starting the remote swapping system, register the PEBS event, use the Perf tool to register the cores and events to be monitored, and specify the specific memory access information to be obtained; the PEBS events include PEBS_ALL_STORES / LOADS, PEBS_STLB_MISS_STORES / LOADS events; S2`, Before running the application, start the ksamplingd process, initialize the statistical data, and configure the metadata; ksamplingd starts continuous monitoring and counts the memory access information; Among them, the statistical data includes the sampling count, read / write behavior count, and page fault count; the metadata includes the sampling frequency and batch size of the initial monitoring behavior; the memory access information includes the address, page number, and operation type corresponding to the memory access behavior; S3`, Count the memory access events, sample this event when the statistical frequency is reached, temporarily store the collected data in the perfring buffer, and pass it to the heat view statistics module and the reinforcement learning decision-making module when the number of events reaches the batch size.

5. The system according to claim 3, characterized in that The hot and cold awareness module uses the eBPF function to implement communication with the behavior decision-making module in the user space; Among them, the eBPF function collects memory access behaviors through the profiling process in the kernel space and batch-transmits them to the reinforcement learning decision-making module in the user space through the eBPF kernel ring buffer; the reinforcement learning module dynamically updates the online model according to the received data and passes the configuration update back to the kernel space through the eBPF map to achieve dynamic adjustment of the system policy.

6. The system according to claim 3, wherein The working process of the heat view statistics module includes the following steps: S1``, When starting the upper-layer application, it is necessary to specify the memgroup to which the upper-layer application belongs; S2``, Receive the memory access behavior of the page from the memory behavior sampling module and update the histogram unit in the heat histogram to which this page belongs; S3``, According to the ratio of the histogram units, mark the pages in the histogram units higher than the heat threshold as hot pages, mark the pages in the histogram units at the heat threshold as warm pages, and mark the pages in the histogram units lower than the heat threshold as cold pages.

7. The system according to claim 5, wherein The online reinforcement learning algorithm adopted by the reinforcement learning decision-making module is the PPO algorithm, and the working process of the reinforcement learning decision-making module includes the following steps: S1```, Receive the address, operation mode, and size of the memory behavior forwarded by the eBPF kernel ring buffer, and organize them into a mini-batch; make decisions on the migration frequency, migration timing, and migration granularity of hot and cold data according to the model weights, pass them back to the kernel state through the eBPF map, and update the configuration of the remote memory interaction client node; S2```, Collect new memory statistics, calculate the reward, and dynamically update the model parameters.

Images