Separated memory system and management method

Abstract

The invention discloses a separated memory system and a management method. The system comprises a host end, a DPU (Data Processing Unit) and a remote memory end which are interconnected through RDMA (Remote Direct Memory Access). The method comprises the following implementation steps that: a cache management module initializes a DRAM memory space; the cache management module manages a read / write request issued by the computing server; the remote memory mapping management module maintains a mapping table and transmits a read / write request; the remote memory service module processes the read / write request; and the remote memory recovery module releases invalid memory pages in the memory server. The DRAM serves as a local cache of a remote memory, persistent cache of data accessed by a computing server at high frequency can be achieved, a high-frequency access request can be directly responded through the DRAM, RDMA network transmission is reduced, and therefore read-write time delay is reduced; and meanwhile, the CPU utilizes the RDMA network interface to aggregate the available memory of the remote memory end, so that the available memory of the operating system of the computing server is expanded.

Description

Technical Field

[0001] The present invention belongs to the field of distributed storage of computer systems, and relates to a disaggregated memory system and a management method thereof. Background Art

[0002] Disaggregated memory is a memory resource pooling management technology that enables a host to access not only local memory but also the idle memory (or memory pool) of other hosts. A disaggregated memory system typically includes compute servers and memory servers. In a disaggregated memory system, an application running on a compute server requests additional memory from a memory server. The compute server and the memory server generally communicate through a high-speed, low-latency Remote Direct Memory Access (RDMA) network, enabling the compute server to directly access the memory of the memory server.

[0003] The disaggregated memory system provides the memory of the memory server to the compute server for use through an RDMA network. Compared with local memory, cross-server memory access in a disaggregated memory system generally has a relatively high latency. Therefore, when designing and evaluating a disaggregated memory system, it is necessary to comprehensively consider factors related to latency to ensure that the system can provide efficient services in different application scenarios.

[0004] The application publication number is CN119149210A, and the name is "Memory Scheduling Method, System and Product Based on Disaggregated Memory System", which discloses a disaggregated memory system and a management method. The invention determines the target memory device accessed by the current task according to the demand parameters of the current task and the actual operating parameters of the disaggregated memory system. Before actually deploying the memory device, according to the execution demand parameters of the current task, the demand parameters corresponding to different tasks and the actual operating parameters of the disaggregated memory system are used to preliminarily determine the target memory device to be accessed by the current task. In order to reduce the access latency corresponding to the current task, the access cost of the current target memory device is estimated based on the historical call times and access latency of the target compute accelerator corresponding to the current task. According to the access cost, the scheduling memory device of the current task is determined, so that the access cost of the scheduling memory device accessed by the target compute accelerator corresponding to each task is relatively small, and the access execution efficiency of the target compute accelerator of the current task is improved. However, it uses the Remote Direct Memory Access (RDMA) network as the underlying communication mechanism, resulting in a still relatively high access latency between the compute server and the memory server, and it is not conducive to the expansion of the available memory capacity of the compute server. Summary of the Invention

[0005] The object of the present invention is to overcome the defects of the above-mentioned prior art, and propose a separated memory system and management method, which are used to solve the problem of high access latency in the existing separated memory system, and expand the available memory capacity of the computing server.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A separated memory system includes a host end composed of multiple computing servers and a remote memory end composed of multiple memory servers interconnected through a Remote Direct Memory Access (RDMA) network; a Data Processing Unit (DPU) is loaded between the host end and the remote memory end; the DPU includes a cascaded Dynamic Random Access Memory (DRAM) and a Central Processing Unit (CPU), where the DRAM is used to cache the data stored by the computing server in the remote memory end; the CPU is used to provide a running environment for aggregating the available memory of the remote memory end.

[0008] As an optimization, the CPU includes a cascaded cache management module, a remote memory mapping management module, and a remote memory service module, as well as a remote memory recycling module connected to the input end of the remote memory mapping management module, where:

[0009] The cache management module is used to search for the block storage page address of the read request or write request sent by the computing server, and divert the read request or write request to the DRAM or the remote memory mapping management module according to the search result;

[0010] The remote memory mapping management module is used to maintain the mapping relationship between the block storage page address and the memory page address of the memory server, and send the read / write request to the remote memory service module according to the mapping result;

[0011] The remote memory service module is used to perform read / write operations on the memory space of the memory server;

[0012] The remote memory recycling module is used to periodically release the invalid memory in the memory server according to the mapping relationship of the remote memory mapping management module.

[0013] A management method for a separated memory system includes the following steps:

[0014] (1) The cache management module initializes the DRAM memory space:

[0015] The cache management module initializes the DRAM memory space, including multiple memory pages, as well as the Least Recently Used (LRU) queue and the First In First Out (FIFO) queue;

[0016] (2) The computing server issues read / write requests;

[0017] (3) The cache management module manages the read / write requests:

[0018] Search for the block storage address of the read request or write request sent by the computing server through the LRU queue and the FIFO queue, and split the read request or write request to the DRAM or the remote memory mapping management module according to the search result;

[0019] (4) The remote memory mapping management module maintains the mapping table and forwards the read / write requests:

[0020] The remote memory mapping management module searches for the mapped memory server address in the mapping table through the addresses of the read request and the write request, writes the memory server address corresponding to the read request into the read request, and at the same time sets the memory server address corresponding to the write request to invalid, and then sends the write request to the remote memory service module;

[0021] (5) The remote memory service module processes the read / write requests:

[0022] The remote memory service module reads data from the memory server according to the memory server address of the read request and returns it to the remote memory mapping management module; collects the memory occupancy rate of the memory server and writes the data of the write request into the memory server with the lowest memory utilization rate;

[0023] (6) The remote memory recycling module releases the invalid memory pages in the memory server:

[0024] The remote memory recycling module periodically releases the invalid memory in the memory server according to the mapping table of the remote memory mapping management module.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] A data processor DPU composed of cascaded dynamic random access memories DRAM and a central processing unit CPU is loaded between the host end and the remote memory end of the present invention. The DRAM therein serves as a local cache for the remote memory, enabling persistent caching of data frequently accessed by the computing server, so that frequent access requests can be directly responded to by the DRAM, avoiding excessive RDMA network transmissions, thereby significantly reducing read / write latency. At the same time, the CPU aggregates the available memory of the remote memory end using the RDMA network interface, and a single computing server can be connected to multiple memory servers simultaneously, expanding the available storage capacity of the operating system of the computing server. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic diagram of the overall structure of the split memory system of the present invention.

[0028] Figure 2 is a flowchart of the management method of the split memory system of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] Referring to Figure 1 , the split memory system of the present invention includes a host end composed of multiple computing servers and a remote memory end composed of multiple memory servers interconnected through a remote direct memory access RDMA network, and a single computing server can be connected to multiple memory servers simultaneously to expand its available storage capacity; a data processor DPU is loaded between the host end and the remote memory end; the DPU includes cascaded dynamic random access memories DRAM and a central processing unit CPU, wherein the DRAM is used to cache data of the computing server stored at the remote memory end; the CPU is used to provide an operating environment for aggregating the available memory of the remote memory end.

[0031] The CPU includes four parts: a cascaded cache management module, a remote memory mapping management module, a remote memory service module, and a remote memory recycling module connected to the input end of the remote memory mapping management module, wherein:

[0032] The cache management module searches for the block storage address of the read request or write request sent by the computing server in the LRU and FIFO queues of the DRAM, and distributes the read request or write request to the DRAM or the remote memory mapping management module according to the search result.

[0033] The remote memory mapping management module manages the remote memory mapping table to record the mapping relationship between the block storage page address and the page address in the memory server at the remote memory end. The mapping table not only contains the specific information of the allocated pages in the memory server, but also reflects the distribution and occupancy rate of the current memory resources. When the cache management module fails to find the required data in the LRU and FIFO queues of the DRAM, it will quickly locate the memory server containing the target page through the mapping table, and send read requests and write requests to the remote memory service module according to the mapping results, and jointly update the memory status in real time with the remote memory service module.

[0034] The remote memory service module uses a B+ tree to manage the memory occupancy of the memory server, and uses the RDMA network to send data to the memory server and read data from the memory server, so as to realize the read / write operation of the memory server. When selecting a memory server to write data, the present invention uses a pre-placement strategy to keep the memory resource usage status of the memory server balanced.

[0035] The remote memory recycling module is used to periodically release the invalid memory in the memory server according to the mapping relationship of the remote memory mapping management module. Since the DPU adopts a remote update mechanism, over time and with the accumulation of multiple write operations, some data may become obsolete or invalid, and the storage space occupied by these invalid data needs to be recycled. The remote memory recycling module will periodically compare the quotient of the number of valid memory pages of each memory server and the total number of memory pages with a preset recycling threshold according to the mapping table of the remote memory mapping management module. If the quotient is less than the recycling threshold, it will search for the invalid memory pages in the mapping table of the remote memory mapping management module, delete the information of the found invalid memory pages from the mapping table, and release these invalid memory pages in the memory server at the same time.

[0036] Refer to Figure 2 , a management method for a split memory system, includes the following steps:

[0037] Step 1) The cache management module initializes the DRAM memory space:

[0038] The cache management module divides the DRAM memory space into multiple memory pages with a fixed capacity, and LRU queues and FIFO queues with a configurable capacity ratio, and assigns a unique identification number to each memory page.

[0039] Step 2) The computing server issues read / write requests:

[0040] When the computing server runs out of memory, the operating system will write the pages that are not frequently used in the computing server from the memory to the DPU, or read the data in the DPU to the memory. Therefore, when accessing the remote memory server, read / write requests will be issued.

[0041] Step 3) The cache management module manages read / write requests:

[0042] (3a) The cache management module checks whether the FIFO queue is full. If the FIFO queue is full, it converts the address at the end of the FIFO queue and the data pointed to by the memory page number into a write request and sends it to the remote memory mapping management module; otherwise, it does nothing.

[0043] (3b) It checks whether the LRU queue is full. If the LRU queue is full, it evicts the address and memory page number at the tail of the LRU queue to the head of the FIFO queue; otherwise, it does nothing.

[0044] (3c) It writes the data of the write request into the memory page in the DRAM memory space, and at the same time inserts the memory page number and the address of the write request into the head of the LRU queue.

[0045] (3d) The cache management module checks whether the read request hits in the DRAM cache. If it does, it returns the data to the computing server and inserts the memory page where the data is located and the address of the read request into the head of the LRU queue; otherwise, it reads from the target memory server according to the mapping table managed by the remote memory mapping management module, returns the read data to the computing server and writes it into the memory page in the DRAM memory space, and then inserts the memory page number and the address of the read request into the head of the LRU queue.

[0046] Step 4) The remote memory mapping management module maintains the mapping table and transfers read / write requests:

[0047] (4a) The remote memory mapping management module checks from the mapping table whether the read request sent by the cache management module has been allocated according to the address of the read request. If it has, it fills the address into the read request and forwards it to the remote memory service module; otherwise, it returns a read error message to the cache management module.

[0048] (4b) The remote memory mapping management module requests to allocate a memory page from the remote memory service module according to the write request sent by the cache management module.

[0049] Step 5) The remote memory service module processes read / write requests:

[0050] The remote memory service module records the page address information in the remote memory server in a B+ tree structure. When it receives a read / write request sent from the remote memory mapping management module, the processing methods are as follows:

[0051] (5a) After the remote memory service module receives a read request sent from the remote memory mapping management module, it finds the memory server mapped by the address of the read request and reads the data from the target memory server and returns it to the remote memory mapping management module.

[0052] After the remote memory service module receives the write request sent from the remote memory mapping management module in (5b), it queries the memory occupancy rate of each memory server at the remote memory end, and then calculates the standard deviation between the memory occupancy rate of the memory server after placing the data of the write request on each memory server in turn and the average memory occupancy rate of all memory servers. Then, it selects the memory server with the smallest standard deviation as the target memory server to write the data of the write request, where :

[0053] ;

[0054] where is the number of memory servers, is the memory occupancy rate of the th memory server, , is the average memory occupancy rate of all memory servers;

[0055] In (5c), the remote memory service module uses the RDMA network to send data to the memory server and reads data from the memory server. The specific operation is to initiate a mapping request using the RDMA bilateral send SEND operation. The memory server receives it through the RECEIVE operation and feeds back the execution result through the bilateral operation as well. The read and write operations of the data are implemented through the RDMA unilateral operations READ / WRITE.

[0056] Step 6) The remote memory recycling module releases the invalid memory pages in the memory server:

[0057] In (6a), the remote memory recycling module periodically determines whether the quotient of the number of valid memory pages of each memory server and the total number of memory pages is less than the preset recycling threshold. If so, it searches for the invalid memory pages in the mapping table of the remote memory mapping management module and records the information of these invalid memory pages.

[0058] In (6b), the remote memory recycling module deletes the information of the found invalid memory pages from the mapping table and releases the invalid memory pages in the memory server.

[0059] The cache management module in the CPU of the present invention uses the DRAM as the local cache of the remote memory, so that the data frequently accessed by the computing server is persistently cached in the DRAM of the DPU. The high-frequency access requests can be directly responded to through the local cache, avoiding excessive RDMA network transmissions, reducing the access latency of cache hit requests, and thus significantly reducing the overall read / write latency. Experimental data shows that the average latency of the present invention is reduced by about 30% compared with the prior art; at the same time, the available block storage of the computing server also increases as the number of memory servers increases.

[0060] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be covered by the protection scope of the present invention.

Claims

1. A split memory system, comprising a host end consisting of a plurality of computing servers and a remote memory end consisting of a plurality of memory servers interconnected via a remote direct memory access (RDMA) network; characterized in that: A data processor DPU is loaded between the host end and the remote memory end; the DPU includes a cascaded dynamic random access memory DRAM and a central processing unit CPU, wherein the DRAM is used to cache the data stored in the remote memory end by the computing server; the CPU is used to provide an operating environment for aggregating the available memory of the remote memory end.

2. The system according to claim 1, characterized in that The CPU comprises a cascaded cache management module, a remote memory mapping management module and a remote memory service module, and a remote memory recycling module connected to an input end of the remote memory mapping management module, wherein: A cache management module, used to search for the block storage page address of the read request or write request sent by the computing server, and to divert the read request or write request to the DRAM or remote memory mapping management module according to the search result; A remote memory mapping management module is used to maintain the mapping relationship between the block storage page address and the memory server memory page address, and send the read / write request to the remote memory service module according to the mapping result; A remote memory service module is used to read / write the memory space of the memory server; The remote memory recycling module is used to periodically release invalid memory in the memory server according to the mapping relationship of the remote memory mapping management module.

3. A method for managing a separate memory system, characterized in that: The steps include: (1) The cache management module initializes the DRAM memory space: The cache management module initializes the DRAM memory space including multiple memory pages, as well as the LRU queue and the FIFO queue; (2) The computing server sends a read / write request; (3) The cache management module manages read / write requests: The block storage address of the read request or write request sent by the computing server is searched through the LRU queue and the FIFO queue, and the read request or write request is diverted to the DRAM or remote memory mapping management module according to the search result; (4) The remote memory mapping management module maintains the mapping table and transmits read / write requests: The remote memory mapping management module searches for the mapped memory server address in the mapping table through the addresses of the read request and the write request, writes the memory server address corresponding to the read request into the read request, and sets the memory server address corresponding to the write request to invalid, and then sends the write request to the remote memory service module; (5) The remote memory service module processes read / write requests: The remote memory service module reads data from the memory server according to the memory server address of the read request and returns it to the remote memory mapping management module; Collect the memory usage of the memory servers and write the data of the write request to the memory server with the lowest memory utilization; (6) The remote memory recovery module releases invalid memory pages in the memory server: The remote memory recovery module periodically releases invalid memory in the memory server according to the mapping table of the remote memory mapping management module.

4. The method according to claim 3, characterized in that: The cache management module described in step (1) initializes the DRAM memory space, and the implementation steps are as follows: The cache management module divides the DRAM memory space into multiple memory pages of fixed capacity, as well as LRU queues and FIFO queues with configurable capacity ratios, and assigns a unique identification number to each memory page.

5. The method according to claim 3, characterized in that: The cache management module described in step (3) manages the read / write request, and the implementation steps are as follows: (3a) The cache management module checks the FIFO queue. When the FIFO queue is full, the address at the end of the queue and the data pointed to by the memory page number are converted into a write request and sent to the remote memory mapping management module. Then the LRU queue is checked. When the LRU queue is full, the address at the end of the queue and the memory page number are driven to the head of the FIFO queue. (3b) The cache management module writes the data of the write request into the memory page of the DRAM memory space, and inserts the memory page number and the address of the write request into the head of the LRU queue; (3c) The cache management module checks whether the read request hits the DRAM cache. If so, it returns the data to the computing server and inserts the memory page where the data is located and the address of the read request into the head of the LRU queue. Otherwise, a read request is initiated to the remote memory mapping management module, the read data is returned to the computing server and written to the memory page of the DRAM memory space, and the memory page number and the address of the read request are inserted into the head of the LRU queue.

6. The method according to claim 3, characterized in that The mapping table described in step (4) includes multiple triplets, each of which contains block storage memory page number information, memory server memory page address information and valid flag bit information, the block storage memory page number and the memory server memory page address are composed of 64 bits, and the valid flag bit is composed of 1 bit.

7. The method according to claim 3, characterized in that The memory server address corresponding to the write request is set to invalid in step (4). The implementation method is: The remote memory mapping management module sets the valid mark position of the triplet corresponding to the block storage memory page number carried by the write request in the mapping table to 0, and writes the memory server page address information corresponding to the mapping table into the write request.

8. The method according to claim 3, characterized in that The remote memory service module described in step (5) processes the read / write request, and the implementation steps are as follows: (5a) After receiving the read request sent from the remote memory mapping management module, the remote memory service module searches for the memory server where the address of the read request is located, reads data from the target memory server, and returns it to the remote memory mapping management module; (5b) After receiving the write request sent from the remote memory mapping management module, the remote memory service module queries the memory occupancy rate of each memory server on the remote memory end, and calculates the standard deviation of the memory occupancy rate on each memory server and the average memory occupancy rate of all memory servers after writing the write request data to each memory server. , select the memory server with the smallest standard deviation as the target memory server to write the data of the write request, where: ； in, is the number of memory servers, For the The memory usage of the memory server, , is the average memory usage of all memory servers; (5c) The remote memory service module uses the RDMA network to interact with the memory server to read data for read requests and write data for write requests.

9. The method according to claim 3, characterized in that: The remote memory recovery module described in step (6) releases the invalid memory pages in the memory server, and the implementation steps are as follows: (6a) The remote memory recycling module periodically determines whether the quotient of the number of valid memory pages of each memory server and the number of all memory pages is less than a preset recycling threshold. If so, it searches for invalid memory pages in the mapping table of the remote memory mapping management module and records information of these invalid memory pages. (6b) The remote memory recycling module deletes the invalid memory page information found from the mapping table and releases the invalid memory page in the memory server.

Images