A persistent memory storage method and system based on RDMA
By managing persistent memory regions and data copying mechanisms through the RDMA network card at the receiving end, the problems of write amplification and CPU latency in the RDMA storage method are solved, achieving low-latency and high-efficiency persistent memory storage, reducing write amplification and lowering CPU utilization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2022-05-27
- Publication Date
- 2026-06-30
AI Technical Summary
Existing persistent memory storage methods based on RDMA are prone to write amplification and system performance degradation when write requests are small, especially when there is multi-threaded concurrent access. Existing methods cannot effectively reduce write amplification and CPU latency.
The receiver periodically allocates free persistent memory regions through the RDMA network card, generates new receive block descriptors, determines whether the memory region is full, and copies write request data to the end of the memory region pointed to by the descriptor. Data storage is performed using the RDMA network connection, avoiding CPU participation in the critical path, and employing reliable connection and anomaly detection mechanisms.
It effectively reduces write amplification of persistent memory, improves storage performance, reduces latency and CPU utilization, and achieves low-latency, high-efficiency persistent memory access.
Smart Images
Figure CN117171059B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of non-volatile main memory storage technology, and in particular to a persistent memory storage method and system based on RDMA. Background Technology
[0002] Persistent memory is a new type of storage medium. Similar to regular memory, persistent memory can be directly plugged into the server's memory slots, allowing the CPU to read and write to it using load / store instructions. Like traditional hard drives, persistent memory can persistently store data, meaning data is not lost after power failure. Intel launched its first commercially viable persistent memory at scale, Optane persistent memory, in 2019. Optane persistent memory is available in three single-module capacities: 128GB, 256GB, and 512GB. A single Optane persistent memory module has a write bandwidth of approximately 2GB / s and a read bandwidth of approximately 6GB / s. The internal media read / write granularity of Optane persistent memory is 256 bytes; therefore, when the CPU issues a persistent memory write request smaller than 256 bytes, Optane persistent memory will trigger a read-after-write operation, causing write amplification and severely impacting the available bandwidth of the persistent memory. Optane persistent memory has an internal write merging buffer that merges write requests with adjacent addresses before writing them to the medium, thus avoiding expensive read-after-write operations.
[0003] Remote Direct Memory Access (RDMA) is a high-speed network technology characterized by low latency (round-trip latency less than 2 microseconds) and high bandwidth (100Gbps). RDMA supports two types of communication primitives: message primitives and memory primitives. Message primitives include `send` and `receive`. The receiving end needs to initialize the receive queue using the `receive` primitive; the sending end uses the `send` primitive to send data to the receiving end. Memory primitives include `write` and `read`. The sending end directly reads and writes to the receiving end's memory using memory primitives, and the receiving end's CPU is unaware of this. Compared to message primitives, memory primitives offer even lower latency and CPU utilization.
[0004] Accessing persistent memory using RDMA network connections can provide low-latency, high-bandwidth remote storage services. However, both existing methods suffer from significant performance issues. The first method uses RDMA's write primitives to perform write operations on the receiving end's persistent memory; however, when there are many concurrent accesses from sending threads and the write requests are small, this method can lead to severe persistent memory write amplification. This is because the addresses of write requests from different sending threads are generally not contiguous, resulting in a large number of random small writes to the receiving end's persistent memory, leading to severe device write amplification. The second method uses RDMA-based Remote Procedure Calls (RPCs) for remote persistent memory write operations. This method introduces queuing and execution latency on the receiving end's CPU along the critical path of the write operation, resulting in degraded system performance. Summary of the Invention
[0005] The purpose of this invention is to at least address one of the aforementioned technical deficiencies.
[0006] Therefore, one objective of this invention is to provide a persistent memory storage method and system based on RDMA. This can effectively reduce device write amplification of persistent memory and improve its storage performance. To achieve the above objective, this invention provides the following technical solution: a persistent memory storage method and system based on RDMA.
[0007] This invention provides a persistent memory storage method based on RDMA, comprising the following steps:
[0008] The receiving end obtains persistent memory write requests through an RDMA network connection. The receiving end is equipped with an RDMA network card and persistent memory. The persistent memory supports direct reading and writing by the receiving end's network card. The background thread of the receiving end periodically allocates idle persistent memory areas and generates new receive block descriptors to add to the shared receive queue.
[0009] If the receiving RDMA network card determines that the persistent memory area pointed to by the current receive block descriptor is full, it retrieves a new receive block descriptor from the shared receive queue and uses it as the current receive block descriptor.
[0010] The receiving RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor, where the receive block descriptor is used to describe a persistent memory region.
[0011] The receiving RDMA network card will return an acknowledgment message to the sending thread of the write request.
[0012] Furthermore, the receiving end obtains persistent memory write requests through the RDMA network connection, specifically,
[0013] The receiving end obtains persistent memory write requests sent by one or more sending ends through an RDMA network connection. The sending end is also equipped with an RDMA network card, and multiple sending end threads are located on the same sending end or different sending ends.
[0014] Furthermore, the receiving RDMA network card determines that the persistent memory region pointed to by the current receive block descriptor is full. Specifically,
[0015] If the sum of the tail offset of the current receive block descriptor and the number of bytes of the write request data is greater than the number of bytes of the current receive block descriptor, the receiving RDMA network card determines that the persistent memory region pointed to by the current receive block descriptor is full.
[0016] The receive block descriptor includes the starting address of the persistent memory region and the number of bytes in the persistent memory region.
[0017] Furthermore, the receiving network card determines that the persistent memory region pointed to by the current receive block descriptor is full, retrieves a new receive descriptor from the shared receive queue as the current receive block descriptor, updates the address and number of bytes of the persistent memory region pointed to by the current receive block descriptor accordingly, sets the tail offset of the current receive block descriptor to the initial value 0, and copies the data in the write request to the tail of the persistent memory region pointed to by the current receive block descriptor.
[0018] Furthermore, the receiving network card stores the data from write requests from different RDMA connections into the persistent memory region pointed to by the receive block descriptor in the shared receive queue.
[0019] Furthermore, the receiving RDMA network card writes the data from persistent memory write requests from different sending threads into a contiguous address space of the persistent memory region.
[0020] Furthermore, the receiving RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor, specifically,
[0021] The receiving RDMA network card stores data on persistent memory media.
[0022] The persistent memory write request mainly consists of an RDMA send command, which includes the data and the number of bytes of that data.
[0023] Furthermore, the receiving RDMA network card stores data on persistent memory medium, specifically,
[0024] After the receiving RDMA network card sends a PCIe write operation to move the data to persistent memory, the receiving RDMA network card also sends a 1-byte PCIe read operation.
[0025] The RDMA send is asynchronous, and the address of the 1-byte PCIe read operation can be any memory address.
[0026] Furthermore, the RDMA network card detects whether any abnormal events have occurred on the hardware. When an abnormal event occurs, the RDMA network card retransmits the network messages.
[0027] The RDMA network connection is a reliable connection.
[0028] Furthermore, the background thread of the receiving end maintains a persistent memory allocator that records free persistent memory regions.
[0029] Furthermore, the receiving end records the free persistent memory region, specifically,
[0030] The background thread of the receiving end locates the idle region bitmap and records the idle persistent memory region.
[0031] Furthermore, the receiving end background thread locates the free region bitmap and records the free persistent memory regions, specifically,
[0032] The free region bitmap is stored in the reserved space of persistent memory. The background thread of the receiving end adds the free persistent memory region to the shared receive queue, and the position corresponding to the free region bitmap is marked.
[0033] Each time the free area bitmap is modified, the CPU calls the persistence instruction.
[0034] This invention provides a persistent memory storage system based on RDMA, comprising:
[0035] The receiving end includes a receiving end RDMA network card and persistent memory, wherein,
[0036] Persistent memory is installed on the receiving end and supports direct reading and writing by the receiving end's RDMA network card; wherein, the background thread of the receiving end periodically allocates idle persistent memory areas and generates new receive block descriptors to add to the shared receive queue;
[0037] The receiving RDMA network card, installed on the receiving end, is used to receive persistent memory write requests, determine whether the persistent memory region pointed to by the current receive block descriptor is full, and if the persistent memory region pointed to by the current receive block descriptor is full, retrieve a new receive block descriptor from the shared receive queue as the current receive block descriptor. The receiving RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor. It is also used to return an acknowledgment message to the sending thread of the write request, wherein the receive block descriptor is used to describe a persistent memory region.
[0038] Furthermore, the receiving end obtains persistent memory write requests sent by one or more sending ends through an RDMA network connection. The sending end is equipped with an RDMA network card, and the sending end threads are located on different sending ends. Multiple sending end threads are located on the same sending end or different sending ends.
[0039] Furthermore, the receiving RDMA network card is also used to determine that the persistent memory region pointed to by the current receive block descriptor is full, specifically,
[0040] The receive block descriptor includes the starting address of the persistent memory region and the number of bytes in the persistent memory region;
[0041] The receiving RDMA network card is used to record the tail offset of the current receive block descriptor. After the receiving RDMA network card copies the data in the write request to the tail of the persistent memory area pointed to by the current receive block descriptor, if the sum of the tail offset value and the number of bytes of the data is greater than the number of bytes of the current receive block descriptor, the receiving RDMA network card determines that the persistent memory area pointed to by the current receive block descriptor is full.
[0042] Furthermore, the receiving RDMA network card determines that the persistent memory region pointed to by the current receive block descriptor is full, retrieves a new receive descriptor from the shared receive queue as the current receive block descriptor, updates the address and number of bytes of the persistent memory region pointed to by the current receive block descriptor accordingly, sets the tail offset of the current receive block descriptor to the initial value 0, and copies the data in the write request to the tail of the persistent memory region pointed to by the current receive block descriptor.
[0043] Furthermore, the receiving RDMA network card is used to store the data in write requests from different RDMA network connections into the persistent memory area pointed to by the receive block descriptor in the shared receive queue;
[0044] The receiving RDMA network card is also used to write data from persistent memory write requests from different sending threads into a contiguous address space of the persistent memory region.
[0045] Furthermore, the receiving RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor, specifically,
[0046] The receiving RDMA network card stores data on persistent memory media, rather than temporarily storing it in the network card or CPU cache.
[0047] Furthermore, the RDMA network card can detect whether any abnormal events have occurred on the hardware. When an abnormal event occurs, the RDMA network card retransmits the network messages, and the RDMA network connection is a reliable connection.
[0048] Furthermore, the background thread of the receiving end maintains a persistent memory allocator to record free persistent memory regions, and the CPU calls the persistent instruction every time it is modified.
[0049] The technical effects and advantages of this invention are as follows: The persistent memory storage method and system based on RDMA according to this invention can improve the performance of accessing remote persistent memory in an RDMA environment and reduce write amplification of persistent memory. Furthermore, this method does not involve the CPU at the receiving end in the write request path, thus offering advantages of low latency and low CPU utilization.
[0050] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description
[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0052] Figure 1 This is a schematic diagram of the structure of a persistent memory storage system based on RDMA in an embodiment of the present invention;
[0053] Figure 2 This is a flowchart of the process of writing requests by the receiving end RDMA network card in an embodiment of the present invention;
[0054] Figure 3 This is a schematic diagram illustrating how the receiving RDMA network card copies data to persistent memory in an embodiment of the present invention;
[0055] Figure 4 This is a schematic diagram of the background thread of the receiving end managing persistent memory in an embodiment of the present invention. Detailed Implementation
[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0057] To address the shortcomings of existing technologies, embodiments of the present invention provide a persistent memory storage method and system based on RDMA, such as... Figure 1 As shown, it includes:
[0058] Both the transmitting and receiving ends are equipped with RDMA network cards and communicate via an RDMA network connection. The receiving end also has persistent memory, and the transmitting threads can reside on different transmitting ends. The receiving end is used to obtain persistent memory write requests sent by the transmitting end via the RDMA network connection. Specifically, the RDMA network can be based on different physical layers, including InfiniBand (Radio Bandwidth: a next-generation network protocol supporting RDMA) and RoCE (RDMA over Converged Ethernet, a network protocol that allows RDMA to be performed over Ethernet). InfiniBand requires a dedicated switch and RDMA network card support; RoCE runs on a regular Ethernet network and only requires the network card to support RDMA functionality. The persistent memory is installed in the receiver's memory slot and can be directly read and written by the RDMA network card.
[0059] One or more sending ends send persistent memory write requests to the receiving end via an RDMA network connection. When the receiving end RDMA network card receives the persistent memory write request, if it determines that the persistent memory region pointed to by the current receive block descriptor is full, it retrieves a new receive block descriptor from the shared receive queue as the current receive block descriptor. Then, the receiving end RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor. Finally, the receiving end RDMA network card returns an acknowledgment message to the sending thread of the write request.
[0060] The shared receive queue stores a set of receive block descriptors, each describing a persistent memory region. The receive block descriptor includes the starting address of the persistent memory region and the number of bytes in the persistent memory region. Figure 1 In this configuration, the receive block descriptor is 4MB in size, with its starting address ranging from 0x000000 to 0x800000. The background thread on the receiving end periodically allocates free persistent memory areas and generates new receive block descriptors, adding them to the shared receive queue.
[0061] Furthermore, such as Figure 2 As shown, the receiving RDMA network card determines whether the persistent memory region pointed to by the current receive block descriptor is full. Specifically, the receiving RDMA network card receives a persistent memory write request, where the number of data bytes in the write request is m; then the receiving RDMA network card reads the current receive block descriptor and records the tail offset of the persistent memory region pointed to by the current receive block descriptor; let the tail offset value be tail, the address of the persistent memory region pointed to by the current receive block descriptor be addr, and the number of bytes be n.
[0062] After the receiving RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor, if the sum of the tail offset and the number of bytes of data is not greater than the number of bytes in the current receive block descriptor, the receiving RDMA network card directly copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor. Conversely, if the sum of the tail offset and the number of bytes of data is greater than the number of bytes in the current receive block descriptor (tail + m > n), the receiving RDMA network card determines that the persistent memory region pointed to by the current receive block descriptor is full. It then retrieves a new receive block descriptor from the shared receive queue as the current receive block descriptor, and updates the address addr and the number of bytes n of the persistent memory region pointed to by the current receive block descriptor accordingly. The offset tail of the current receive block descriptor is initialized to 0, and then the data is copied to the end of the persistent memory region pointed to by the current receive block descriptor. At this time, the tail offset is equal to the tail offset before copying plus the number of bytes of data, tail = tail + m. Finally, the receiving RDMA network card returns an acknowledgment message to the sending thread of the write request.
[0063] Furthermore, the receiver-side RDMA network card can be used to store data from write requests from different RDMA network connections into a persistent memory region pointed to by the receive block descriptor in the shared receive queue; the receiver-side RDMA network card can also be used to write data from persistent memory write requests from different sender threads into a contiguous address space within the persistent memory region. Therefore, the write merging buffer within persistent memory can merge these write operations into the medium, eliminating expensive read-after-write operations and reducing device write amplification of persistent memory. In addition, as shown in the figure, the receiver-side CPU is not involved in the critical path of the entire data operation process, thus fully utilizing hardware performance to achieve the goal of low latency.
[0064] Furthermore, persistent memory write requests mainly consist of RDMA send commands, including data and the number of bytes of that data; the sending thread notifies the sending RDMA network card to generate an RDMA send network packet with the destination address of the receiving end by calling the RDMA user-space driver. Figure 2 This illustration demonstrates the process of a receiving RDMA network card handling write requests in this invention. Specifically, existing RDMA network cards support multiple RDMA send requests being appended to the same address region pointed to by the receive block descriptor. The RDMA network card maintains necessary metadata in SRAM (Static Random-Access Memory) space, including the tail offset of the current receive block descriptor. The RDMA network card guarantees the correctness of concurrency, meaning that different write requests will not be written to the same address.
[0065] Furthermore, such as Figure 3 As shown, when the receiving RDMA network card copies the data in the write request to the persistent memory region pointed to by the current receive block descriptor, it stores the data on the persistent memory medium and does not temporarily store it in the network card (memory controller) or CPU cache. Specifically, since RDMA send is asynchronous, meaning the receiving network card returns an acknowledgment before the data successfully reaches persistent memory, to ensure data persistence, after sending a PCIe (peripheral component interconnect express, a high-speed serial computer expansion bus standard) write operation to move the data to persistent memory, the receiving RDMA network card also needs to send a 1-byte PCIe read operation to ensure the previous PCIe write operation was successfully completed, further guaranteeing data persistence. The address of the 1-byte PCIe read operation can be any memory address.
[0066] Furthermore, the RDMA network connection is a reliable connection. The RDMA network card guarantees the reliability of network messages; that is, when the sending end receives the acknowledgment information from the receiving end's RDMA network card, it can be guaranteed that the network message has been successfully executed by the receiving end's network card. The receiving end's RDMA network card can detect abnormal events such as network packet loss in hardware. When network packet loss occurs, the receiving end's RDMA network card retransmits the network message to ensure that some network messages can safely and reliably reach their destination. The sending end thread establishes an RDMA network connection with the receiving end during initialization.
[0067] Furthermore, such as Figure 4As shown, the persistent memory region is divided into free and used regions. The receiving end background thread maintains a persistent memory allocator to record and manage free persistent memory regions. To ensure the consistency of the free region bitmap after a crash (i.e., after the receiving server crashes and restarts, the bitmap can accurately locate the free persistent memory region), this free region bitmap is stored in a reserved space in persistent memory, and the CPU calls a persistence instruction each time it is modified. When the receiving end background thread adds a free persistent memory region to the shared receive queue, the corresponding position in the free region bitmap is marked. The white and black squares in the free region bitmap mark the free and used regions, respectively.
[0068] The persistent memory storage method based on RDMA according to the present invention can improve the performance of accessing remote persistent memory in an RDMA environment and reduce write amplification of persistent memory. Furthermore, this method does not involve the CPU at the receiving end in the write request path, thus offering advantages of low latency and low CPU utilization.
[0069] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A persistent memory storage method based on RDMA, characterized in that, The method includes, The receiving end obtains persistent memory write requests through an RDMA network connection. The receiving end is equipped with an RDMA network card and persistent memory. The persistent memory supports direct reading and writing by the receiving end's network card. The background thread of the receiving end periodically allocates idle persistent memory areas and generates new receive block descriptors to add to the shared receive queue. If the receiving RDMA network card determines that the persistent memory area pointed to by the current receive block descriptor is full, it retrieves a new receive block descriptor from the shared receive queue and uses it as the current receive block descriptor. The receiving RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor, where the receive block descriptor is used to describe a persistent memory region. The receiving RDMA network card will return an acknowledgment message to the sending thread of the write request.
2. The persistent memory storage method based on RDMA according to claim 1, wherein, The receiving end obtains persistent memory write requests through an RDMA network connection, specifically: The receiving end obtains persistent memory write requests sent by one or more sending ends through an RDMA network connection. The sending end is also equipped with an RDMA network card, and multiple sending end threads are located on the same sending end or different sending ends.
3. A persistent memory storage method based on RDMA according to claim 1 or 2, wherein, The receiving RDMA network card determines that the persistent memory region pointed to by the current receive block descriptor is full. Specifically, If the sum of the tail offset of the current receive block descriptor and the number of bytes of the write request data is greater than the number of bytes of the current receive block descriptor, the receiving RDMA network card determines that the persistent memory region pointed to by the current receive block descriptor is full. The receive block descriptor includes the starting address of the persistent memory region and the number of bytes in the persistent memory region.
4. A persistent memory storage method based on RDMA according to claim 1 or 2, characterized in that, The receiving network card determines that the persistent memory region pointed to by the current receive block descriptor is full, retrieves a new receive descriptor from the shared receive queue as the current receive block descriptor, updates the address and number of bytes of the persistent memory region pointed to by the current receive block descriptor accordingly, sets the tail offset of the current receive block descriptor to the initial value 0, and copies the data in the write request to the tail of the persistent memory region pointed to by the current receive block descriptor.
5. The persistent memory storage method based on RDMA according to claim 1, characterized in that, The receiving network card stores the data from write requests from different RDMA connections into the persistent memory region pointed to by the receive block descriptor in the shared receive queue.
6. The persistent memory storage method based on RDMA according to claim 5, characterized in that, The receiving RDMA network card writes data from persistent memory write requests from different sending threads into a contiguous address space of the persistent memory region.
7. The persistent memory storage method based on RDMA according to claim 1, wherein, The receiving RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor, specifically, The receiving RDMA network card stores data on persistent memory media. The persistent memory write request mainly consists of an RDMA send command, which includes the data and the number of bytes of that data.
8. A persistent memory storage method based on RDMA according to claim 7, wherein, The receiving RDMA network card stores data on persistent memory medium, specifically, After the receiving RDMA network card sends a PCIe write operation to move the data to persistent memory, the receiving RDMA network card also sends a 1-byte PCIe read operation. The RDMA send is asynchronous, and the address of the 1-byte PCIe read operation can be any memory address.
9. A persistent memory storage method based on RDMA according to claim 1 or 2, characterized in that, The RDMA network card detects whether any abnormal events have occurred in the hardware. When an abnormal event occurs, the RDMA network card retransmits the network messages. The RDMA network connection is a reliable connection.
10. A persistent memory storage method based on RDMA according to claim 1, characterized in that, The background thread of the receiving end maintains a persistent memory allocator that records free persistent memory regions.
11. A persistent memory storage method based on RDMA according to claim 10, wherein, The receiving end records a free persistent memory region, specifically, The background thread of the receiving end locates the idle region bitmap and records the idle persistent memory region.
12. A persistent memory storage method based on RDMA according to claim 11, wherein, The receiving end background thread locates the free region bitmap and records the free persistent memory regions, specifically, The free region bitmap is stored in the reserved space of persistent memory. The background thread of the receiving end adds the free persistent memory region to the shared receive queue, and the position corresponding to the free region bitmap is marked. Each time the free area bitmap is modified, the CPU calls the persistence instruction.
13. A persistent memory storage system based on RDMA, the system comprising: The receiving end includes a receiving end RDMA network card and persistent memory, wherein, Persistent memory is installed on the receiving end and supports direct reading and writing by the receiving end's RDMA network card; wherein, the background thread of the receiving end periodically allocates idle persistent memory areas and generates new receive block descriptors to add to the shared receive queue; The receiving RDMA network card, installed on the receiving end, is used to receive persistent memory write requests, determine whether the persistent memory region pointed to by the current receive block descriptor is full, and if the persistent memory region pointed to by the current receive block descriptor is full, retrieve a new receive block descriptor from the shared receive queue as the current receive block descriptor. The receiving RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor. It is also used to return an acknowledgment message to the sending thread of the write request, wherein the receive block descriptor is used to describe a persistent memory region.
14. A persistent memory storage system based on RDMA according to claim 13, characterized in that, The receiving end obtains persistent memory write requests sent by one or more sending ends through an RDMA network connection. The sending end is equipped with an RDMA network card. The sending end threads are located on different sending ends, and multiple sending end threads are located on the same sending end or different sending ends.
15. A persistent memory storage system based on RDMA according to claim 13 or 14, wherein, The receiving RDMA network card is also used to determine that the persistent memory region pointed to by the current receive block descriptor is full, specifically, The receive block descriptor includes the starting address of the persistent memory region and the number of bytes in the persistent memory region; The receiving RDMA network card is used to record the tail offset value of the current receive block descriptor. After the receiving RDMA network card copies the data in the write request to the tail of the persistent memory area pointed to by the current receive block descriptor, if the sum of the tail offset value and the number of bytes of the data is greater than the number of bytes of the current receive block descriptor, the receiving RDMA network card determines that the persistent memory area pointed to by the current receive block descriptor is full.
16. A persistent memory storage system based on RDMA according to claim 13 or 14, characterized in that, The receiving RDMA network card determines that the persistent memory region pointed to by the current receive block descriptor is full, retrieves a new receive descriptor from the shared receive queue as the current receive block descriptor, updates the address and number of bytes of the persistent memory region pointed to by the current receive block descriptor accordingly, sets the tail offset of the current receive block descriptor to the initial value 0, and copies the data in the write request to the tail of the persistent memory region pointed to by the current receive block descriptor.
17. A persistent memory storage system based on RDMA according to claim 13 or 14, characterized in that, The receiving RDMA network card is used to store the data in write requests from different RDMA network connections into the persistent memory area pointed to by the receive block descriptor in the shared receive queue; The receiving RDMA network card is also used to write data from persistent memory write requests from different sending threads into a contiguous address space of the persistent memory region.
18. A persistent memory storage system based on RDMA according to claim 13, wherein, The receiving RDMA network card copies the data in the write request to the end of the persistent memory region pointed to by the current receive block descriptor, specifically, The receiving RDMA network card stores data on persistent memory media, rather than temporarily storing it in the network card or CPU cache.
19. A persistent memory storage system based on RDMA according to claim 13 or 14, characterized in that, The RDMA network card can detect whether any abnormal events have occurred on the hardware. When an abnormal event occurs, the RDMA network card retransmits the network message, and the RDMA network connection is a reliable connection.
20. A persistent memory storage system based on RDMA according to claim 13, characterized in that, The background thread of the receiving end maintains a persistent memory allocator to record free persistent memory areas, and the CPU calls the persistent instruction every time it is modified.