Remote writing method and device of data, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a kind of data remote write method, device, electronic equipment and storage medium, involve data transceiver technical field, the method comprises: in the process of remote write, by target remote key in data packet, sending node can directly specify the target thread of receiving node, data is directly written into the memory space of this thread special after reaching, support thread level accurate delivery, and by message total length, dynamically reserve space for target message in shared memory area, effectively improve memory utilization, and based on the message identifier and segment offset carried by data packet, in the process of writing, so that each data segment can be independently identified and positioned, support disorderly receiving, improve the flexibility of data transmission.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data transceiving, in particular to a remote data writing method, a remote data writing device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] For remote direct memory access, a maximum size of a receiving buffer is consumed regardless of the message size, and a hole is easily formed because the message is smaller than the receiving buffer, resulting in low memory utilization. Moreover, in the process of transmitting a message, it is impossible to determine which message the message belongs to, and it is also impossible to determine the offset of the message in the message, resulting in not supporting out-of-order reception and not supporting multi-path load balancing of shared memory space. In addition, due to the sharing of the queue pair by multiple RX threads, the messages of each thread are mixed in a queue and the messages need to be distributed to each receiving thread again. SUMMARY

[0003] Embodiments of the present application provide a remote data writing method, device, electronic device and computer readable storage medium to solve or partially solve the problems of low memory utilization, not supporting out-of-order reception and needing multiple distribution in the process of remote direct memory access.

[0004] Embodiments of the present application disclose a remote data writing method, involving a sending node and a receiving node, establishing a queue pair between the sending node and the receiving node, and registering a corresponding shared memory area for each receiving thread in the receiving node, each shared memory area corresponding to a remote key; wherein the method comprises:

[0005] The sending node generates a target message, splits the target message into at least one data segment, generates a data packet corresponding to each data segment, and the data packet carries at least a target remote key for specifying a receiving thread, a message identifier for identifying the target message, a message total length of the target message and a segment offset corresponding to the data segment in the target message;

[0006] The receiving node receives the data packet through the queue pair, and locates a target receiving thread corresponding to the target remote key from the receiving thread;

[0007] The receiving node accesses a target shared memory area corresponding to the target receiving thread according to the target remote key, and reserves memory space for the target message or locates the reserved memory space in the target shared memory area based on the message identifier and the message total length;

[0008] The receiving node writes the payload in the data packet directly into the memory space where the target shared memory region is located based on the segment offset.

[0009] In the embodiment of the application, in the process of remote writing, the sending node can directly specify the target thread of the receiving node through the target remote key in the data packet, and the data is directly written into the memory space dedicated to the thread after reaching the thread, so that thread-level accurate delivery is supported, and the memory utilization is effectively improved by dynamically reserving space for the target message in the shared memory region through the total message length. In addition, each data segment can be independently identified and located in the writing process based on the message identifier and the segment offset carried by the data packet, so that out-of-order reception is supported, and the flexibility of data transmission is improved.

[0010] In some possible implementation manners, the receiving node maintains a message reservation table, the message reservation table includes a mapping relationship between a key value and a memory region offset, and the data packet further includes at least a source IP address and a queue pair number. The method of reserving memory space for the target message or locating the reserved memory space in the target shared memory region based on the message identifier and the total message length includes the following steps.

[0011] The source IP address in the data packet, the queue pair number, the message identifier, and the target remote key are used to construct a corresponding query key value.

[0012] If the query key value is found in the message reservation table, a memory region offset is obtained from a corresponding first mapping relationship, and the reserved memory space is located based on the memory region offset.

[0013] If the query key value is not found in the message reservation table, a current free offset of the target shared memory region is obtained, a second mapping relationship between the query key value and the current free offset is established in the message reservation table, memory space with the current free offset as a start and with the total message length as a length is reserved for the target message, and the current free offset of the target shared memory region is increased by the total message length.

[0014] In the embodiment of the application, by dynamically reserving memory space for the corresponding message in the shared memory region, memory waste caused by pre-allocating a buffer according to the maximum message size is avoided, and the memory utilization is improved.

[0015] In some possible implementation manners, the mapping relationship in the message reservation table further includes a remaining to-be-received length corresponding to the key value. After the second mapping relationship between the query key value and the current free offset is established in the message reservation table, the method further includes the following steps.

[0016] set the remaining to receive length corresponding to the second mapping relationship as the total length of the message.

[0017] In the embodiment of the present application, by setting the remaining to receive length as the total length of the corresponding message, the receiving progress of the target message can be effectively tracked.

[0018] In some possible implementation manners, the receiving node also maintains a corresponding memory protection table and a memory translation table, and the writing of the payload in the data packet directly into the memory space located at the target shared memory region based on the segment offset comprises:

[0019] adding the memory region offset and the segment offset in the data packet to obtain a target memory offset of the payload in the data packet in the target shared memory region;

[0020] adopting the target remote key and the target memory offset to perform address translation through the memory protection table and the memory translation table to obtain a target physical address for writing the payload;

[0021] writing the payload directly into the target physical address.

[0022] In the embodiment of the present application, by combining the logical memory offset (memory region offset and segment offset) with the hardware address translation mechanism (memory protection table and memory translation table), the receiving end network card can directly and correctly write the payload of any out-of-order arriving data segment into the final host physical memory address without relying on the CPU, which not only guarantees the legality and security of memory access, but also completely eliminates the overhead of kernel participation and data copying, and is the key to realizing low-delay and high-throughput data transmission.

[0023] In some possible implementation manners, after the writing of the payload directly into the target physical address, the method further comprises:

[0024] The receiving node subtracts the length of the payload that has been written from the remaining to receive length corresponding to the query key value.

[0025] In the embodiment of the present application, accurate and fine-grained message receiving progress tracking can be realized.

[0026] In some possible implementation manners, the method further comprises:

[0027] The receiving node generates a completion notification event on the completion queue associated with the queue in response to the remaining to receive length corresponding to the query key value decreasing to zero.

[0028] In some possible implementation manners, further comprising:

[0029] After the completion notification event is generated, the receiving node deletes the mapping corresponding to the query key in the message reservation table.

[0030] The embodiment of the application further discloses a device for remote writing of data, relating to a sending node and a receiving node, wherein a queue pair is established between the sending node and the receiving node, and a corresponding shared memory area is registered for each receiving thread in the receiving node, and each shared memory area corresponds to a remote key; wherein the device comprises:

[0031] A message processing module located in the sending node, configured to generate a target message, split the target message into at least one data segment, and generate a data packet corresponding to each data segment, wherein the data packet at least carries a target remote key used for specifying a receiving thread, a message identifier used for identifying the target message, a message total length of the target message, and a segment offset of the data segment in the target message;

[0032] A thread positioning module located in the receiving node, configured to receive the data packet through the queue pair, and locate a target receiving thread corresponding to the target remote key from the receiving thread;

[0033] A memory space positioning module located in the receiving node, configured to access a target shared memory area corresponding to the target receiving thread according to the target remote key, and reserve a memory space for the target message or locate a reserved memory space in the target shared memory area based on the message identifier and the message total length;

[0034] A data writing module located in the receiving node, configured to write a payload in the data packet directly into the memory space located by the target shared memory area based on the segment offset.

[0035] The embodiment of the application further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus.

[0036] The memory is used for storing a computer program.

[0037] The processor is used for executing the program stored on the memory, and realizes the method as described in the embodiment of the application.

[0038] The embodiment of the present application further discloses a computer readable storage medium, which stores instructions, and when the instructions are executed by one or more processors, the processors execute the method according to the embodiment of the present application.

[0039] The embodiment of the present application has the following advantages:

[0040] In the embodiment of the present application, a sending node and a receiving node are involved, a queue pair is established between the sending node and the receiving node, and a corresponding shared memory region is registered for each receiving thread in the receiving node, each shared memory region corresponding to a remote key, then the sending node remotely sends corresponding data to the receiving node, when the corresponding data is written on the receiving node, the sending node generates a target message, splits the target message into at least one data segment, generates a data packet corresponding to each data segment, the data packet at least carrying a target remote key for specifying a receiving thread, a message identifier for identifying the target message, a message total length of the target message and a segment offset of the data segment in the target message, then the receiving node receives the data packet through the queue pair, locates a target receiving thread corresponding to the target remote key from the receiving thread, then accesses a target shared memory region corresponding to the target receiving thread according to the target remote key, and reserves a memory space for the target message or locates a reserved memory space in the target shared memory region based on the message identifier and the message total length, finally, based on the segment offset, the payload in the data packet is directly written into the memory space located in the target shared memory region, so that in the process of remote writing, the sending node can directly specify a target thread of the receiving node through the target remote key in the data packet, the data is directly written into the memory space dedicated to the thread after reaching the data, thread-level accurate delivery is supported, and through the message total length, a space is dynamically reserved for the target message in the shared memory region, the memory utilization is effectively improved, and based on the message identifier and the segment offset carried in the data packet, each data segment can be independently identified and located in the process of writing, out-of-order receiving is supported, and the flexibility of data transmission is improved. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a step flow chart of a data remote writing method provided in the embodiment of the present application;

[0042] Figure 2 is a structural schematic diagram of a control plane provided in the embodiment of the present application;

[0043] Figure 3 is a schematic diagram of a message format provided in the embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a memory protection table provided in the embodiment of the present application;

[0045] Figure 5 is a schematic diagram of address translation provided in embodiments of the present application;

[0046] Figure 6 is a schematic diagram of a scenario of remote access provided in embodiments of the present application;

[0047] Figure 7 is a structural block diagram of a data remote writing device provided in embodiments of the present application. DETAILED DESCRIPTION

[0048] In order to make the above objectives, features and advantages of the present application more apparent, further detailed description of the present application will be given below with reference to the accompanying drawings and specific embodiments.

[0049] As an example, in the related art, RDMA send (Remote Direct Memory Access) takes a receive buffer from a receive queue to store a message, which has the disadvantage that the receive queue must be filled with a buffer capable of receiving the largest message. However, the message size of most applications is normally distributed, and the proportion of the largest size message is not high, so the memory utilization is not high and the memory is wasted; in addition, the RDMA RC type of QP does not support out-of-order transmission; and the RDMA send can only write a message to one receive queue, which leads to the growth of the receive memory according to "the largest message x the queue depth x the number of QPs", and the number of QPs is very large (10^3) when the cluster size is large, and at the same time, it also leads to the mixing of messages of each rx thread in one queue, which needs to be distributed again.

[0050] To this end, in the present application, by establishing a queue pair between the sending node and the receiving node, and registering a corresponding shared memory area for each receiving thread in the receiving node, each shared memory area corresponds to a remote key, then when the sending node remotely sends corresponding data to the receiving node to write corresponding data on the receiving node, the sending node generates a target message, splits the target message into at least one data segment, generates a data packet corresponding to each data segment, the data packet at least carries a target remote key for specifying a receiving thread, a message identifier for identifying the target message, a message total length of the target message, and a segment offset of the data segment in the target message, then the receiving node receives the data packet through the queue pair, locates a target receiving thread corresponding to the target remote key from the receiving thread, then accesses a target shared memory area corresponding to the target receiving thread according to the target remote key, and based on the message identifier and the message total length, reserves memory space for the target message in the target shared memory area or locates the reserved memory space, and finally based on the segment offset, writes the payload in the data packet directly into the memory space located in the target shared memory area, so that in the process of remote writing, through the target remote key in the data packet, the sending node can directly specify the target thread of the receiving node, and the data is directly written into the memory space dedicated to the thread after arriving, supporting thread-level accurate delivery, and through the message total length, dynamically reserving space for the target message in the shared memory area, effectively improving the memory utilization, and based on the message identifier and the segment offset carried in the data packet, in the process of writing, each data segment can be independently identified and located, supporting out-of-order receiving, and improving the flexibility of data transmission.

[0051] In order for those skilled in the art to better understand the technical solutions in the embodiments of the present application, some technical features involved in the embodiments of the present application are explained and described as follows:

[0052] QP (Queue Pair, queue pair): a pair of receive and send queues, including Receive Queue (RQ, receiving queue) and Send Queue (SQ, sending queue), used for software to submit a receiving buffer and a sending request to hardware. Among them, the number of QP is QPN.

[0053] QPC (QP Context, Queue Pair Context): QP context, records the producer pointer (CI, Consumer Index) and consumer pointer (PI, Producer Index) of RQ and SQ. Among them, the producer pointer (CI): usually indicates the position of the data in the queue that has been processed (consumed). In the receiving queue, it marks the position that the software (such as the operating system or application) has processed; in the sending queue, it marks the position of the data that the hardware has completed sending; the consumer pointer (PI): generally indicates the next data position that can be processed (consumed) in the queue. In the receiving queue, it indicates the position where the hardware can place the newly received data; in the sending queue, it indicates the position where the software can add new sending requests to the queue.

[0054] MR (Memory Region, Memory Region): The memory region registered by the RDMA application to the network card. The application tells the network card the virtual address VA and length length and access permission of the memory region, and the network card returns Lkey and Rkey for local and remote access to this memory, respectively.

[0055] Rkey (Remote Key, Remote Key): The security key returned when registering MR. When the remote node uses the RDMAread / write semantics to read and write this memory, it needs to verify this key.

[0056] MPT (Memory Protection Table, Memory Protection Table): Memory protection table, indexed by Lkey (local key) or Rkey (remote key), records the access permission of MR, the base address of MTT table, and the IOVA (Input / Output Virtual Address, Input / Output Virtual Address) base address. Each entry in the memory protection table can be named Mkey entry or MPT entry.

[0057] MTT (Memory Translation Table, Memory Translation Table): Memory translation table, used to translate IOVA address to PA (Physical Address) physical address. Each entry of it is an 8-byte physical page frame number PFN (Physical Frame Number).

[0058] PSN (Packet Sequence Number, Packet Sequence Number): The packet sequence number carried in the RDMA BTH (Base Transport Header, Base Transport Header) header.

[0059] DDP (Data Direct Place): refers to not buffering the message and directly writing the payload into the memory.

[0060] WQE (Work Queue Element): a sending request in a QP, filled by software and consumed by hardware.

[0061] In addition, the English abbreviations and Chinese names involved in the embodiments of the present application are introduced as follows in Table 1:

[0062]

[0063] Table 1

[0064] Referring to Figure 1 , a step flow chart of a data remote writing method provided in the embodiments of the present application is shown, involving a sending node and a receiving node, a queue pair is established between the sending node and the receiving node, and a corresponding shared memory area is registered for each receiving thread in the receiving node, and each shared memory area corresponds to a remote key; wherein, specifically can include the following steps:

[0065] Step 101, the sending node generates a target message, splits the target message into at least one data segment, generates a data packet corresponding to each data segment, and the data packet carries at least a target remote key for specifying a receiving thread, a message identifier for identifying the target message, a message total length of the target message, and a segment offset corresponding to the data segment in the target message;

[0066] In the embodiments of the present application, before communication between the sending node and the receiving node, the control plane can be initialized first, one or more queue pairs are established between the sending node and the receiving node, each queue pair can be shared by multiple threads on the node, and a corresponding shared memory area is registered for each thread on the node, and each shared memory area can correspond to a remote key, which can be a security credential for accessing the corresponding shared memory area, so as to support thread-level accurate delivery in the process of data writing.

[0067] For example, Node1 contains threads P1, P2 and P3, and registers a corresponding shared memory area, i.e. RX buffer, for each thread. Node2 is configured in the same way. The two nodes on the control plane can communicate necessary control information through a control plane protocol (such as a custom negotiation message). For example, Node1 sends the queue pair (QP) number (QPN), initial packet sequence number (PSN), and Rkey corresponding to each receiving thread MR to Node2 through the QP, so as to complete initialization on the control plane, and then perform direct remote memory write.

[0068] In the embodiment of the application, when an application in the sending node needs to send data to a specific thread (such as P1 or P2) in the receiving node, it can submit a corresponding sending request (such as Post Send). In the request, in addition to the target message to be sent, the target receiving thread in the receiving node and the target remote key (such as RKey1 or RKey2) corresponding to the target receiving thread can also be specified. In the process of direct remote memory write, the target thread of the receiving node can be specified through the target remote key, and the data is directly written into the memory space dedicated to the thread after reaching the thread, supporting thread-level accurate delivery.

[0069] In addition, the data sent by the sending node to the receiving node can also include a message identifier for identifying the target message, a total length of the target message, and a segment offset of a data segment corresponding to the target message. The target message can be uniquely identified in the queue pair through the message identifier, and the data segment can be a plurality of data obtained by splitting the target message. All data segments in the same target message have the same message identifier, and the segment offset refers to the starting byte offset of the current data segment in the complete target message.

[0070] In a specific implementation, for a target message sent to the receiving node, the network card hardware of the sending node obtains the corresponding request, splits the target message corresponding to the request into a plurality of data segments according to the maximum transmission unit of the network, and generates an independent data packet for each data segment. Each data packet can carry corresponding key metadata, i.e. target remote key, message identifier, total length of the message, segment offset, source IP address, and queue pair number, in the payload or extension header in addition to the standard RoCEv2 protocol header (such as Ethernet, IP, UDP, BTH), so as to achieve accurate delivery in the process of direct remote memory write.

[0071] Step 102, the receiving node receives the data packet through the queue pair, and locates a target receiving thread corresponding to the target remote key from the receiving thread.

[0072] For the receiving node, after receiving the corresponding data packet through the queue pair, the data packet can be parsed to obtain the target remote key contained in the data packet, and then the corresponding target receiving thread is located from the receiving thread based on the target remote key, so as to receive the corresponding data based on the target receiving thread and locate the target shared memory area corresponding to the target receiving thread.

[0073] In a specific implementation, after the network card hardware of the receiving node receives the data packet through the shared queue pair, the target remote key (Rkey) carried in the message header is first parsed. Then, the network card matches the Rkey with each MR registered locally according to the pre-configured control plane information, thereby locating the target receiving thread expected by the sending party and the target shared memory area (MR) corresponding to the target receiving thread, so that the initial distribution of the message is completed at the hardware level, and secondary scheduling at the software level is avoided.

[0074] In step 103, the receiving node accesses the target shared memory area corresponding to the target receiving thread according to the target remote key, and reserves memory space for the target message or locates the already reserved memory space in the target shared memory area based on the message identifier and the total length of the message.

[0075] After the target receiving thread is determined, since the control plane has allocated a corresponding shared memory area for each thread, the receiving node can access the target shared memory area corresponding to the target receiving thread based on the received target remote key, and then reserve corresponding memory space for the target message or locate the already reserved memory space in the target shared memory area based on the message identifier and the total length of the message, so that the memory space is dynamically reserved for the corresponding message in the shared memory area, avoiding the memory waste caused by pre-allocating the buffer according to the maximum message size, and improving the memory utilization.

[0076] In some possible implementation manners, a message reservation table is maintained in the receiving node, and the message reservation table includes a mapping relationship between a key value and a memory area offset. The data packet further includes at least a source IP address and a queue pair number. The message reservation table can be a dynamic and entry-based internal database, and each record included in the message reservation table corresponds to a complete message (such as a target message) in a transmission process. Through the message reservation table, the network card of the receiving node can unambiguously associate any arriving data segment to the parent message to which the data segment belongs, and determine the storage position of the data segment in the final memory, thereby supporting the out-of-order transmission in the remote data writing process.

[0077] The key value in the message reservation table refers to a unique index corresponding to each record, which is used to uniquely identify an ongoing data transmission transaction in a global scope. The key value can be a composite structure, for example, it can be composed of a source IP address, a queue pair number, a message identifier, and a remote key. Based on this composite structure, any data segment can be quickly and accurately identified in complex network topology and concurrent communication scenarios.

[0078] The memory region offset in the message reservation table represents the starting data byte of the data segment in the target message. The determination of the offset occurs when the first data segment of the message arrives. The network card hardware in the receiving node queries the table. If no record corresponding to the key value is found, a memory allocation routine is performed: a continuous logical address space with a length equal to the "total message length" carried by the packet is allocated from the current free offset of the target shared memory region. At this time, the current "free offset" is recorded as the "memory region offset" of the message, and then the "free offset" pointer is advanced by the corresponding length. Thereafter, regardless of any data segment (including the first segment) of the message arriving, the network card can query the "memory region offset" through the key value, and then perform arithmetic addition with the "segment offset" carried in the data segment packet to calculate the exact destination address of the data segment payload in the shared memory region.

[0079] Based on the content recorded in the message reservation table and the information carried in the data packet, the receiving node can first use the source IP address, queue pair number, message identifier, and target remote key in the data packet to construct the corresponding query key value. If the query key value is found in the message reservation table, the memory region offset is obtained from the corresponding first mapping relationship, and the reserved memory space is located based on the memory region offset. If the query key value is not found in the message reservation table, the current free offset of the target shared memory region is obtained, and a second mapping relationship between the query key value and the current free offset is established in the message reservation table. A memory space with a length of the total message length starting from the current free offset is reserved for the target message, and the current free offset of the target shared memory region is increased by the total message length. Thus, by dynamically reserving memory space in the shared memory region for the corresponding message, memory waste caused by pre-allocating buffers according to the maximum message size is avoided, and memory utilization is improved.

[0080] In some examples, the process of locating or reserving memory space can be as follows:

[0081] 1. The receiving node constructs a corresponding query key value: uses the source IP address, queue pair number (QPN), message identifier (MsgID), and target remote key (Rkey) in the data packet to combine and construct a globally unique query key value.

[0082] 2. Query the message reservation table using the query key value.

[0083] 3. During the query process, different branch processes can be executed based on the query result:

[0084] Case 1: Locate the reserved space (query hit)

[0085] If the key value is found in the table, the memory region offset (MR_Offset) reserved for the message is directly obtained from the corresponding (first) mapping relationship. This offset indicates the starting position of the entire message in the target shared memory region. Thus, the reserved memory space is successfully located.

[0086] Case 2: Reserve new space (query miss)

[0087] If the key value is not found in the table, it indicates that the target message is first appearing and needs to be reserved. At this time:

[0088] a. Obtain the current free offset of the target shared memory region (this value is usually recorded in the memory protection table entry corresponding to Rkey or other hardware registers).

[0089] b. Establish a new (second) mapping relationship in the message reservation table, associating the query key value with the current free offset.

[0090] c. Increase the current free offset of the target shared memory region by the total length of the message. This operation is equivalent to allocating a continuous space with a length equal to the total length of the message from the shared memory pool to the current message and updating the water line of the pool. At this time, the memory space starting from the original free offset with a length equal to the total length of the message is reserved for the target message.

[0091] d. Subsequently, the process can naturally return to the "query hit" state, thereby obtaining the memory region offset reserved for the new message.

[0092] In addition, the mapping relationship in the message reservation table also includes the remaining to-be-received length corresponding to the key value. Therefore, after establishing the new mapping relationship, the receiving node can set the remaining to-be-received length corresponding to the second mapping relationship to the total length of the message, so as to track the receiving progress of the target message.

[0093] Step 104: The receiving node directly writes the payload in the data packet into the memory space located in the target shared memory region based on the segment offset.

[0094] In the embodiments of the present application, after the corresponding target shared memory space is located, and the position for writing the data segment in the target shared memory space is located, the payload in the data packet can be written into the memory space where the target shared memory region is located based on the segment offset, so that in the process of remote writing, the target thread of the receiving node can be directly specified by the target remote key in the data packet, and the data is directly written into the memory space dedicated to the thread after reaching the target thread, supporting thread-level accurate delivery. In addition, the total length of the message is used to dynamically reserve space for the target message in the shared memory region, effectively improving the memory utilization, and based on the message identifier and the segment offset carried by the data packet, each data segment can be independently identified and located in the writing process, supporting out-of-order reception and improving the flexibility of data transmission.

[0095] In some possible implementation manners, the receiving node also maintains a corresponding memory protection table and a memory translation table, which together constitute a security and address translation infrastructure for the network card to access the host memory. The memory protection table is a static table structure pre-configured in the network card hardware by the driver on the control plane, and the memory translation table is actively queried and used by the network card hardware in the data plane writing process, and is used for safely completing the conversion from a virtual address to a physical address and authorizing DMA operations.

[0096] The memory protection table is used for permission verification and translation navigation. For access permission verification, the access permission attribute (such as whether it is readable, writable, or remotely accessible) of the corresponding memory region (MR) is stored in the MPT entry. When the network card receives an RDMA write request carrying a target remote key (Rkey), the Rkey is first used as an index to search the MPT. The primary purpose of this is to verify whether the request has the legal permission to write to the target memory region. If the permissions do not match (for example, the memory space is configured to be read-only, and the request is to write), the network card will discard the packet and may report an error. This is the first barrier to protect the security of the receiving end memory.

[0097] For the address translation navigation process, after the permission verification passes, the network card extracts the key information necessary for address translation from the hit MPT entry, of which the most important is the memory translation table base address. This base address points to the starting position of the memory translation table corresponding to the memory space in the hardware. In addition, the MPT entry usually contains the IOVA base address of the memory space, which is used for subsequent page table index calculation.

[0098] For memory translation table, which is used to implement the final conversion of virtual address to physical address, the memory translation table is essentially a page table, each entry of which records a virtual memory page corresponding to a physical page frame number (PFN). The memory translation table of a memory space contains the physical address mapping of all memory pages of the memory space. These mapping relationships are filled into the memory translation table by the driver when the memory space is registered.

[0099] Optionally, for the process of address translation, it can include:

[0100] After obtaining the MTT base address and the target memory offset, the network card performs address translation:

[0101] a. Decompose the target memory offset into a page index and an intra-page offset.

[0102] b. Calculate the target entry address in the MTT using the MTT base address + page index * entry size.

[0103] c. Read the physical page frame number (PFN) from the MTT entry.

[0104] d. Combine the PFN with the intra-page offset to obtain the final target physical address (PA).

[0105] Based on the memory protection table and the memory translation table, in the process of data writing, the receiving node can add the memory region offset to the segment offset in the data packet to obtain the target memory offset of the payload in the target shared memory region, then use the target remote key and the target memory offset to perform address translation through the memory protection table and the memory translation table to obtain the target physical address for writing the payload, and finally write the payload directly to the target physical address.

[0106] For example, in the process of data writing, the collaborative process between the memory protection table and the memory translation table is as follows:

[0107] 1. Index MPT: The network card uses the target remote key (Rkey) in the packet to index the memory protection table (MPT), performs permission check, and obtains the memory translation table (MTT) base address and IOVA base address.

[0108] 2. Calculate MTT index: The network card takes the target memory offset (obtained by adding the memory region offset and the segment offset) as the offset in the MR, performs division operation by the page size, and obtains the index in the MTT.

[0109] 3. Query MTT: The network card queries the memory translation table (MTT) by combining the MTT base address and the calculated index, and obtains the corresponding physical page frame number (PFN).

[0110] 4. Synthesize physical address: the NIC combines the PFN with the low-order intra-page offset of the target memory offset to generate the final target physical address (PA).

[0111] 5. Perform DMA write: the NIC DMA engine writes the payload in the packet directly to this target physical address.

[0112] By combining the logical memory offset (memory region offset and segment offset) with the hardware address translation mechanism (memory protection table and memory translation table), the receiving NIC can directly and accurately write the payload of any out-of-order arriving data segment to its final host physical memory address without relying on the CPU, which not only guarantees the legality and security of memory access, but also completely eliminates the overhead of kernel involvement and data copying, and is the key to achieving low-latency and high-throughput data transmission.

[0113] In addition, in some possible implementations, after the payload is directly written to the target physical address, the receiving node can also subtract the length of the payload that has been written from the remaining to-be-received length corresponding to the query key. In specific implementation, after the receiving NIC successfully writes the payload of a certain data segment to the target physical memory through DMA, it immediately uses the query key corresponding to the data segment to search the message reservation table, and subtracts the length of the just-written payload from the remaining to-be-received length field in the corresponding record of the message reservation table, thereby achieving accurate and fine-grained message reception progress tracking.

[0114] For example, assuming that a target message with a total length of 10,000 bytes is split into 10 data segments. When the first data segment (carrying MsgID=100, with a payload length of 1,000 bytes) arrives and is written, the NIC finds the corresponding record in the message reservation table and updates the remaining to-be-received length from 10,000 to 9,000. Subsequently, the third data segment (also belonging to MsgID=100, with a length of 1,500 bytes) arrives out of order and is written, and the NIC updates the field again, reducing it from 9,000 to 7,500. This process continues regardless of the order of the data segments.

[0115] In some possible implementations, the receiving node can also generate a completion notification event on the associated completion queue in response to the remaining to-be-received length corresponding to the query key being reduced to zero, thereby reliably announcing to the upper-layer software that a complete message is ready, so as to asynchronously notify the corresponding application of the message reception completion event.

[0116] For example, when the last segment of the message with Msg ID = 100 is written, and its payload length (supposedly 500 bytes) is subtracted from the remaining length to be received (supposedly 500 at this time), the value becomes 0. The net card hardware immediately detects this state, and then generates a CQE in the corresponding CQ. The CQE usually contains the context information (such as the corresponding QP number) of the message inside, thereby notifying the application program that is polling the CQ: "the message with Msg ID = 100 has been completely received, and can be processed".

[0117] In some feasible implementations, after the completion notification event is generated, the mapping relationship in the message reservation table corresponding to the query key is deleted, thereby releasing the hardware table item resource, and preventing resource exhaustion caused by infinite accumulation of state information. For example, after the CQE for the message with Msg ID = 100 is generated, the net card immediately acts, uses the constructed key value <SrcIP, QPN, 100, RKey> to query the message reservation table, finds the record for the message, and then performs a deletion operation. The table item occupied by the record is immediately released, and can be immediately reused by the next newly-arrived message (for example, Msg ID = 101) to establish a new mapping relationship, and start a new round of life cycle management.

[0118] It should be noted that the embodiments of the present application include but are not limited to the above examples, and it can be understood that a person skilled in the art can also set according to actual needs under the guidance of the idea of the embodiments of the present application, and the present application does not limit this.

[0119] In the embodiments of the present application, a sending node and a receiving node are involved, a queue pair is established between the sending node and the receiving node, and a corresponding shared memory region is registered for each receiving thread in the receiving node, each shared memory region corresponding to a remote key, then when the sending node remotely sends corresponding data to the receiving node to write corresponding data on the receiving node, the sending node generates a target message, splits the target message into at least one data segment, generates a data packet corresponding to each data segment, the data packet carrying at least a target remote key for specifying a receiving thread, a message identifier for identifying the target message, a message total length of the target message, and a segment offset of the data segment in the target message, then the receiving node receives the data packet through the queue pair, locates a target receiving thread corresponding to the target remote key from the receiving thread, then accesses a target shared memory region corresponding to the target receiving thread according to the target remote key, and based on the message identifier and the message total length, reserves memory space for the target message in the target shared memory region or locates the reserved memory space, and finally based on the segment offset, writes the payload in the data packet directly into the memory space located in the target shared memory region, so that in the process of remote writing, the sending node can directly specify the target thread of the receiving node through the target remote key in the data packet, and the data is directly written into the memory space dedicated to the thread after arriving, supporting thread-level accurate delivery, and through the message total length, the space for the target message is dynamically reserved in the shared memory region, effectively improving the memory utilization, and based on the message identifier and the segment offset carried in the data packet, each data segment can be independently identified and located in the writing process, supporting out-of-order receiving, and improving the flexibility of data transmission.

[0120] In order for those skilled in the art to better understand the technical solutions in the embodiments of the present application, the following will be exemplarily illustrated by corresponding examples:

[0121] As an example, in the process of converting remote direct access sending to remote direct access writing using shared memory space, the corresponding data processing flow can include:

[0122] Step 1: Control plane negotiation RX buffer

[0123] Referring to Figure 2, which is a structural schematic view of a control plane provided in the embodiment of the present application, each thread P1-P3 on the communication nodes Node1 and Node2 registers a receiving buffer MR of itself, and obtains remote keys RKey1-RKey3 of the respective MRs. Wherein, Node1 and Node2 negotiate to establish a multi-thread shared queue pair QP for communication between the two nodes. The two nodes on the control plane mutually advertise (QPN, PSN, RKey1, RKey2, RKey3, etc.). If P1 in Node1 wants to send a message to P2 in Node2, the RKey2 of the opposite end P2 is specified when sending PostSend.

[0124] Step 2: Definition of message format

[0125] Referring to Figure 3 , a schematic view of a message format provided in the embodiment of the present application is shown. The message (corresponding to one Post Send of the application) generated by each RDM Asend will be split into multiple segments (each corresponding to a packet) according to MTU, and each segment will carry a regular RoCE header (ETH, IP, UDP, BTH). In addition, Rkey is carried for specifying which thread's receiving buffer of the opposite end the message is written to. In order to support multiple segments of the message to arrive at the receiving end in disorder, each message also carries <msg_id, msg_len, seg_offset>. Wherein, msg_id is used to identify which message the segment belongs to, and it is incremented within QP, that is, the message sent by one QP will not have a repeated msg_ID; msg_len refers to the length of the entire message rather than the length of the segment, and is used to reserve a receiving buffer for the entire message; seg_offset indicates the offset of the segment in the message.

[0126] When any segment of the message first arrives at the receiving side, a buffer is reserved for the entire message according to msg_len, and msg_id is used as the key to record this reservation. When any segment arrives, it can be placed at the specified offset of the reserved buffer according to seg_offset.

[0127] Step 3: Modification of MPTentry

[0128] Referring to Figure 4, shows a schematic diagram of the memory protection table provided in the embodiment of the application, when using MR to receive RDMA send message, the structure of the memory protection entry can be optimized, a pair of producer pointer and consumer pointer is added in the queue. The extended MPT entry adds used (offset of the message which has been received by hardware but not consumed by software, i.e. consumer pointer), free (offset of the idle memory, i.e. producer pointer). When MR receives the message, it reserves memory for the message from free and updates free. After the software consumes the message, MR_PostRecv updates used on the QP.

[0129] Step 4: message reserve record

[0130] When the segment is received, it is needed to know where it is finally stored, i.e. the physical address of the receiving buffer, and if it is wanted to use MTT to translate the physical address, it needs two inputs: MTT base (Memory Translation Table Base) and MR_offset (i.e. memory region offset), the former is already contained in MPT entry (i.e. memory protection table entry), and the latter is obtained by using the state table message reserve table. When the segment arrives, the key of the message can be used to index the message reserve table, i.e. <SrcIP (i.e. source IP address), QPN (i.e. queue pair number), msg_id (i.e. message identifier), RKey (i.e. remote key)>. If it hits, the MR_Offset can be taken out, and the length of the current segment is subtracted from remain_to_recv (i.e. remaining to receive length) (when it becomes 0, the record is deleted). If it does not hit, a new record is added: key=<SrcIP, QPN, msg_id, RKey> value=MPT_entry.free remain_to_recv=msg_len, and MPT_entry.free is added by msg_len. Then, the hit process is run again.

[0131] Step 5: MTT address translation

[0132] Referring to Figure 5, shows a schematic diagram of address translation provided in the embodiment of the application, after Segment is received, Rkey is used to look up MPT table, and <SrcIP, QPN, msg_id, RKey> is used as key to look up message reservation table, if the message reservation table hits, MR_Offset can be taken out from it, if it does not hit, record: key = <SrcIP, QPN, msg_id, RKey> value = MPT_entry.free is set, remain_to_recv = msg_len is set, and MPT_entry.free is added by msg_len. Then the hit flow is run again. MR_Offset represents the storage offset position of message in MR, it is added by seg_offset to obtain the index of MTT. MTT base (obtained by looking up MPT) and MTT index (obtained by looking up message reservation table) are used to look up MTT to obtain PFN, it is left shifted by 12b and added by intra-page offset (MR_Offset low 12b) to obtain the final PA.

[0133] Step 6: completion notification

[0134] When remain_to_recv of message reserve record becomes 0, it means that all segments of the message have been received, and completion notification can be generated on the CQ associated with the QP, the completion notification should carry Rkey and MR_Offset to indicate where the message is placed in which MR. After the completion notification, the current message reserve record can be deleted, and the space can be reused when the next message is received.

[0135] In addition, referring to Figure 6 , a scenario schematic diagram of remote access provided in the embodiment of the application is shown, and the specific process can include:

[0136] The software and hardware framework of RDMA ROCE network card: the software is divided into kernel mode driver and user mode driver, the command of control plane (such as register MR) is first sent from user mode to kernel mode, and then the kernel mode driver programs and configures the hardware table. The command of data plane (such as post send) is directly sent from user mode to hardware through QP, without passing through kernel mode. The hardware accepts the configuration of the kernel driver, actively reads the WQE request sent by software in the QP, analyzes the WQE and translates the VA in it to execute DMA to generate the final packet and send it out.

[0137] RDMA send to RDMA write control plane flow: each communication thread in user space registers its own receive buffer MR, and gets the Lkey and Rkey of the MR. Before communicating with the remote side, the control plane information (QPN, PSN, Rkey of each thread) needs to be exchanged. When sending a message to a specific thread Px on the opposite side, the Rkey of the opposite Px thread needs to be specified when posting the send. The kernel mode driver registers the MR to generate the MPT table and MTT table according to the conventional process, but adds two new fields, used and free, in the MPT entry, whose initial values point to the beginning of the MR.

[0138] RDMA send to RDMA write data plane flow: on the sending side, the user mode driver puts a send WQE into the QP after the upper layer application calls ibv_post_send, which contains the Rkey of the opposite receive buffer MR, which actually indicates which receive thread on the opposite side the message is intended to be sent to. The ROCE network card hardware engine reads the WQE in the QP and parses it, which will cut a large message into many segments according to the network MTU, and each segment corresponds to a network transmission packet. In addition to the conventional RoCE packet header such as ETH / IP / UDP / BTH, the engine will also add <message id, message length, segment offset> to support segment out-of-order transmission. The message id is used to identify which message the segment belongs to, which is incremented in the QP, i.e. a message sent by a QP will not have duplicate message ids. The message length refers to the length of the entire message, not the length of the segment, which is used to reserve the receive buffer for the entire message. The segment offset indicates the offset of the segment in the message.

[0139] On the receiving side, when any segment of the message arrives for the first time, the Rkey in the packet is used to look up the MPT table, and in parallel, the <source IP, QPN, message id, Rkey> key is used to look up the message reserve table. If the message reserve table hits, the MR_Offset can be taken out to know the offset of the message in the receive buffer MR. If it does not hit, a new record is added: key = <source IP, QPN, message id, Rkey>, set value = MPT_entry. free, i.e. the new message is placed at the MR offset pointed by free, and set remain_to_recv = message length, then add the found MPT_entry. free with message length, i.e. skip the reserved area of the new message. Then go through the hit flow again to get MR_Offset. MR_Offset represents the message storage offset in the MR, it is added with segment offset to get the segment storage offset in the MR, which is the index of MTT. Use MTT base (found by searching MPT) and MTT index (found by searching message reserve table) to search MTT to get PFN, left shift 12b of PFN and add the intra-page offset (MR_Offset low 12b) to get the physical memory address PA where the segment should be written.

[0140] After the network card DMA engine writes the payload of the segment into the memory pointed by PA, it will subtract the length of the segment from the remain_to_recv of the message reserve record. When the remain_to_recv of the message reserve record becomes 0, it means that all segments of the message have been received, and a completion notification should be generated on the CQ associated with the QP, which should carry Rkey and MR_Offset to indicate where the message is placed. After the completion notification, the current message reserve record can be deleted, and its space can be reused when receiving the next message. After receiving the notification, the software consumes the message, and then executes MR_PostRecv on the QP to update the used field of the MPT entry, which is equivalent to putting the free memory back into the receiving MR.

[0141] It should be noted that, for the method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited to the action sequence described, because according to the embodiments of the present application, some steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of the present application.

[0142] Referring to Figure 7, shows a structural block diagram of a data remote writing device provided in an embodiment of the present application, relates to a sending node and a receiving node, a queue pair is established between the sending node and the receiving node, and a corresponding shared memory area is registered for each receiving thread in the receiving node, and each shared memory area corresponds to a remote key; wherein, the following modules can be specifically included:

[0143] A message processing module 701 located in the sending node, used for generating a target message, splitting the target message into at least one data segment, generating a data packet corresponding to each data segment, and the data packet at least carrying a target remote key used for specifying a receiving thread, a message identifier used for identifying the target message, a message total length of the target message, and a segment offset corresponding to the data segment in the target message;

[0144] A thread positioning module 702 located in the receiving node, used for positioning a target receiving thread corresponding to the target remote key from the receiving thread by receiving the data packet through the queue pair;

[0145] A memory space positioning module 703 located in the receiving node, used for accessing a target shared memory area corresponding to the target receiving thread according to the target remote key, and reserving a memory space for the target message or positioning a reserved memory space in the target shared memory area based on the message identifier and the message total length;

[0146] A data writing module 704 located in the receiving node, used for directly writing a payload in the data packet into the memory space positioned by the target shared memory area based on the segment offset.

[0147] In some possible implementation manners, a message reservation table is maintained in the receiving node, the message reservation table includes a mapping relationship between a key value and a memory area offset, and the data packet at least further includes a source IP address and a queue pair number, and the memory space positioning module is specifically used for:

[0148] constructing a corresponding query key value by using the source IP address, the queue pair number, the message identifier and the target remote key in the data packet;

[0149] if the query key value is queried in the message reservation table, obtaining a memory area offset from a corresponding first mapping relationship, and positioning a reserved memory space based on the memory area offset;

[0150] If the query key value is not found in the message reservation table, a current free offset of the target shared memory region is obtained, a second mapping relationship between the query key value and the current free offset is established in the message reservation table, a memory space with the current free offset as a start and a length of the message total length is reserved for the target message, and the current free offset of the target shared memory region is increased by the message total length.

[0151] In some possible implementation manners, the mapping relationship in the message reservation table further includes a remaining to-be-received length corresponding to the key value, and the apparatus further includes:

[0152] a message length setting module located at the receiving node and configured to set the remaining to-be-received length corresponding to the second mapping relationship as the message total length.

[0153] In some possible implementation manners, the receiving node further maintains a corresponding memory protection table and a memory translation table, and the data writing module is specifically configured to:

[0154] add the memory region offset and a segment offset in the data packet to obtain a target memory offset of a payload in the data packet in the target shared memory region;

[0155] perform address translation on the target remote key and the target memory offset through the memory protection table and the memory translation table to obtain a target physical address for writing the payload;

[0156] write the payload directly into the target physical address.

[0157] In some possible implementation manners, the apparatus further includes:

[0158] a length updating module located at the receiving node and configured to subtract a length corresponding to the payload that has been written from a remaining to-be-received length corresponding to the query key value.

[0159] In some possible implementation manners, the apparatus further includes:

[0160] an event generating module located at the receiving node and configured to generate a completion notification event on an associated completion queue in the queue in response to the remaining to-be-received length corresponding to the query key value decreasing to zero.

[0161] In some possible implementation manners, the apparatus further includes:

[0162] a mapping relationship updating module located at the receiving node and configured to delete the mapping relationship corresponding to the query key value in the message reservation table after the completion notification event is generated.

[0163] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts refer to the part of the method embodiment.

[0164] In addition, the embodiment of the present application also provides an electronic device, comprising: a processor, a memory, a computer program stored on the memory and executable on the processor, which implements each process of the above-mentioned data remote writing method embodiment when executed by the processor, and can achieve the same technical effect, to avoid repetition, which will not be repeated here.

[0165] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement each process of the above-mentioned data remote writing method embodiment, and can achieve the same technical effect, to avoid repetition, which will not be repeated here. The computer readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0166] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts of each embodiment can be referred to each other.

[0167] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer usable program code.

[0168] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams according to the method, terminal device (system), and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a machine that implements the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the block or blocks.

[0169] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the block or blocks.

[0170] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the block or blocks.

[0171] Although preferred embodiments of the application have been described, those skilled in the art will recognize that additional modifications and changes can be made thereto without departing from the scope of the present application. Accordingly, the appended claims are intended to cover all such modifications and changes as fall within the scope of the application.

[0172] Finally, it should be noted that the terms "comprises", "comprising", or other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "comprises", "comprising", or other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process or method.

[0173] The above describes the remote data writing method and the remote data writing device in detail, and the principles and implementation manners of the present application are described by using specific examples. The above description of the examples is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the present application should not be understood as a limitation of the present application.

Claims

1. A method for remotely writing data, characterized in that, The method involves sending and receiving nodes, establishing queue pairs between the sending and receiving nodes, and registering corresponding shared memory regions for each receiving thread in the receiving node, with each shared memory region corresponding to a remote key; wherein, the method includes: The sending node generates a target message, splits the target message into at least one data segment, and generates a data packet corresponding to each data segment. The data packet carries at least a target remote key for specifying the receiving thread, a message identifier for identifying the target message, the total message length of the target message, and the segment offset of the data segment in the target message. The receiving node receives the data packet through the queue and locates the target receiving thread corresponding to the target remote key from the receiving threads; The receiving node accesses the target shared memory region corresponding to the target receiving thread according to the target remote key, and reserves memory space for the target message or locates the reserved memory space in the target shared memory region based on the message identifier and the total message length; Based on the segment offset, the receiving node directly writes the payload in the data packet into the memory space located in the target shared memory region.

2. The method according to claim 1, characterized in that, The receiving node maintains a message reservation table, which includes a mapping relationship between key values ​​and memory region offsets. The data packet also includes at least a source IP address and a queue pair number. The step of reserving memory space for the target message or locating already reserved memory space in the target shared memory region based on the message identifier and the total message length includes: The corresponding query key is constructed using the source IP address, the queue pair number, the message identifier, and the target remote key in the data packet. If the query key value is found in the message reservation table, the memory region offset is obtained from the corresponding first mapping relationship, and the reserved memory space is located based on the memory region offset; If the query key value is not found in the message reservation table, the current free offset of the target shared memory region is obtained, and a second mapping relationship between the query key value and the current free offset is established in the message reservation table. Memory space starting from the current free offset and with a length equal to the total length of the message is reserved for the target message, and the current free offset of the target shared memory region is increased by the total length of the message.

3. The method according to claim 2, characterized in that, The mapping relationship in the message reservation table also includes the remaining length to be received corresponding to the key value. After establishing the second mapping relationship between the query key value and the current idle offset in the message reservation table, the method further includes: Set the remaining length to be received corresponding to the second mapping relationship as the total message length.

4. The method according to claim 3, characterized in that, The receiving node also maintains a corresponding memory protection table and a memory translation table. The step of directly writing the payload in the data packet into the memory space located in the target shared memory region based on the segment offset includes: Add the memory region offset to the segment offset in the data packet to obtain the target memory offset of the payload in the data packet within the target shared memory region; Using the target remote key and the target memory offset, address translation is performed through the memory protection table and the memory translation table to obtain the target physical address for writing the payload; The payload is written directly to the target physical address.

5. The method according to claim 4, characterized in that, After directly writing the payload to the target physical address, the method further includes: The receiving node subtracts the length corresponding to the written payload from the remaining length to be received corresponding to the query key value.

6. The method according to claim 5, characterized in that, Also includes: The receiving node generates a completion notification event on the completion queue associated with the queue pair in response to the remaining length to be received corresponding to the query key value decreasing to zero.

7. The method according to claim 6, characterized in that, Also includes: After generating the completion notification event, the receiving node deletes the mapping relationship corresponding to the query key value in the message reservation table.

8. A remote data writing device, characterized in that, The device involves sending and receiving nodes, establishing queue pairs between the sending and receiving nodes, and registering corresponding shared memory regions for each receiving thread in the receiving node, with each shared memory region corresponding to a remote key; wherein, the device includes: The message processing module located at the sending node is used to generate a target message, split the target message into at least one data segment, generate a data packet corresponding to each data segment, and the data packet carries at least a target remote key for specifying the receiving thread, a message identifier for identifying the target message, the total message length of the target message, and the segment offset of the data segment in the target message. The thread location module located at the receiving node is used to receive the data packet through the queue and locate the target receiving thread corresponding to the target remote key from the receiving threads; The memory space positioning module located at the receiving node is used to access the target shared memory region corresponding to the target receiving thread according to the target remote key, and reserve memory space for the target message or locate the reserved memory space in the target shared memory region based on the message identifier and the total length of the message; The data writing module located at the receiving node is used to directly write the payload in the data packet into the memory space located in the target shared memory region based on the segment offset.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-7.

Images