Data transmission method and system based on connectionless RDMA network, electronic equipment and storage medium

Abstract

The invention provides a data transmission method and system based on a connectionless RDMA network, electronic equipment and a storage medium, and relates to the technical field of computer network communication. The data transmission method comprises the following steps: initializing a sending queue; controlling the sending end and the receiving end to establish a virtual session and exchange a global identifier and a queue pair number; controlling a sending end to construct data messages, and storing the data messages into a sending queue in sequence; the sending end is controlled to send the data message in the sending queue to the connectionless RDMA network, so that the receiving end receives the data message, and the sending end is controlled to receive a response message returned by the receiving end; judging whether the sending queue is empty or not; and under the condition that the sending queue is empty, controlling the sending end and the receiving end to end the virtual session. According to the data transmission method and system provided by the invention, efficient and reliable data transmission under the connectionless RDMA network is realized through the technical means of message format design, session management, a sequence number mechanism and automatic retransmission.

Description

Technical Field

[0001] The present application relates to the field of computer network communication technology, and in particular to a data transmission method and system based on a connectionless RDMA network, an electronic device, and a storage medium. Background Art

[0002] Remote Direct Memory Access (RDMA) is widely used in modern data centers for applications such as file storage, data processing, and large language models (LLMs) due to its kernel bypass, low latency, and high bandwidth. Data transfer from one machine to another is quite common in these applications. For example, in a distributed caching system, clients can replicate data to multiple cache nodes on demand to improve the local cache hit rate. Data transmission should be reliable; that is, from the user's perspective, the received data should be exactly the same as the data sent. Furthermore, data transmission should be scalable to meet the actual needs of large-scale data centers.

[0003] Almost all RDMA devices provide a reliable connection (RC) transmission mode, which implements reliable connection-based data transmission through hardware. In this transmission mode, each queue pair (QP) must form a fixed connection with another queue pair (QP). However, many researchers have found that as the number of connections increases, the throughput of processing requests (also known as IOPS) drops sharply. For example, by starting multiple clients and exchanging data with each other, the transmission throughput drops by more than 50% when the number of clients increases. This is mainly because as the network scale expands, the number of queue pairs (QPs) increases in a quadratic manner, which destroys the concurrent processing capabilities of the network card and causes frequent cache swapping, leading to further performance deterioration. Therefore, it is difficult to directly use reliable connection (RC) transmission for data transmission in large clusters.

[0004] Unreliable Datagram (UD) is another RDMA transmission type. This transmission mode does not require establishing a hardware connection before data transmission. In other words, the local QP can send data packets to any remote QP without reconfiguration, which greatly reduces the connection management overhead.

[0005] However, the connectionlessness and instability of unreliable datagram transmission pose significant challenges for practical applications. When transmitting data from one machine to another, unreliable datagram transmission does not establish a connection beforehand, and queue pairs (QPs) may simultaneously receive different data fragments from different machines. This increases the complexity of ensuring the in-order delivery of data packets, which must be implemented in software. Furthermore, to achieve efficient data transmission, software-based congestion control mechanisms are required to prevent network congestion and ensure smooth data flow.

[0006] The primary drawback of connected RDMA networks based on the reliable connection (RC) transmission model is their poor scalability. As data center scale and the number of connections increase, the number of queue pairs (QPs) grows quadratically. This not only undermines the concurrent processing capabilities of the network interface card (NIC) but can also cause cache corruption, leading to a sharp drop in request processing throughput (IOPS), severely limiting its application in large clusters.

[0007] Connectionless RDMA networks based on the unreliable datagram (UD) transmission model suffer from limitations in transmission reliability and efficiency. Because this transmission model is connectionless and unstable, messages can be lost, and the receiver can simultaneously receive different data fragments from different senders, making it difficult to deliver packets in sequence.

[0008] In traditional Ethernet networks, TCP is commonly used for reliable data transmission at the application layer. To further reduce latency and improve throughput, open-source reliable transmission frameworks such as KCP have been proposed both domestically and internationally. However, due to the high-bandwidth and low-latency characteristics of RDMA, TCP's frequent acknowledgments and retransmissions consume significant network bandwidth and processing resources in high-concurrency scenarios. Furthermore, support for RDMA-specific features (such as immediate data transfer) is very limited. Summary of the Invention

[0009] In order to solve at least one of the above problems, the present application proposes a data transmission method and system, electronic device and storage medium based on a connectionless RDMA network, which realizes efficient and reliable transmission through software layer optimization, solves the scalability bottleneck of the RC mode and the reliability problem of the UD mode, and fully utilizes the low latency and high bandwidth characteristics of RDMA.

[0010] According to the first aspect of the present application, at least one embodiment of the present application provides a data transmission method based on a connectionless RDMA network, which is applied to a sending end, and the data transmission method includes: initializing a sending queue; controlling the sending end to establish a virtual session with a receiving end; controlling the sending end to exchange a global identifier and a queue pair number with the receiving end; controlling the sending end to construct a data message and store the data message in the sending queue in sequence; controlling the sending end to send the data message in the sending queue to the connectionless RDMA network according to the global identifier and the queue pair number, so that the receiving end receives the data message, and controlling the sending end to receive a response message returned by the receiving end; judging whether the sending queue is empty; and if the sending queue is empty, controlling the sending end to end the virtual session with the receiving end.

[0011] For example, in some embodiments of the present application, the initializing the sending queue includes: pre-allocating a message buffer required by the sending end to send the data message.

[0012] For example, in some embodiments of the present application, the controlling the sending end and the receiving end to establish a virtual session includes: when the sending end and the receiving end establish a virtual session for the first time, allocating a sending queue dedicated to the sending end from the initialized sending queue for data transmission of the virtual session; applying for N address ranges from the message buffer, the address ranges being used to provide registered memory space for data transmission, where N is an integer greater than or equal to 1.

[0013] For example, in some embodiments of the present application, the control of the sending end to construct a data message and store the data message in the sending queue in sequence includes: controlling the sending end to divide the user data to be transmitted into messages of a set length, compiling sequence numbers for the messages and filling immediate fields in the messages, generating the data message, and storing the data message in the sending queue in sequence.

[0014] For example, in some embodiments of the present application, controlling the sender to send the data packet in the sending queue to the connectionless RDMA network according to the global identifier and the queue pair number, so that the receiving end receives the data packet, and controlling the sender to receive the response packet returned by the receiving end, includes: controlling the sender to adopt a unicast mode or a multicast mode to send the data packet to the connectionless RDMA network, and starting a timer to wait for receiving the response packet; if the sender receives the response packet within a set time threshold, controlling the sender to update the sending queue according to the response packet; if the sender does not receive the response packet within the set time threshold, controlling the sender to re-send the data packet to the connectionless RDMA network.

[0015] For example, in some embodiments of the present application, controlling the sending end to adopt a unicast mode or a multicast mode, sending the data packet to the connectionless RDMA network, and starting a timer to wait for receiving the response packet, includes: controlling the sending end to construct a data packet according to the global identifier, the queue pair number and the data packet, and sending the data packet to the connectionless RDMA network.

[0016] For example, in some embodiments of the present application, controlling the sending end to send the data packets in the sending queue to the connectionless RDMA network according to the global identifier and the queue pair number so that the receiving end receives the data packets, and controlling the sending end to receive the response packets returned by the receiving end, further includes: setting a sliding window mechanism to constrain the length of the sending queue; wherein setting the sliding window mechanism includes: A tail pointer and a head pointer are set, wherein the tail pointer corresponds to the minimum message sequence number of the data messages in the sending queue that have been sent to the connectionless RDMA network but for which no response message returned by the receiving end has been received, and the head pointer corresponds to the message sequence number of the next data message to be sent in the sending queue; the message sequence number of the response message returned by the receiving end is parsed; and when the tail pointer is less than or equal to the head pointer and the message sequence number of the response message is between the tail pointer and the head pointer, or when the tail pointer is greater than the head pointer and the message sequence number of the response message is less than or equal to the head pointer or greater than the tail pointer, the tail pointer is updated to the message sequence number of the response message.

[0017] According to the second aspect of the present application, at least one embodiment of the present application provides a data transmission method based on a connectionless RDMA network, which is applied to a receiving end, and the data transmission method includes: initializing a receiving queue; controlling the receiving end to establish a virtual session with a sending end; controlling the receiving end to exchange a global identifier and a queue pair number with the sending end; controlling the receiving end to receive a data packet sent by the sending end from the connectionless RDMA network according to the global identifier and the queue pair number, and returning a response packet to the sending end; determining whether the receiving queue is empty; and if the receiving queue is empty, controlling the sending end to end the virtual session with the receiving end.

[0018] For example, in some embodiments of the present application, the initializing the receiving queue includes: pre-allocating a message buffer required by the receiving end to receive the data message sent by the sending end.

[0019] For example, in some embodiments of the present application, the controlling the receiving end to establish a virtual session with the sending end includes: when the sending end and the receiving end establish a virtual session for the first time, allocating a receiving queue dedicated to the receiving end from the initialized receiving queue for data transmission of the virtual session; applying for N address ranges from the message buffer, the address ranges being used to provide registered memory space for data transmission, where N is an integer greater than or equal to 1.

[0020] For example, in some embodiments of the present application, the controlling the receiving end receives the data packet sent by the sending end from the connectionless RDMA network according to the global identifier and the queue pair number, and returns a response packet to the sending end, including: controlling the receiving end to parse the data packet sent by the sending end, obtaining the global identifier and the queue pair number of the sending end to determine the sending end; and controlling the receiving end to obtain the immediate field of the data packet to extract the session ID, splicing the global identifier, the queue pair number and the session ID, and determining the session type and specific session content of the data packet.

[0021] For example, in some embodiments of the present application, the controlling the receiving end to receive the data packet sent by the sending end from the connectionless RDMA network according to the global identifier and the queue pair number, and return a response message to the sending end, further includes: controlling the receiving end to obtain the message sequence number of the received data packet; controlling the receiving end to query and determine the highest message sequence number among the consecutive message sequence numbers in the received message sequence numbers at a set interval; controlling the receiving end to construct the response message according to the highest message sequence number, and sending the response message to the sending end through the connectionless RDMA network.

[0022] For example, in some embodiments of the present application, the controlling the receiving end receives the data packet sent by the sending end from the connectionless RDMA network according to the global identifier and the queue pair number, and returns a response message to the sending end, and also includes: controlling the receiving end to remove the data packet to be received corresponding to the message sequence number in the receiving queue according to the received message sequence number.

[0023] According to the third aspect of the present application, at least one embodiment of the present application provides a data transmission system based on a connectionless RDMA network, which is used to execute the data transmission method as described in any one of the first aspect or the second aspect, and the data transmission system includes: an interface component, which is used to provide a sending interface and a receiving interface for the sending end and the receiving end; a session management component, which is used to exchange the global identifiers and queue pair numbers of the sending end and the receiving end in an out-of-band manner according to the sending interface and the receiving interface, establish a virtual session and maintain the session ID; a virtual session management component, which is used to manage the global identifiers and the queue pair numbers of the sending end and the receiving end to provide routing positioning of data packets in the connectionless RDMA network; a transmission execution component, which is used to control the sending end and the receiving end, and drive the sending queue and the receiving queue to perform the data packet processing; and is used to set a sliding window mechanism to limit the length of the sending queue and the receiving queue.

[0024] According to the fourth aspect of the present application, at least one embodiment of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors execute a method as described in any one of the first aspect or the second aspect.

[0025] According to the fifth aspect of the present application, at least one embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspect or the second aspect.

[0026] Through the above exemplary embodiments, the present application provides a data transmission method and system based on a connectionless RDMA network. By means of innovative message format design, session management, sequence number mechanism, out-of-order detection, automatic retransmission and other technical means, efficient and reliable data transmission in a connectionless RDMA network environment is achieved, and at least one of the following beneficial effects is achieved: High scalability: Using the connectionless UD mode, the number of QPs remains constant, avoiding performance crashes caused by the increase in the number of connections in the RC mode, and supporting large-scale clusters.

[0027] Reliable transmission guarantee: Through sequence numbers, sliding windows, and timeout retransmission mechanisms, it solves the disorder and packet loss problems of UD mode, and combines ACK feedback to achieve efficient and reliable transmission.

[0028] Transmission efficiency optimization: Utilize the zero-copy feature of RDMA to pre-allocate address ranges and register hardware, reducing CPU involvement; lightweight header design increases data payload ratio.

[0029] Multi-scenario adaptation: supports unicast / multicast transmission, and multicast can reduce bandwidth usage; asynchronous API and event-driven mechanism improve concurrent processing capabilities and adapt to distributed applications.

[0030] It should be understood that the foregoing general description and the following detailed description are merely illustrative and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] By describing in detail exemplary embodiments thereof with reference to the accompanying drawings, the above and other objects, features and advantages of the present application will become more apparent. The drawings described below are only some embodiments of the present application, and are not intended to limit the present application.

[0032] Figure 1 Flowchart of a data transmission method based on a connectionless RDMA network according to an exemplary embodiment; Figure 2 The format design of data message and response message for an exemplary embodiment; Figure 3 A flow chart of a data transmission method performed by a transmitting end according to an exemplary embodiment; Figure 4 A flow chart of a data transmission method performed by a receiving end according to an exemplary embodiment; Figure 5 Schematic diagram of a data transmission system based on a connectionless RDMA network according to an exemplary embodiment; Figure 6 This is a structural diagram of an electronic device provided in this application. DETAILED DESCRIPTION

[0033] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. Like reference numerals in the drawings represent like or similar parts, and thus repetitive description thereof will be omitted.

[0034] The described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of these specific details, or other methods, components, materials, devices, etc. may be employed. In these cases, well-known structures, methods, devices, implementations, materials or operations will not be shown or described in detail.

[0035] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0036] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0037] Those skilled in the art will understand that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present application, and therefore cannot be used to limit the scope of protection of the present application.

[0038] In this application, networks based on the unreliable datagram (UD) transmission model are collectively referred to as "connectionless RDMA networks." Therefore, regardless of the number of connected targets, the client can maintain a constant number of queue pairs (QPs). Furthermore, connectionless datagram transmission supports multicast communication, allowing a single sender to efficiently broadcast information to multiple receivers. Multicast technology reduces the sender's outbound bandwidth requirements, thus benefiting certain distributed applications, such as data replication.

[0039] like Figure 1 As shown, the data transmission method based on the connectionless RDMA network includes steps S1-S6.

[0040] In step S1, the phase is initialized.

[0041] Specifically, it includes initializing the send queue and receive queue. The send queue is used to store data packets to be sent and manage the sending tasks in sequence. The receive queue is used to buffer received packets and verify the data according to the sequence number.

[0042] According to the example embodiment, key data structures such as the sending queue and the receiving queue are initialized, and the message buffer required by the sending end to send data messages and the message buffer required by the receiving end to receive data messages sent by the sending end are pre-allocated. The relevant buffers will be used in subsequent steps.

[0043] In step S2, the session preparation phase begins.

[0044] Specifically, the process includes: step S2.1, controlling the sending end and the receiving end to establish a virtual session; step S2.2, controlling the sending end and the receiving end to exchange their respective global identifiers (GIDs) and queue pair numbers (QPs); step S2.3, controlling the sending end to construct data packets and store the data packets in the sending queue in order.

[0045] According to an exemplary embodiment, the sending end and the receiving end respectively establish a virtual session through a specified API interface of the interface component, for example, the specified API interfaces: send() and recv().

[0046] According to an example embodiment, during an API call, if a virtual session has not previously been established between the sender and the receiver (i.e., this is the first transmission between the sender and the receiver), a receive queue dedicated to the receiver is allocated from the initialized receive queues, and a send queue dedicated to the sender is allocated from the initialized send queues for data transmission within the virtual session. Furthermore, N address ranges are pre-allocated from the message buffer, each with a length equal to the MTU. The address ranges are used to provide registered memory space for data transmission, where N is an integer greater than or equal to 1.

[0047] According to some embodiments, a queue requires physical memory space to store message data, and therefore requests an address range from the message buffer as the queue's storage medium. In connectionless RDMA networks, data must be fragmented according to the Maximum Transmission Unit (MTU). Matching the length of each address range to the MTU ensures that fragmented messages can be stored intact. Furthermore, by pre-requiring N address ranges, temporary memory allocation can be avoided during each send / receive operation, reducing latency.

[0048] When establishing a virtual session, the sender and receiver preemptively send N receive message requests to the RDMA hardware, specifying an address range as the data storage location. Upon receiving the request, the RDMA hardware proactively writes the message directly to the corresponding address range, implementing a "passive receive" mode and reducing software intervention overhead.

[0049] RDMA hardware (such as a network card) cannot actively monitor network data. The software must send a "receive request" in advance to inform the hardware to prepare to receive N messages and specify the address range for storing the messages. Each receive request corresponds to an address range. After the hardware receives the message, it will write the data directly to the corresponding address range according to the request, achieving "zero-copy" transmission. However, RDMA hardware can only process one message per receive request. After the hardware processes a message, the software must send a receive request again to continue monitoring the network. Therefore, after the RDMA hardware completes processing a message, the software must resend the request to ensure that the hardware continues to monitor network data and maintain transmission continuity.

[0050] After a session is established, the sender and receiver exchange their respective global identifiers (GIDs) and all registered queue pair numbers (QPs) through out-of-band communication (including but not limited to a separately maintained TCP connection). This information exchange enables the sender and receiver to accurately identify each other and prepare for subsequent data transmission.

[0051] Step S2.3 includes: controlling the sending end to cut the user data to be transmitted into messages that meet the set length, compiling sequence numbers for the messages and filling immediate number fields in the messages, generating data messages, and storing the data messages in sequence in the sending queue; controlling the receiving end to cut the data to be received into messages that meet the set length, compiling sequence numbers for the messages and filling immediate number fields in the messages, generating data messages, and storing them in the receiving queue.

[0052] For example, the sender divides the user data to be transmitted into packets that meet the MTU size, with 2 24 Assign a sequence number to the module, construct a header and fill it with the immediate field corresponding to the message, generate a data message, and store it in the sending queue in sequence.

[0053] For example, the receiving end divides the user data to be received into packets that meet the MTU size, with 2 24 Assign a sequence number to the module, construct a header to fill in the immediate field corresponding to the message, generate a data message, and store it in the receive queue in sequence.

[0054] According to some embodiments, both data packets and response packets (ACKs) use the same message format at both the sending and receiving ends. To maximize the proportion of data in the message, the header consists of two parts, one preceding the immediate field and the other preceding the payload. The header preceding the immediate field can be used to transmit core data, while the header preceding the payload can be used to convey additional information for window management and retransmission management. The header contains the fields shown in Table 1.

[0055] Table 1. Message header fields for reliable data transmission over a connectionless RDMA network

[0056] Figure 2 The format design of data packets (DATA) and response packets (ACK) is shown. Both types of packets contain header information and a global routing header (GRH) generated by the RDMA hardware. Data packets also include a payload. The header information is distributed in the extended area before the immediate value and the payload.

[0057] In step S3, the message sending phase begins.

[0058] Specifically, it includes: controlling the sending end to send the data message in the sending queue to the connectionless RDMA network according to the global identifier and queue pair number, so that the receiving end receives the data message, and controlling the sending end to receive the response message returned by the receiving end. It includes: S3.1, controlling the sending end to use unicast mode or multicast mode to send the data message to the connectionless RDMA network, and starting the timer to wait for receiving the response message; S3.2, when the sending end receives the response message sent by the receiving end within the set time threshold, controlling the sending end to update the sending queue according to the response message; S3.3, when the sending end does not receive the response message within the set time threshold, controlling the sending end to resend the data message to the connectionless RDMA network. The sending end executes the data transmission method process as follows Figure 3 shown.

[0059] According to some embodiments, in step S3.1, the unicast mode means that the address information of the sending target needs to be specified, that is, the address range obtained in step S2, and filled into the data message for sending; the multicast mode means that the multicast address needs to be filled.

[0060] Furthermore, in step S3.1, the control sender constructs a data packet based on the global identifier, queue pair number, and data message, and sends the data packet to the connectionless RDMA network. The message in the send queue is sent to the network using unicast or multicast mode, and a timer is started to wait for a response message from the receiver.

[0061] According to some embodiments, the sending end divides the user data to be transmitted in the sending queue into messages of a set length, fills them into the registered address range, and Figure 2 The header and digital payload are constructed according to the format of Table 1 to form a data packet (including header and data payload) that conforms to the RDMA format. The data packets are stored in the sending queue in sequence and then read by the RDMA hardware and sent to the connectionless RDMA network.

[0062] Step S3.2: When the sender receives a response message within the set time threshold, it indicates that the corresponding data message has been confirmed, and the data message at the end of the queue will be removed from the send queue.

[0063] Step S3.3: When the sender does not receive a response message within the set time threshold, it triggers the retransmission mechanism to retransmit all the messages still in the send queue. Among them, data messages whose time since the last transmission is less than the specified interval do not need to be resent. This solution can avoid the situation of a large number of retries reducing the bandwidth utilization rate.

[0064] According to some embodiments, for the unicast mode, the end of the send queue represents the smallest unacknowledged message of the receiver, and all the messages still in the send queue are retransmitted; for the multicast mode, the smallest unacknowledged message of each receiver is recorded separately, and all the messages still in the send queue are retransmitted, and all the above messages need to be covered (which can be understood as the earliest one among all the smallest unacknowledged messages: for example, if the smallest unacknowledged message number of receiver target 1 is 1001 and that of receiver target 2 is 1003, the end of the send queue should cover 1001 and the subsequent part).

[0065] According to some embodiments, the set time threshold Ts can be selected as 4 us.

[0066] This application also sets a sliding window mechanism to dynamically adjust the number of unacknowledged messages in the send queue to restrict the length of the send queue, avoid network congestion, and ensure the sequential transmission of data.

[0067] According to the exemplary embodiments, setting the sliding window mechanism includes: setting a tail pointer and a head pointer. The tail pointer tail corresponds to the smallest message sequence number of the data message that has been sent to the connectionless RDMA network but has not received a response message returned by the receiver in the send queue, and the head pointer head corresponds to the message sequence number of the next data message to be sent to the receiver in the send queue. The sender analyzes the message sequence number of the response message returned by the receiver. When the tail pointer is less than or equal to the head pointer and the message sequence number of the response message is between the tail pointer and the head pointer, or when the tail pointer is greater than the head pointer and the message sequence number of the response message is less than or equal to the head pointer or greater than the tail pointer, the tail pointer is updated to the message sequence number of the response message.

[0068] For example, when the receiver returns a response message, if the sequence number (sn) in the ACK is greater than tail, then tail is updated to sn, indicating that the data messages before sn have been reliably received. If tail ≤ head, the sliding window is between tail and head. At this time, if tail < sn ≤ head, then change tail = sn.

[0069] According to some embodiments, the sequence number (sn) is expressed in 2 24 This is a modular loop, so it's necessary to handle edge cases under modular operations (for example, when tail > head, the window covers both the tail and the head of the queue). Therefore, if tail > head, the sliding window is positioned from tail to the tail of the send queue and from the head of the send queue to the head. In this case, if sn ≤ head or sn > tail, change tail to sn.

[0070] In other cases, the tail content is not modified.

[0071] According to some embodiments, for multiple receivers, the send queue maintains a tail array, where each element corresponds to the minimum unacknowledged sequence number of a receiver. The window size of the entire queue is determined by the difference between the minimum value of all tails and the head (i.e., the earliest unacknowledged message is used as the window boundary).

[0072] According to an exemplary embodiment, the length of the sending queue is limited by the size of the sending sliding window. When the sending sliding window is full, only when the tail advances can the data packets in the sending packet buffer be moved into the sending queue.

[0073] In step S4, the message receiving stage. The receiving end executes the data transmission method flow as follows Figure 4 The message receiving stage specifically includes: Step S4.1 to Step S4.4.

[0074] In step S4.1, after receiving a valid message, the receiving end parses the data message sent by the sending end, obtains the global identifier and queue pair number of the sending end to determine the sending end, and controls the receiving end to obtain the immediate field of the data message to extract the session ID, and concatenates the global identifier, queue pair number and session ID to determine the session type and specific session content of the data message.

[0075] According to some embodiments, after receiving a valid message, the receiver extracts the header information from the immediate value and determines the session based on the Work Completion Entry (WCE) and the Global Route Header (GRH) area of the message. Specifically, the receiver breaks it down into two stages: Use the <SGID, queue pair number> pair to determine the sending instance of the message. After performing the RECV operation under the RDMA UD protocol, 40-byte GRH header data is written before the data area, which contains the global address information identifying the sender (i.e., the SGID field). At the same time, the corresponding work completion record also contains the queue pair number used by the sender. These two fields together can uniquely determine a specific queue pair within the cluster, and thus can locate a certain Rapid Transfer instance. The above mode can also correctly handle the single-machine multi-instance scenario. Based on the above information, the message sender / receiver can determine an internally maintained session prefix. The session prefixes for a certain instance among different instances may not be unified, but this will not affect message distribution and will not be reflected in the session field.

[0076] Use the session field in the message header to determine the specific session. Concatenate the session prefix obtained in the previous step with the session field (unsigned extended to 8 bits) to form a complete session number, which is unique in each instance.

[0077] Step S4.2: Determine the message type. If it is a response message, go to step S4.3. Otherwise, execute step S4.4.

[0078] Step S4.3: The sender updates the tail of the sliding window of the sending queue according to the message sequence number carried in the response message.

[0079] Step S4.4: The receiver obtains the message sequence number of the received data message, and determines whether the message is the expected message according to whether the message sequence number is in the receiving queue; if it is the expected message, and the message is in the receiving queue and has not been received previously, write it into the message buffer specified by the user; and control the receiver to query and determine the highest message sequence number among the consecutive message sequence numbers of the received message sequence numbers at an interval of the set time; control the receiver to construct a response message according to the highest message sequence number and send the response message to the sender through the connectionless RDMA network.

[0080] For example, when the receiver receives messages..., 1003, 1004, 1006, 1007, the "highest data message sequence number continuously received in the current session" is 1004, construct a response message with 1004, fill in the SN, and send it to the sender of this session.

[0081] According to some embodiments, after the RDMA hardware receives a data packet, it directly writes the data into the corresponding memory according to the pre-registered address range. The receiving queue reads the data from this memory space, and checks the legality and sequence of the message through the sequence number. After passing the check, the data is submitted to the message buffer specified by the user.

[0082] Step S4.4: Control the receiving end to remove the to-be-received data message corresponding to the message sequence number in the receiving queue according to the received message sequence number.

[0083] For example, when the receiver receives messages ..., 1003, 1004, 1006, 1007, the receiver removes the data packets corresponding to 1003 and 1004 in the receiving queue.

[0084] In step S5, it is determined whether the queue is in the empty stage.

[0085] The stage of determining whether the queue is empty specifically includes: determining whether the sending queue and the receiving queue are empty: if the sending queue / receiving queue is not empty, repeating steps S3 and S4; if the sending queue and the receiving queue are empty, going to step S6.

[0086] In step S6, the session ends.

[0087] If the send and receive queues are empty, the sender and receiver are controlled to terminate the virtual session. This is because at the end of a session, when the send and receive queues are empty, the transmission is complete. After the data transmission is complete, the sender and receiver terminate the session through the API. This releases session-related memory resources such as message buffers and queues, clearing the session state.

[0088] In this application, the transmitting end can also be used as the receiving end to execute the receiving end's data transmission method; the receiving end can also be used as the transmitting end to execute the transmitting end's data transmission method. The above data transmission method is described from the perspective of the transmitting end and the receiving end respectively in order to better illustrate the steps of the transmitting end and the receiving end in executing the data transmission method. This application only uses this as an example, but is not limited to this.

[0089] The present application also provides a data transmission system based on a connectionless RDMA network, which is used to execute the data transmission method based on a connectionless RDMA network as described above.

[0090] like Figure 5 As shown, the data transmission system based on the connectionless RDMA network includes: an interface component, a session management component, a virtual session management component and a transmission execution component.

[0091] According to an example embodiment, the interface component is used to provide a sending interface and a receiving interface for the sending and receiving ends. The interface component provides a set of asynchronous and stream-based data transmission application programming interfaces (APIs), similar to traditional connection-based APIs (such as the TCP socket interface). In addition, the interface component supports one-to-many reliable data transmission, which can simultaneously meet the transmission needs of multiple users or multiple data streams.

[0092] According to some embodiments, a reliable data transmission system based on a connectionless RDMA network provides multiple APIs for asynchronous and reliable data transmission. Users submit send and receive tasks through a sending interface (such as send) and a receiving interface (such as recv), respectively. Both methods require the peer's network address, the memory address of the data buffer, and a session ID. Users can use getStatus() to query whether a task has completed or failed.

[0093] Reliable data transfer systems based on connectionless RDMA networks are stream-based, that is, all bytes received are identical to the bytes sent in each virtual session.

[0094] For example, there is a virtual session between Alice (the party that calls send()) and Bob (the party that calls recv()): (1) Alice's peer network address is Bob, (2) Bob's peer network address is Alice, and (3) they have the same session ID. The third rule ensures that different transmission streams between any two nodes are isolated. This allows multiple files to be transferred simultaneously. Each transmission path supports up to 2 6 =64 sessions.

[0095] To further ensure the integrity and orderliness of data transmission, this application establishes an independent "session" for each file transfer. Different sessions are isolated from each other and do not interfere with each other.

[0096] According to an example embodiment, a session management component is configured to establish a virtual session and maintain a session ID by exchanging global identifiers and queue pair numbers between the sender and receiver through an out-of-band method based on the sending and receiving interfaces. The session management component implements data sequencing and timeout retransmission functions through the send / receive circular queue, ensuring that data is transmitted in the correct order and enabling timely retransmission in the event of a transmission error, thereby ensuring data integrity.

[0097] According to some embodiments, in a reliable data transmission system based on a connectionless RDMA network, a virtual point-to-point transmission path is formed between a sending instance and a receiving instance. To further distinguish different transmission tasks within the same path, this application introduces the concept of a session, thereby supporting the concurrent transmission of multiple files between two instances (rather than transferring files one by one).

[0098] In sending and receiving operations, users can specify the session number used in a specific transmission channel, which will be reflected in the session number field in the header of each message involved. 6= 64 sessions. The session ID used in grouped send and receive operations must be consistent. All send and receive operations within the same session are executed in the order submitted; there is no order constraint between different sessions.

[0099] Under the same transmission path and session, this application uses a sequence number mechanism to determine the expected order of message transmission. For each transmission request, it will be divided into several messages according to the MTU. Each message is assigned a sequence number in a continuously increasing manner and sent to the remote node together with the data itself. The remote node tracks the sequence number in order to determine whether the specified message has arrived locally. The range of the sequence number is 0 to 2 24 -1. If the current sequence number is 2 24 If the value of the sequence number is -1, the next sequence number will be 0. Because the sliding window size is much smaller than the sequence number range, allocating 24 bits to the sequence number field is sufficient. For the sender, once a message is constructed, its sequence number remains unchanged even if it is retransmitted. This design ensures the integrity and validity of received messages.

[0100] The sliding window is a key concept in flow control and congestion control mechanisms, ensuring reliable and efficient data transmission. The send window defines how much data a sender can send before waiting for an acknowledgment from the receiver. The send window size determines the amount of data a sender can send continuously without waiting for an acknowledgment. The receive window defines the amount of data a receiver can receive but has not yet acknowledged. The receive window size determines how much unacknowledged data a receiver can temporarily store. The sliding window within the send / receive queue ranges from 1 to 65535 and is automatically adjusted by the congestion control algorithm.

[0101] Different from the traditional way of maintaining connections at the device level, this application provides point-to-point data transmission abstraction through "virtual sessions".

[0102] According to an example embodiment, a virtual session management component manages global identifiers and queue pair numbers for senders and receivers to provide routing for data packets in a connectionless RDMA network. The virtual session management component manages these virtual sessions to ensure that data is accurately transmitted from the sender to the receiver in the connectionless RDMA network.

[0103] According to some embodiments, a "virtual session" provides a point-to-point data transfer abstraction. Each virtual session represents a remote node and contains an address handle (AH) object that specifies the remote queue pair (QP) for the packet. Furthermore, the virtual session management component manages multiple QPs and completion queues (CQs), each associated with a QP. It's important to note that the number of QPs and CQs does not necessarily equal the number of AHs in the cluster.

[0104] When sending a data packet, the virtual session management component selects the corresponding AH object and any QP object (as long as there are sufficient available work requests) from the endpoint storage. The endpoint storage receives the data packet by traversing all active CQs. To improve transmission efficiency, the sender can use RDMA's zero-copy feature to send user data directly to the RDMA hardware. In this mode, there is no need to allocate additional memory buffers, but instead the user-provided memory area is used directly. However, the user-provided memory area must be registered in advance. If the user-provided memory area is not registered, memory space must be allocated from the memory buffer pool and the data must be copied to the allocated memory space.

[0105] When sending a response (ACK) packet, the virtual session management component reserves a registered memory area to store the timestamp (ts) and window size (wnd) fields. Based on the improved Go-Back-N (GBN) retransmission algorithm, there is no need to transmit a corresponding ACK packet for each data packet. Instead, a fixed-size memory area is allocated for each session to construct an extension header and send it back with the ACK packet. Furthermore, the virtual session management component ensures that the next ACK packet is prepared and sent only after the previous ACK packet successfully reaches the remote node or is confirmed to be lost, thus avoiding packet loss and ensuring reliable and efficient data transmission.

[0106] When receiving packets, the virtual session management component efficiently processes them due to the low packet loss rate of RDMA networks. Using the GBN retransmission algorithm described above, the receiver can accurately identify lost packets and request retransmission, ensuring data integrity and consistency. Furthermore, the virtual session management component relies on the send / receive queues of each session to implement a packet sorting mechanism, ensuring that packets are processed in the correct order.

[0107] To drive the implementation of the above functions, this application uses a dedicated thread to implement an event-driven transmission execution component.

[0108] The transmission execution component controls the sender and receiver, including driving the send and receive queues to process data packets and setting a sliding window mechanism to limit the length of the send and receive queues. The transmission execution component is responsible for monitoring and handling various transmission events, such as data send requests, data receipt confirmations, and timeout events, to efficiently drive the entire data transmission process.

[0109] This application provides a data transmission method and system based on a connectionless RDMA network. Through innovative message format design, session management, sequence numbering, out-of-order detection, and automatic retransmission techniques, this method achieves efficient and reliable data transmission in a connectionless RDMA network environment. Specifically, this application rationally utilizes metadata within RDMA UD standard packets to minimize the space cost of data transmission (e.g., packet header fields). Compared to Ethernet, connectionless RDMA networks offer ultra-low latency and packet loss rates. Therefore, the reliable data transmission method based on connectionless RDMA networks incorporates multiple optimizations for this scenario to improve packet processing performance. Furthermore, this application implements a reliable data transmission system based on a connectionless RDMA network, supporting concurrent data transmission to multiple destinations through a stream-based asynchronous application programming interface (API). Regardless of cluster size, the reliable data transmission system requires only a constant number of queue pairs (QPs), eliminating cache space contention on RDMA devices and improving data transmission scalability. Furthermore, the reliable data transmission system supports reliable data transmission using multicast protocols, avoiding the need to send duplicate packets to multiple destinations. By optimizing resource utilization and minimizing hardware dependencies, the reliable data transmission system based on the connectionless RDMA network provides a flexible and efficient solution for modern data-intensive applications, achieving high-throughput and low-latency data transmission in large clusters.

[0110] Figure 6 This is a structural diagram of an electronic device provided in this application.

[0111] See Figure 6 , Figure 6 An electronic device is provided, comprising a processor and a memory. The memory stores computer instructions, and when the computer instructions are executed by the processor, the processor executes the computer instructions to achieve the following Figure 1 、 Figure 3 and Figure 4 The method and refinement scheme shown.

[0112] It should be understood that the above-described device embodiments are merely illustrative, and the devices disclosed herein may also be implemented in other ways. For example, the division of units / modules in the above-described embodiments is merely a logical functional division, and actual implementations may employ other division methods. For example, multiple units, modules, or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0113] In addition, unless otherwise specified, the functional units / modules in the various embodiments of the present application may be integrated into a single unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The aforementioned integrated units / modules may be implemented in the form of hardware or software program modules.

[0114] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, and so on. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, and so on. Unless otherwise specified, a processor or chip can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, or ASIC. Unless otherwise specified, on-chip cache, off-chip memory, and storage can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), and hybrid memory cube (HMC).

[0115] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method disclosed herein. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk, or optical disk, etc., various media that can store program code.

[0116] The embodiment of the present application also provides a non-transitory computer storage medium storing a computer program, which, when executed by multiple processors, causes the processors to execute the following Figure 1 、 Figure 3 and Figure 4The method and refinement scheme shown.

[0117] It should be clearly understood that this application describes how to form and use specific examples, but this application is not limited to any details of these examples. On the contrary, based on the teaching of the content disclosed in this application, these principles can be applied to many other embodiments.

[0118] Furthermore, it should be noted that the aforementioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present application and are not intended to be limiting. It is readily understood that the processes illustrated in the aforementioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0119] The exemplary embodiments of the present application are specifically shown and described above. It should be understood that the present application is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present application is intended to cover various modifications and equivalent configurations within the scope and purpose of the appended claims.

Claims

1. A data transmission method based on a connectionless RDMA network, applied to a sending end, characterized in that: The data transmission method includes: Initialize the send queue; Controlling the sending end and the receiving end to establish a virtual session; Controlling the sending end and the receiving end to exchange global identifiers and queue pair numbers; Controlling the sending end to construct data messages and storing the data messages in the sending queue in order; Controlling the sending end to send the data message in the sending queue to the connectionless RDMA network according to the global identifier and the queue pair number, so that the receiving end receives the data message, and controlling the sending end to receive a response message returned by the receiving end; Determine whether the sending queue is empty; When the sending queue is empty, the sending end and the receiving end are controlled to end the virtual session.

2. The data transmission method according to claim 1, wherein: The initialization sending queue includes: Pre-allocate a message buffer required by the sending end to send the data message.

3. The data transmission method according to claim 2, wherein: The controlling the sending end and the receiving end to establish a virtual session includes: When the sending end and the receiving end establish a virtual session for the first time, allocating a sending queue dedicated to the sending end from the initialized sending queues for data transmission of the virtual session; Apply for N address ranges from the message buffer, where the address ranges are used to provide registered memory space for data transmission, and N is an integer greater than or equal to 1.

4. The data transmission method according to claim 1, wherein: The controlling the sending end to construct a data message and storing the data message in the sending queue in order includes: The transmitting end is controlled to cut the user data to be transmitted into messages of a set length, sequence numbers are compiled for the messages and immediate fields are filled in the messages, the data messages are generated, and the data messages are sequentially stored in the transmitting queue.

5. The data transmission method according to claim 2, wherein: The controlling the sending end to send the data message in the sending queue to the connectionless RDMA network according to the global identifier and the queue pair number, so that the receiving end receives the data message, and controlling the sending end to receive a response message returned by the receiving end, includes: Controlling the sending end to adopt a unicast mode or a multicast mode to send the data message to the connectionless RDMA network, and starting a timer to wait for receiving the response message; When the sending end receives the response message within a set time threshold, controlling the sending end to update the sending queue according to the response message; When the sending end does not receive the response message within the set time threshold, the sending end is controlled to resend the data message to the connectionless RDMA network.

6. The data transmission method according to claim 5, wherein: The controlling the sending end to adopt a unicast mode or a multicast mode to send the data message to the connectionless RDMA network, and starting a timer to wait for receiving the response message, including: The sending end is controlled to construct a data packet according to the global identifier, the queue pair number and the data message, and send the data packet to the connectionless RDMA network.

7. The data transmission method according to claim 5, wherein: The controlling the sending end to send the data message in the sending queue to the connectionless RDMA network according to the global identifier and the queue pair number, so that the receiving end receives the data message, and controlling the sending end to receive a response message returned by the receiving end, further includes: Setting a sliding window mechanism to constrain the length of the sending queue; The sliding window mechanism is set up as follows: Setting a tail pointer and a head pointer, wherein the tail pointer corresponds to the minimum message sequence number of the data message in the sending queue that has been sent to the connectionless RDMA network but has not received a response message returned by the receiving end, and the head pointer corresponds to the message sequence number of the next data message to be sent in the sending queue; Parsing the message sequence number of the response message returned by the receiving end; When the tail pointer is less than or equal to the head pointer and the message sequence number of the response message is between the tail pointer and the head pointer, or when the tail pointer is greater than the head pointer and the message sequence number of the response message is less than or equal to the head pointer or greater than the tail pointer, the tail pointer is updated to the message sequence number of the response message.

8. A data transmission method based on a connectionless RDMA network, applied to a receiving end, characterized in that: The data transmission method includes: Initialize the receive queue; Controlling the receiving end to establish a virtual session with the sending end; Controlling the receiving end to exchange a global identifier and a queue pair number with the sending end; Controlling the receiving end to receive the data message sent by the sending end from the connectionless RDMA network according to the global identifier and the queue pair number, and returning a response message to the sending end; Determine whether the receiving queue is empty; When the receiving queue is empty, the sending end and the receiving end are controlled to end the virtual session.

9. The data transmission method according to claim 8, wherein: The initialization receiving queue includes: A message buffer required by the receiving end to receive the data message sent by the sending end is pre-allocated.

10. The data transmission method according to claim 9, wherein: The controlling the receiving end to establish a virtual session with the sending end includes: When the sending end and the receiving end establish a virtual session for the first time, allocating a receiving queue dedicated to the receiving end from the initialized receiving queues for data transmission of the virtual session; Apply for N address ranges from the message buffer, where the address ranges are used to provide registered memory space for data transmission, and N is an integer greater than or equal to 1.

11. The data transmission method according to claim 8, wherein: The controlling the receiving end to receive the data message sent by the sending end from the connectionless RDMA network according to the global identifier and the queue pair number, and returning a response message to the sending end, includes: Controlling the receiving end to parse the data message sent by the sending end, obtaining the global identifier and the queue pair number of the sending end, so as to determine the sending end; and The receiving end is controlled to obtain an immediate field of the data message to extract a session ID, and the global identifier, the queue pair number and the session ID are concatenated to determine a session type and specific session content of the data message.

12. The data transmission method according to claim 11, wherein: The controlling the receiving end to receive the data message sent by the sending end from the connectionless RDMA network according to the global identifier and the queue pair number, and returning a response message to the sending end further includes: Controlling the receiving end to obtain a message sequence number of a received data message; Controlling the receiving end to set an interval time, querying and determining the highest message sequence number among the consecutive message sequence numbers in the received message sequence numbers; The receiving end is controlled to construct the response message according to the highest message sequence number, and the response message is sent to the sending end through the connectionless RDMA network.

13. The data transmission method according to claim 12, wherein: The controlling the receiving end to receive the data message sent by the sending end from the connectionless RDMA network according to the global identifier and the queue pair number, and returning a response message to the sending end further includes: The receiving end is controlled to remove the to-be-received data message corresponding to the message sequence number in the receiving queue according to the received message sequence number.

14. A data transmission system based on a connectionless RDMA network, characterized in that: Used to perform the data transmission method according to any one of claims 1 to 13, the data transmission system comprising: The interface component is used to provide a sending interface and a receiving interface for the sending end and the receiving end; a session management component, configured to exchange the global identifiers and queue pair numbers of the sending end and the receiving end in an out-of-band manner according to the sending interface and the receiving interface, establish a virtual session, and maintain a session ID; A virtual session management component, configured to manage the global identifiers and queue pair numbers of the sender and the receiver to provide routing positioning of data packets in a connectionless RDMA network; A transmission execution component is used to control the sending end and the receiving end, to drive the sending queue and the receiving queue to process the data message; and to set a sliding window mechanism to limit the length of the sending queue and the receiving queue.

15. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images