A reliable multicast transmission method for RoCE protocol

Abstract

The present invention relates to the field of communication technology, and specifically to a reliable multicast transmission method for the RoCE protocol, the method comprising: S1 divides the multicast source and the multicast group in a hierarchical manner; S2 the sending end sends control information in the UDP protocol, the receiving end listens to the control message, establishes a connection through the RDMA CM API and completes the QP ready state transition; S3 the sending end and the receiving end use the RC mode for reliable data transmission, the forwarding node performs concurrent forwarding in the Reactor multi-threaded mode, the main thread polls and receives CQ events, determines the forwarding target node, copies the message content to the sending buffer and initiates an asynchronous operation; S4 develops an API interface to realize the SEND-RECEIVE bilateral operation and the READ-WRITE unilateral operation in the multicast scenario. The advantages of the present invention are: realizing the reliable multicast transmission function and supporting many-to-many concurrent forwarding scenarios.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a reliable multicast transmission method oriented to the RoCE protocol. Background Art

[0002] In the financial information services sector, market information systems are facing increasing demands for high-throughput and low-latency technical architectures. To address this challenge, market information systems are adopting more advanced network technologies and facilities to reduce latency in market information transmission and enhance system performance. To better adapt to market demands and improve user satisfaction, many companies have begun researching RDMA technology, which demonstrates significant advantages in improving data transmission efficiency, reducing latency, and enhancing system stability, and holds great potential for application.

[0003] Currently, market information systems primarily utilize traditional TCP / IP network communication (BSD Sockets) for data transmission. UDP multicast is widely used in market information systems due to its efficient data distribution capabilities, ensuring that multiple receivers can receive real-time market information almost simultaneously, achieving data synchronization. However, some existing multicast solutions still face hardware limitations, packet loss, and out-of-order data transmission. These issues increase data transmission or processing latency, making it difficult to ensure that receivers can receive data reliably and with low latency.

[0004] RDMA, as an efficient network communication technology, supports multiple network protocols, including InfiniBand, iWARP, and RoCE (RDMA over Converged Ethernet). The RoCE protocol runs on existing Ethernet infrastructure, avoiding network stack processing overhead, significantly reducing network communication latency, and alleviating CPU load. It is ideally suited for efficient data transmission in market systems and production environments. RoCE transmission methods primarily include unreliable datagram (UD) and reliable connection (RC) transmission modes. UD mode supports multicast and offers high transmission efficiency and throughput, but has lower reliability and the risk of packet loss or out-of-order transmission. It also does not support unilateral RDMA read and write operations, making it unsuitable for low-latency applications with high security requirements. RC mode offers higher transmission reliability, similar to a TCP connection, and provides efficient data transmission and low latency. It is typically used for point-to-point communication.

[0005] In order to meet the market system's demand for high-performance network communication, while balancing transmission efficiency and reliability, RDMA technology is used to implement low-latency multicast functionality in many-to-many forwarding scenarios, avoid problems such as packet loss, errors, duplication, and disorder, and meet the localization requirements in actual production deployment. This is an issue worthy of in-depth study. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a reliable multicast transmission method for the RoCE protocol, which is used to implement reliable multicast transmission function through UDP control messages and RDMA one-to-one reliable connection (RC) mode, while supporting many-to-many concurrent forwarding scenarios, effectively utilizing the outstanding characteristics of RDMA, and meeting the low-latency reliable multicast scenario application of the market system.

[0007] In order to achieve the above purpose, a reliable multicast transmission method for RoCE protocol is designed, which includes: S1. Multicast group division: According to the actual market forwarding scenario, the multicast source and multicast group are divided into a hierarchical manner, and the nodes are divided into sending nodes, forwarding nodes and receiving nodes. The sending node, as the multicast source, is only responsible for producing and sending data; the forwarding node is responsible for receiving messages from the sending node or other forwarding nodes, and forwarding them to other forwarding nodes or receiving nodes; the receiving node, as the end of the multicast group, only aggregates and receives the received data; S2. Control plane: The sending end sends control information in the user datagram protocol UDP, sends its own Internet protocol IP and port to the receiving end, and the receiving end listens to the control message, synchronizes the application layer routing table, and creates a control plane for building a multicast group. Establish a communication endpoint EP for communication connection, establish a connection through the application programming interface API of the communication manager CM in the remote direct memory access technology RDMA and complete the ready state transition of the work queue to QP; S3. Data plane: the sender and the receiver use the reliable connection mode RC mode for reliable data transmission, and the forwarding node uses the reactor multi-threaded mode Reactor multi-threaded mode for concurrent forwarding. The main thread polls the reception completion queue CQ event, determines the forwarding target node according to the application layer routing table, copies the message content to the sending buffer and initiates an asynchronous operation to perform the send-receive bilateral operation SEND-RECEIVE operation and the read-write unilateral operation READ-WRITE operation; S4. Interface call: Develop C and JAVA API interfaces to implement the send-receive bilateral operation SEND-RECEIVE operation and the read-write unilateral operation READ-WRITE operation of C and JAVA programs in multicast scenarios, and set the number of data items to be sent or received or read or written, the data packet size and the number of transmission times through parameters.

[0008] Preferably, the method of the present invention also includes other technical features, wherein in the control plane step, the sending end includes a sending node and a forwarding node, the sending end uses the User Datagram Protocol UDP protocol as an envelope to send control information, and sends its own Internet Protocol IP and port to the receiving end, the receiving end includes a receiving node and a forwarding node, the receiving end monitors control messages, synchronizes the application layer routing table, and sequentially creates a communication endpoint EP for establishing a communication connection, establishes a connection processing handshake process based on the communication endpoint EP information and the application programming interface API, creates a work queue pair QP on the communication endpoint EP, associates the work queue pair QP with the communication endpoint through the application programming interface API, and completes the ready state transition of the work queue pair QP.

[0009] Preferably, the method of the present invention also includes other technical features, wherein in the data plane step, the sending end and the receiving end adopt the reliable connection mode RC mode to reliably transmit data, and when the forwarding node forwards data, it performs concurrent forwarding in the reactor multi-threaded mode Reactor multi-threaded mode, initializes an instance to monitor and distribute network events, registers events for each work queue QP, and the main thread polls the reception completion queue CQ event. After receiving the message, the forwarding target node is determined according to the application layer routing table, and a thread is allocated from the thread pool for each forwarding connection. The message content is copied to the sending buffer and an asynchronous operation is initiated. When performing a two-end send-receive bilateral operation SEND-RECEIVE operation, the sending end data is encapsulated into a data packet that can be transmitted on the link, and the receiving end parses the data. When performing a single-end read-write unilateral operation READ-WRITE operation, after obtaining the destination address through the send-receive bilateral operation SEND-RECEIVE operation, the remote memory can be read and written directly without the need for the remote CPU to participate in sensing the data being read and written in the memory.

[0010] Preferably, the method of the present invention also includes other technical features, wherein in the interface calling step, C and JAVA API interface development are respectively performed to implement a set of data transmission solutions, supporting C and JAVA programs sending and receiving send-receive bilateral operations SEND-RECEIVE operations and read-write unilateral operations READ-WRITE operations in multicast scenarios, and setting the number of data items sent or received or read or written, data packet size and number of transmission times through parameters.

[0011] Compared with the prior art, the present invention has the following advantages:

[0012] Compared with the prior art, the advantages of the present invention are embodied in the following aspects:

[0013] 1) It effectively integrates the UDP transmission protocol in the traditional communication mode and the reliable transmission mode RC in RDMA technology, which not only meets the demand for multicast functions in the production environment, but also ensures the integrity and sequence of data transmission, achieving low-latency network communication while ensuring the reliability of data transmission.

[0014] 2) Supports multi-threaded concurrent processing in forwarding scenarios, reducing the delay caused by simulating many-to-many connections, improving the system's ability to handle high-concurrency requests, and optimizing resource utilization efficiency.

[0015] 3) It provides C and JAVA language calling interfaces, making it easier for different users to use or integrate this technology, thus enhancing the universality and ease of use of the technology.

[0016] 4) Adapting to domestic hardware equipment, developed based on Ethernet, and supporting the RoCE v2 protocol, the implementation solution has certain scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 , is a flow chart of this method;

[0018] Figure 2 , is a schematic diagram of multicast group division;

[0019] Figure 3 , is the application layer routing table diagram;

[0020] Figure 4 , is the sending node workflow diagram;

[0021] Figure 5 , is the forwarding node workflow diagram;

[0022] Figure 6 , is the accepting node workflow diagram;

[0023] Figure 7 , is a timing diagram;

[0024] Figure 8 , is a diagram of data message flow. DETAILED DESCRIPTION

[0025] In order to make the purpose, principle and structure of the present invention more clear, the following is further described in conjunction with the accompanying drawings and specific embodiments. The following embodiments take the use of the method of the present invention in a market information system under a production environment as an example to illustrate the present invention.

[0026] Figure 1 This application embodiment is based on a domestic server and operating system and uses the method of the present invention to achieve reliable multicast.

[0027] It is worth mentioning that:

[0028] UDP protocol refers to User Datagram Protocol, which is a connectionless, simple transport layer protocol that provides services based on the IP protocol.

[0029] Reactor multi-threaded mode, also known as reactor multi-threaded mode, is an event-driven model in network programming.

[0030] Based on the current status of the market information system, this paper provides a reliable multicast transmission method for the RoCE protocol. It combines the UDP protocol multicast function and the RDMA RC reliable transmission characteristics at the application layer, supports low-latency concurrent processing in many-to-many forwarding scenarios, provides C and JAVA interface calls, and meets the deployment environment requirements based on domestic hardware equipment. The method is specifically as follows:

[0031] S1. Multicast Group Division. Based on actual market forwarding scenarios, multicast sources and multicast groups are divided into hierarchical layers, with nodes divided into sending nodes, forwarding nodes, and receiving nodes. Specifically, sending nodes, as multicast sources, are solely responsible for generating and sending data; forwarding nodes are responsible for receiving messages from sending nodes or other forwarding nodes and forwarding them to other forwarding nodes or receiving nodes; and receiving nodes, as the end points of a multicast group, only aggregate and receive received data.

[0032] S2. Control Plane. Within the partitioned area, the sender (sending node / forwarding node) sends control information using the UDP protocol as an envelope, including its own IP address and port number to the receiver (receiving node / forwarding node). The receiver listens for control messages, synchronizes the application layer routing table, and sequentially creates communication endpoints (EPs) for establishing a communication connection. Based on the EP information and the RDMA CM API, a connection handshake is established. A work queue pair (QP) is created on the EP, and the QP is associated with the communication endpoint using the RDMA CM API. The QP then transitions to the ready state.

[0033] S3. Data plane. Based on the established connection, the sender and the receiver use the RC mode for reliable data transmission. When the forwarding node forwards data, it performs concurrent forwarding in the Reactor multi-threaded mode, initializes an instance to monitor and distribute network events, and registers events for each QP. The main thread polls to receive CQ events. After receiving the message, it determines the forwarding target node according to the application layer routing table, allocates a thread for each forwarding connection from the thread pool, copies the message content to the send buffer, and initiates an asynchronous operation. When performing a two-way send-receive operation SEND-RECEIVE, the sender encapsulates the data into a data packet that can be transmitted on the link, and the receiver parses the data. When performing a single-end read-write operation READ-WRITE, after obtaining the destination address through the SEND-RECEIVE operation, the remote memory can be read and written directly without the need for the other end CPU to participate in perceiving the data being read and written in the memory.

[0034] S4. Interface Calling. Develop C and JAVA APIs to implement a data transmission solution. Support bilateral send-receive operations, unilateral read and write operations, and parameterize the number of data items sent / received / read / written, packet size, and number of transmissions for both C and JAVA programs in multicast scenarios.

[0035] Specific embodiments are described below.

[0036] Example 1: Multicast Group Division

[0037] The production scenario structure of the simulated market system is as follows: Figure 2 The diagram shows a hierarchical multicast structure, with sending, forwarding, and receiving nodes configured based on different component types. Sending node S1 is responsible for generating and sending market information messages; forwarding node F1 is responsible for receiving messages from the upper-level node and forwarding them to the lower-level nodes; receiving nodes R1, R2, R3, and R4 are similar to leaf nodes in a tree structure and are only responsible for receiving market data.

[0038] Every two layers of nodes form a partitioned area. Each area contains a multicast source and a multicast group. The multicast groups in each area are assigned different IP addresses. Forwarding nodes act as both senders and receivers in different partitioned areas, acting as receivers in the multicast groups of the upper-layer area and as senders in the multicast sources of the lower-layer area. Specifically, in partitioned area 1, sending node S1 is the multicast source, and second-layer nodes R1, R2, and F1 form a multicast group. In partitioned area 2, forwarding node F1 is the multicast source, and second-layer nodes R3 and R4 form a multicast group. This same approach applies to other partitioned areas as the layers deepen.

[0039] Example 2: Control message sending and receiving and connection establishment

[0040] Based on Ethernet and the above multicast structure, data sending and receiving operations are implemented. Socket communication is used to send control messages and establish connections based on the RDMA CM function library. In the divided area 1, the sending node S1 uses the UDP protocol as an envelope, and the main thread encapsulates the control information. The control information includes S1's IP address IP1 and port information Port1. The forwarding node F1 and the receiving nodes R1 and R2 listen to the UDP message and parse the message content after receiving it. In the divided area 2, the forwarding node F1 acts as the sender to send the control UDP message, including F1's IP address IP2 and port information Port2. The members of multicast group 2 listen to and parse the message. Figure 3 An application layer routing table is created simultaneously for subsequent data message transmission between nodes. The table entries include: multicast source, IP address and communication port of nodes in the multicast group.

[0041] Within region 1, receivers R1, R2, and F1 resolve address information and send connection requests. Transmitter S1, based on RC, monitors the receiver's connection requests and, in the order in which the requests are processed, creates communication endpoints (EPs) for S1, R1, R2, and F1. For each connection pair, a pair of EPs must be established. The RDMA CM library exchanges the MR addresses of the two end points' memories to handle the connection handshake, creates a work queue pair (QP) on the EP, and sets the QP ready state transition. Similarly, within region 2, forwarding node F1 must establish EPs corresponding to different connections based on the connection status, thus implementing a multicast forwarding simulation scenario.

[0042] Example 3: C-interface data plane message

[0043] For the data plane dual-end send-receive operation SEND-RECEIVE, data is sent, forwarded and received through RC transmission based on the established connection. The working processes of the sending node, forwarding node and receiving node are as follows: Figure 4 、 5 ,6. The sending node starts multi-threading to send control messages and data messages respectively, the forwarding node starts multi-threading to complete data sending and receiving concurrently, and the receiving node receives data. The timing relationship is shown in Figure 7 shown.

[0044] Taking the SEND-RECEIVE operation as an example, after creating the QP, a send or receive event is registered for each QP. The data of the sender S1 in the divided area 1 is encapsulated into a data packet that can be transmitted on the link. After the transmission is based on the established connection, F1, as the receiving end, parses the data and stores the data in the specified memory area according to the work queue element WQE. It returns the ACK information and puts the completion information into the completion queue CQ. In particular, the forwarding node F1 performs concurrent forwarding in the Reactor multi-threaded mode. The main thread initializes an instance, which is responsible for listening to and distributing events. The instance cyclically listens for the reception completion CQ event, determines the forwarding target node information in combination with the application layer routing table, and allocates a working thread from the thread pool for each pair of data transmission on the forwarding connection. The forwarding function is called back in the receiving event, such as Figure 8 As shown, the message content is copied to the F1 forwarding EP and asynchronously sent to multicast group 2 members via SEND. During a single-ended READ / WRITE operation, after obtaining the destination address through a SEND-RECEIVE operation, remote memory READ / WRITE operations can be performed in the same manner, eliminating the need for the remote CPU to participate in the data being read or written to memory. This point-to-point simulated multicast mode and the RC's inherent reliable transmission mode enable reliable multicast communication.

[0045] Example 4: JAVA interface call

[0046] Encapsulates the developed C interface, adds JNI function declarations for each encapsulated function, defines the JAVA class and its native methods after compilation, further implements the JNI method, interacts with the C interface through JNI library loading and JAVA interface calls, implements JAVA interface SEND-RECEIVE and READ-WRITE operations in multicast scenarios, and supports setting parameters such as the number of sent / received / read / write data items, packet size, and number of sends.

[0047] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent replacement or change made by any technician familiar with the technical field within the technical scope disclosed by the present invention based on the technical solution and novel concept of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A reliable multicast transmission method for RoCE protocol, characterized in that: The method comprises S1. Multicast group division: According to the actual market forwarding scenario, multicast sources and multicast groups are divided in a hierarchical manner, and nodes are divided into sending nodes, forwarding nodes and receiving nodes. The sending node, as a multicast source, is only responsible for producing and sending data; the forwarding node is responsible for receiving messages from the sending node or other forwarding nodes and forwarding them to other forwarding nodes or receiving nodes; the receiving node, as the end of the multicast group, only aggregates and receives the received data; S2. Control plane: The sender sends control information in User Datagram Protocol (UDP), sends its own Internet Protocol (IP) and port to the receiver, and the receiver listens to the control information, synchronizes the application layer routing table, creates a communication endpoint (EP) for establishing a communication connection, establishes a connection through the application programming interface (API) of the communication manager (CM) in the remote direct memory access (RDMA) technology, and completes the ready state transition of the work queue to the QP. S3. Data plane: The sender and the receiver use the reliable connection mode RC mode for reliable data transmission. The forwarding node uses the reactor multi-threaded mode for concurrent forwarding. The main thread polls the receive completion queue CQ event, determines the forwarding target node according to the application layer routing table, copies the message content to the send buffer and initiates asynchronous operations to perform send-receive bilateral operations and read-write unilateral operations. S4. Interface call: Develop C and JAVA API interfaces to implement send-receive bilateral operations and read-write unilateral operations of C and JAVA programs in multicast scenarios, and set the number of data items sent or received or read or written, data packet size, and number of transmission times through parameters.

2. A reliable multicast transmission method for RoCE protocol as claimed in claim 1, characterized in that: In the control plane step, the sending end includes a sending node and a forwarding node. The sending end uses the User Datagram Protocol UDP protocol as an envelope to send control information and sends its own Internet Protocol IP and port to the receiving end. The receiving end includes a receiving node and a forwarding node. The receiving end monitors the control information, synchronizes the application layer routing table, and sequentially creates a communication endpoint EP for establishing a communication connection. A connection processing handshake process is established based on the communication endpoint EP information and the application programming interface API, a work queue pair QP is created on the communication endpoint EP, the work queue pair QP is associated with the communication endpoint through the application programming interface API, and the ready state transition of the work queue pair QP is completed.

3. A reliable multicast transmission method for RoCE protocol as claimed in claim 1, characterized in that: In the data plane step, the sending end and the receiving end use the reliable connection mode RC mode to reliably transmit data. When the forwarding node forwards data, it performs concurrent forwarding in the reactor multi-threaded mode, initializes an instance responsible for listening to and distributing network events, registers events for each work queue QP, and the main thread polls the reception completion queue CQ event. After receiving the message, the forwarding target node is determined according to the application layer routing table, and a thread is allocated from the thread pool for each forwarding connection. The message content is copied to the sending buffer and an asynchronous operation is initiated. When performing a two-end send-receive bilateral operation, the sending end data is encapsulated into a data packet that can be transmitted on the link, and the receiving end parses the data. When performing a single-end read-write unilateral operation, after obtaining the destination address through the send-receive bilateral operation, the remote memory can be read and written directly, without the need for the remote CPU to participate in sensing the data being read and written in the memory.

4. A reliable multicast transmission method for RoCE protocol as claimed in claim 1, characterized in that: In the interface calling step, C and JAVA API interface development are performed respectively to implement a set of data transmission solutions, supporting C and JAVA programs to send and receive send-receive bilateral operations, read-write unilateral operations of unilateral read and unilateral write operations in multicast scenarios, and setting the number of data items sent or received or read or written, data packet size and number of transmission times through parameters.

Images