RDMA multicast processing system in RoCE network and implementation method

Abstract

The invention belongs to the technical field of computer networks, and particularly relates to an RDMA (Remote Direct Memory Access) multicast processing system and an implementation method in an RoCE (Role of Compact Element) network, which are characterized in that the multicast technology of an Ethernet on a link layer and a network layer is utilized, and an Ethernet multicast forwarding function of an Ethernet switch is used on a switching side; multicast processing functions are respectively designed on a network card and a protocol stack on a network end side, and a multicast function of the RoCE network is realized by combining an RDMA message of a transmission layer; according to the invention, the multicast technology of the Ethernet is utilized in the link layer, no special requirement is required for the switch, and the RDMA multicast function of the RoCE network can be realized by utilizing the common Ethernet switch for networking.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer networks, and in particular relates to a RDMA (Remote Direct Memory Access) multicast processing system in a RoCE (RDMA over Coverage Ethernet) network and an implementation method thereof. Background Art

[0002] RoCE networks have been widely deployed and used in data center networks for application scenarios such as AI and supercomputing that have high requirements for network latency and CPU utilization due to their low end-to-end latency, low CPU resource consumption, and low deployment cost.

[0003] RoCE is an RDMA network implemented on the basis of Ethernet IP / UDP protocol. Its principle is to use InfiniBand (the main implementation form of RDMA network, the protocol specification is InfiniBand TM The transport layer message of RoCE (RoCE Architecture Specification Volume 1 Release 1.4) is encapsulated into the Payload of UDP message, and Infiniband message is transmitted through Ethernet IP / UDP protocol, and then Infiniband is used to implement RDMA network in the upper layer protocol (see RoCE protocol specification Supplement to InfiniBandTM Architecture Specification Volume 1 Release 1.2.1 Annex A17: RoCEv2).

[0004] Multicast is a one-to-many communication mode. With multicast technology, an application can send only one message to multiple destination nodes. Multicast technology can greatly improve the communication efficiency of a group of terminal nodes and is widely used in many scenarios, especially cloud computing and data centers.

[0005] Infiniband protocol supports multicast technology. However, Infiniband multicast uses the Infiniband protocol link layer header information (Local Routing Header, LRH), and there is no Infiniband protocol link layer information in the RoCE network. Therefore, the multicast technology designed by the Infiniband protocol cannot be used in the RoCE network. The RoCE network specification does not support multicast either.

[0006] In the RoCE network, RDMA packets are encapsulated on Ethernet. The link layer and network layer of the RoCE network both use Ethernet, and only the transport layer and above use the RDMA network. Therefore, multicast in the RoCE network can be achieved with the help of Ethernet multicast features.

[0007] The present invention utilizes the Ethernet multicast characteristic, combines the addressing technology of the Infiniband protocol, is compatible with the Ethernet switch on the switching side, utilizes the Ethernet multicast technology to complete multicast forwarding, and designs the RoCE network multicast processing system on the end system side to realize the multicast function of the RDMA message in the RoCE network. Summary of the invention

[0008] The technical problem to be solved by the present invention is how to use the Ethernet multicast feature to realize the multicast function of RDMA in the RoCE network. The present invention uses the Ethernet multicast technology at the link layer and the network layer, uses the Ethernet multicast forwarding function of the Ethernet switch on the switching side, designs the multicast processing function on the network card and the protocol stack on the network end side, and combines the RDMA message of the transport layer to realize the multicast function of the RoCE network.

[0009] The technical solution adopted by the present invention is an RDMA multicast processing system in a RoCE network, which runs on a RoCE network end system and is deployed together with a RoCE network card and a driver.

[0010] The RDMA multicast processing system consists of a multicast group configuration file, an initialization module, a multicast group management module, a multicast group address calculation module, a multicast group data buffer, a multicast forwarding module, a multicast message parsing module, a sending data acquisition module, a receiving data uploading module and an external message receiving module. The multicast group configuration file is a file of the RoCE network card driver, which is used to store the maximum number of multicast groups N, the minimum value MIN_ID of the multicast group ID, the maximum value MAX_ID, the member QP (Queue Pair, its meaning is shown in Supplement to InfiniBandTM ArchitectureSpecification Volume 1Release 1.2.1Annex A17:RoCEv2) supported by each multicast group, and the maximum number of QPs QN; the initialization module, multicast group management module, and multicast group address calculation module are software-implemented modules, which are deployed as part of the RoCE network card driver; the multicast group data buffer exists inside the RoCE network card as network card memory; the multicast forwarding module, multicast message parsing module, sending data acquisition module, receiving data uploading module, and external message receiving module are hardware-implemented functional modules and are deployed inside the RoCE network card.

[0011] The maximum number of multicast groups supported by the RoCE network card stored in the multicast group configuration file is N, where N is a positive integer and N is less than or equal to 8,388,607, i.e., 0X7FFFFF. The minimum value of the multicast group ID is MIN_ID, where MIN_ID is a positive integer and MIN_ID is less than or equal to 8,388,607. The maximum value of the multicast group ID is MAX_ID, where MAX_ID is a positive integer, MAX_ID is greater than or equal to MIN_ID, and MAX_ID is less than MIN_ID + N. The maximum number of member QPs supported by each multicast group is QN.

[0012] When loading the RoCE network card driver, the initialization module reads the maximum number of multicast groups N, the minimum value of the multicast group ID MIN_ID, the maximum value of the multicast group ID MAX_ID, and the maximum number of member QPs supported by each multicast group QN from the multicast group configuration file, and writes the read values of N, MIN_ID, MAX_ID, and QN into the multicast group data buffer respectively.

[0013] The multicast group data buffer exists in the RoCE network card hardware and stores the maximum number of multicast groups N, the minimum value of the multicast group ID MIN_ID, the maximum value of the multicast group ID MAX_ID, the maximum number of member QPs supported by each multicast group QN, the current number of multicast groups cn, and an array of N possible multicast group structures mcast_array[i], where i represents the i-th multicast group structure data, 0 ≤ i < N; each multicast group structure in the array contains 5 elements, which are:

[0014] 1) The multicast ID of the multicast group, mid;

[0015] 2) The Ethernet multicast IP address of the multicast group, mip;

[0016] 3) The Ethernet multicast MAC address of the multicast group, mmac;

[0017] 4) The current number of valid QPs in the multicast group, qn;

[0018] 5) An array recording the valid QPs in the multicast group, qp[j], where qp[j] represents the QP number of the j-th QP, 0 ≤ j < qn.

[0019] During system operation, the multicast group management module obtains the qpn for multicast management operations, the operation type opc (join or leave), and the multicast ID of the multicast group to be operated on from the application program, calls the multicast group address calculation module to obtain the corresponding Ethernet multicast IP address and Ethernet multicast MAC address of the multicast group, and updates them to the multicast group data buffer according to the operation type.

[0020] The multicast group address calculation module is used to receive the call of the multicast group management module, calculate the multicast group Ethernet multicast IP address and the multicast group Ethernet multicast MAC address according to the multicast group ID received from the multicast group management module, and return them to the multicast group management module.

[0021] The multicast forwarding module is used to obtain the message data and address data to be sent from the sending data acquisition module, and judge whether the message is a multicast message and whether there are multicast members of the multicast group to which the message belongs in the RoCE network card according to the address data and the multicast group structure array. If yes, the message is copied to each multicast member and sent to the receiving data sending module, and then the message data and address data are sent to the receiving data sending module. At the same time, the multicast forwarding module receives the message data and address data received from the outside from the multicast message parsing module, and judges whether the message is a multicast message and whether there are multicast members of the multicast group to which the multicast message belongs in the RoCE network card according to the multicast group structure array. If yes, the message is copied to each multicast member and sent to the receiving data sending module. If not, the message is discarded.

[0022] The multicast message parsing module is used to receive the external input RoCE message from the external message receiving and processing module, obtain the destination MAC address from it, and determine whether it is a multicast message according to the multicast group structure array. If it is a multicast message, the RoCE message is sent to the multicast forwarding module. If it is not a multicast message, the message is discarded.

[0023] The sending data acquisition module is used to acquire the data information to be sent when the application sends data, and send the sending data information to the multicast forwarding module.

[0024] The received data sending module is used to receive the multicast data forwarded locally from the multicast forwarding module, and send the data to each QP under the multicast group.

[0025] The external message receiving module is used to receive RoCE messages from the network and send the RoCE messages to the multicast message parsing module.

[0026] The present invention also proposes a method for implementing RDMA multicast in a RoCE network based on the above system, comprising the following steps:

[0027] The first step is to build the RDMA multicast processing system in the RoCE network as described above.

[0028] The second step is to initialize the multicast group configuration file.

[0029] Write the number of multicast groups N supported by the RoCE network card, the minimum multicast ID MIN_ID, the maximum multicast ID MAX_ID, and the maximum number of member QPs QN supported by each multicast group into the multicast group configuration file.

[0030] In the third step, the initialization module initializes the RoCE network multicast processing system:

[0031] 3.1 The initialization module reads N, MIN_ID, MAX_ID, and QN from the multicast group configuration file.

[0032] 3.2 Send N, MIN_ID, MAX_ID, QN to the multicast group data buffer.

[0033] In the fourth step, the RoCE network multicast processing system realizes the multicast function of RoCE messages by processing the multicast joining and leaving of the application and the messages to be sent and received by the RoCE network card. This step is completed in parallel by the multicast group management module, multicast address calculation module, multicast forwarding module, multicast message parsing module, sending data acquisition module, receiving data uploading module and external message receiving module; the specific steps are as follows:

[0034] 4.1 The multicast address calculation module receives the call of the multicast group management module, receives the multicast group ID input by the multicast group management module, calculates the multicast MAC address according to the Ethernet multicast protocol, and returns it to the multicast group management module. The specific steps are as follows:

[0035] 4.1.1 The multicast address calculation module monitors whether it is called by the multicast group management module. If it is called, execute 4.1.2. If it is not called, go to 4.1.1 and continue monitoring.

[0036] 4.1.2 Input the multicast ID mi_id of the multicast group from the multicast group management module.

[0037] 4.1.3 Calculation:

[0038] mac0 = m_id & 0x00000000ff;

[0039] mac1=(m_id>>8)&0X000000FF;

[0040] mac2 = (m_id>>16) & 0X00007F;

[0041] 4.1.4 According to the link layer Ethernet multicast protocol, construct the multicast MAC address m_mac:

[0042] m_mac=01:00:5e:mac2:mac1:mac0.

[0043] 4.1.5 Calculation:

[0044] ip0 = m_id & 0x000000ff;

[0045] ip1 = (m_id >> 8) & 0X0000FF;

[0046] ip2 = (m_id>>16) & 0x00FF;

[0047] ip3 = (m_id>>24) & 0X00E0;

[0048] 4.1.6 According to the link layer Ethernet multicast protocol, construct the multicast IP address m_ip:

[0049] m_ip = ip3.ip2.ip1.ip0;

[0050] 4.1.7 returns the multicast MAC address m_mac and the multicast IP address m_ip to the multicast group management module, and goes to 4.1.1 to continue monitoring.

[0051] 4.2 The multicast group management module receives the application multicast group management request, processes the multicast group management request, and updates the multicast group data buffer. The specific steps are:

[0052] 4.2.1 The multicast group management module reads N, MIN_ID, MAX_ID, QN from the multicast group configuration file; reads the current multicast group number cn and the multicast group structure array mcast_array[i] from the multicast group data buffer.

[0053] 4.2.2 The multicast group management module monitors whether it is called by the application. If it is called, execute 4.2.3. If it is not called, go to 4.2.2 and continue to monitor.

[0054] 4.2.3 The multicast group management module receives a multicast group management command from the application, the multicast group management command being "join" or "leave", as well as the QP number m_qpn of the operation and the multicast ID m_id of the operation. If cn≥N or m_id<MIN_ID or m_id>MAX_ID, go to step 4.2.2 to continue monitoring, otherwise go to step 4.2.4.

[0055] 4.2.4 Call the multicast group address calculation module to obtain the Ethernet multicast MAC address m_mac and multicast IP address m_ip.

[0056] 4.2.5 Determine whether there is an element in the multicast group structure array mcast_array[i] whose mid field value is equal to m_id. If not and the multicast group management command is "join", go to 4.2.6; if not and the multicast group management command is "leave", go to step 4.2.2 to continue monitoring; if so, let the element be the m_index element in the multicast group structure array, if the multicast group management command is "join", go to step 4.2.7; if the multicast group management command is "leave", go to step 4.2.8.

[0057] 4.2.6 Add a new multicast group with multicast ID m_id, multicast MAC address m_mac and multicast IP address m_ip to the multicast group structure array, and add a member with QP number m_qpn to the multicast group; go to 4.2.9.

[0058] 4.2.7 Add a member with QP number m_qpn to the multicast group with multicast ID m_id; go to 4.2.9.

[0059] 4.2.8 In the multicast group with multicast ID m_id, delete the member with member qpn m_qpn; go to 4.2.9.

[0060] 4.2.9 Update the updated cn and mcast_array[i] to the multicast group data buffer. Go to 4.2.2 to continue monitoring.

[0061] 4.3 The multicast forwarding module receives the message to be sent to the network from the sending data acquisition module, determines whether it is a multicast message and whether there is a QP belonging to the destination multicast group locally. If yes, the message is forwarded locally, otherwise the message is discarded. At the same time, the external message receiving module receives the RoCE message received from the network, determines whether it is a multicast message and whether there is a QP belonging to the destination multicast group locally. If yes, the message is forwarded locally, otherwise the message is discarded. The specific steps are as follows:

[0062] 4.3.1 The multicast forwarding module monitors whether there is data from the sending data acquisition module. If so, go to 4.3.3. If not, go to 4.3.2 to continue monitoring.

[0063] 4.3.2 The multicast forwarding module monitors whether there is data from the multicast message parsing module. If so, execute 4.3.4. If not, go to 4.3.1 to continue monitoring.

[0064] 4.3.3 Multicast the sent data locally by:

[0065] 4.3.3.1 The multicast forwarding module receives the data to be sent out_data and the destination MAC address dst_mac from the sending data acquisition module.

[0066] 4.3.3.2 The multicast forwarding module determines whether dst_mac is an Ethernet multicast address according to the Ethernet multicast protocol. The method is:

[0067] If the following conditions are met at the same time:

[0068] 1) dst_mac[5] = 0x01;

[0069] 2) dst_mac[4] = 0x00;

[0070] 3) dst_mac[3] = 0x5E;

[0071] 4)dst_mac[2]≤0X7F;

[0072] If dst_mac is an Ethernet multicast address, go to step 4.3.3.3; otherwise, if dst_mac is not an Ethernet multicast address, go to step 4.3.1 to continue monitoring.

[0073] 4.3.3.3 Calculate the multicast ID dst_mid of the multicast group based on the Ethernet multicast protocol and dst_mac:

[0074] dst_mid=dst_mac[2]<<16+dst_mac[1]<<8+dst_mac[0].

[0075] 4.3.3.4 Read the multicast group number N and N elements of the multicast group structure array mcast_array[i] from the multicast group data buffer.

[0076] 4.3.3.5 Determine whether there is an element in the multicast group structure array mcast_array[i] with a mid field value equal to dst_mid. If so, let the element be mcast1 and go to step 4.3.3.6; if not, go to step 4.3.1 to continue monitoring.

[0077] 4.3.3.6 Copy the data out_data to be sent to the qn field value of mcast1, and send it to the receiving data sending module with each element in the qp[j] array field of mcast1 (the qp[j] array has qn elements, and each element value is 1 QP number). Go to 4.3.1 to continue monitoring.

[0078] 4.3.4 Multicast the received data locally by:

[0079] 4.3.4.1 Receive the message data in_data and the received message multicast group ID number in_mid from the multicast message parsing module.

[0080] 4.3.4.2 Read the multicast group number N and N elements of the multicast group structure array mcast_array[i] from the multicast group data buffer.

[0081] 4.3.4.3 Determine whether there is an element in the multicast group structure array mcast_array[i] with a mid field value equal to in_mid. If so, let the element be mcast2 and go to step 4.3.4.4; if not, go to step 4.3.1 to continue monitoring.

[0082] 4.3.4.4 Copy the message data in_data to the qn field value of mcast2, and send it to the receiving data sending module together with each element in the qp[j] array field of mcast2. Go to 4.3.1 to continue monitoring.

[0083] 4.4 The multicast message parsing module receives the RoCE message from the external message receiving module, determines whether it is a multicast message, and if so, calculates the multicast MAC address and multicast ID, and sends the multicast MAC address and multicast ID as well as the RoCE message to the multicast forwarding module. The specific steps are as follows:

[0084] 4.4.1 The multicast message parsing module monitors whether there is a RoCE network message arriving. If yes, execute 4.4.2; if no, go to 4.4.1 to continue monitoring.

[0085] 4.4.2 Receive RoCE network messages.

[0086] 4.4.3 According to the RoCE network message format, extract the data pkt_data and the destination MAC address pkt_mac from the RoCE network message.

[0087] 4.4.4 Determine whether pkt_mac is an Ethernet multicast address based on the Ethernet link layer multicast protocol. The method is:

[0088] If the following conditions are met at the same time:

[0089] 1) pkt_mac[5] = 0x01;

[0090] 2) pkt_mac[4] = 0x00;

[0091] 3) pkt_mac[3] = 0x5E;

[0092] 4)pkt_mac[2]≤0X7F;

[0093] If pkt_mac is an Ethernet multicast address, go to step 4.4.5; otherwise, if pkt_mac is not an Ethernet multicast address, go to step 4.4.1 to continue monitoring.

[0094] 4.4.5 Calculate the multicast ID pkt_mid based on the Ethernet multicast protocol and pkt_mac:

[0095] pkt_mid=pkt_mac[2]<<16+pkt_mac[1]<<8+pkt_mac[0].

[0096] 4.4.6 Send pkt_data and pkt_mid to the multicast forwarding module. Go to step 4.4.1 to continue monitoring.

[0097] 4.5 Sending data The acquisition module monitors whether the application sends data. When the application sends data, it obtains the information of the data to be sent and sends the data to be sent and the destination MAC address to the multicast forwarding module.

[0098] 4.6 The receiving data sending module monitors the multicast forwarding module. When the multicast forwarding module wants to forward a multicast message, it receives the multicast message data and the QP number from the multicast forwarding module, and sends the multicast message data to the QP with the specified QP number according to the RoCE protocol.

[0099] 4.7 The external message receiving module monitors the network card input terminal, receives the RoCE message when a RoCE message arrives on the network, and sends the RoCE message to the multicast message parsing module.

[0100] The following technical effects can be achieved by adopting the present invention:

[0101] 1) The present invention constructs a RoCE network multicast processing system, utilizes Ethernet multicast characteristics and Infiniband addressing technology, implements multicast group management, multicast address calculation and multicast forwarding processing of RDMA messages in the RoCE network on the end system side, and realizes the RDMA multicast function on the end system side.

[0102] 2) The multicast group data structure designed in the first step of the present invention and the multicast group address calculation module designed in step 4.1 associate the multicast group ID number of RDMA with the Ethernet multicast address according to the Ethernet multicast protocol, so that the multicast function of the ordinary Ethernet switch can be used on the switching side to assist in realizing the multicast function of the RoCE network.

[0103] 3) In step 4.3 of the present invention, a multicast forwarding module is designed, which utilizes the existing RoCE processing logic in the RoCE network card to implement multicast forwarding of RoCE messages forwarded by Ethernet multicast to the QP of the RDMA application, thereby completing the RDMA multicast forwarding function on the end system.

[0104] 4) The present invention utilizes Ethernet multicast technology at the link layer, has no special requirements on switches, and can utilize common Ethernet switches to form a network and realize the RDMA multicast function of the RoCE network. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] Figure 1 It is a structural composition diagram of an RDMA multicast processing system in a RoCE network according to the present invention;

[0106] Figure 2 It is an overall flow chart of the method of the present invention. DETAILED DESCRIPTION

[0107] The present invention will be further described below in conjunction with the accompanying drawings.

[0108] Figure 1 The present invention is a structural composition diagram of an RDMA multicast processing system in a RoCE network; the RDMA multicast processing system consists of a multicast group configuration file, an initialization module, a multicast group management module, a multicast group address calculation module, a multicast group data buffer, a multicast forwarding module, a multicast message parsing module, a sending data acquisition module, a receiving data sending module and an external message receiving module. The multicast group configuration file is a file of the RoCE network card driver, which is used to store the maximum number of multicast groups N, the minimum value MIN_ID of the multicast group ID, the maximum value MAX_ID, the member QP (QueuePair, its meaning is shown in Supplement to InfiniBandTM Architecture Specification Volume1Release 1.2.1Annex A17:RoCEv2) supported by each multicast group, and the maximum number of QPs QN; the initialization module, multicast group management module, and multicast group address calculation module are software-implemented modules, which are deployed as part of the RoCE network card driver; the multicast group data buffer exists inside the RoCE network card as network card memory; the multicast forwarding module, multicast message parsing module, sending data acquisition module, receiving data uploading module, and external message receiving module are hardware-implemented functional modules and are deployed inside the RoCE network card.

[0109] The maximum number of multicast groups supported by the RoCE network card stored in the multicast group configuration file is N, where N is a positive integer and N is less than or equal to 8388607, that is, 0X7FFFFF. The minimum value of the multicast group ID is MIN_ID, where MIN_ID is a positive integer and MIN_ID is less than or equal to 8388607. The maximum value of the multicast group ID is MAX_ID, where MAX_ID is a positive integer, MAX_ID is greater than or equal to MIN_ID, and MAX_ID is less than MIN_ID + N. The maximum number of member QPs supported by each multicast group is QN.

[0110] When loading the RoCE network card driver, the initialization module reads the maximum number of multicast groups N, the minimum value of the multicast group ID MIN_ID, the maximum value of the multicast group ID MAX_ID, and the maximum number of member QPs supported by each multicast group QN from the multicast group configuration file, and writes the read values of N, MIN_ID, MAX_ID, and QN into the multicast group data buffer respectively.

[0111] The multicast group data buffer exists in the RoCE network card hardware and stores the maximum number of multicast groups N, the minimum value of the multicast group ID MIN_ID, the maximum value of the multicast group ID MAX_ID, the maximum number of member QPs supported by each multicast group QN, the current number of multicast groups cn, and an array of N possible multicast group structures mcast_array[i], where i represents the i-th multicast group structure data, 0 ≤ i < N; each multicast group structure in the array contains 5 elements, which are:

[0112] 1) The multicast ID of the multicast group, mid;

[0113] 2) The Ethernet multicast IP address of the multicast group, mip;

[0114] 3) The Ethernet multicast MAC address of the multicast group, mmac;

[0115] 4) The current number of valid QPs in the multicast group, qn;

[0116] 5) An array recording the valid QPs in the multicast group, qp[j], where qp[j] represents the QP number of the j-th QP, 0 ≤ j < qn.

[0117] During system operation, the multicast group management module obtains the qpn for multicast management operations, the operation type opc (join or leave), and the multicast ID of the multicast group to be operated on from the application program, calls the multicast group address calculation module to obtain the corresponding Ethernet multicast IP address and Ethernet multicast MAC address of the multicast group, and updates them to the multicast group data buffer according to the operation type.

[0118] The multicast group address calculation module is used to receive the call of the multicast group management module, calculate the multicast group Ethernet multicast IP address and the multicast group Ethernet multicast MAC address according to the multicast group ID received from the multicast group management module, and return them to the multicast group management module.

[0119] The multicast forwarding module is used to obtain the message data and address data to be sent from the sending data acquisition module, and judge whether the message is a multicast message and whether there are multicast members of the multicast group to which the message belongs in the RoCE network card according to the address data and the multicast group structure array. If yes, the message is copied to each multicast member and sent to the receiving data sending module, and then the message data and address data are sent to the receiving data sending module. At the same time, the multicast forwarding module receives the message data and address data received from the outside from the multicast message parsing module, and judges whether the message is a multicast message and whether there are multicast members of the multicast group to which the multicast message belongs in the RoCE network card according to the multicast group structure array. If yes, the message is copied to each multicast member and sent to the receiving data sending module. If not, the message is discarded.

[0120] The multicast message parsing module is used to receive the external input RoCE message from the external message receiving and processing module, obtain the destination MAC address from it, and determine whether it is a multicast message according to the multicast group structure array. If it is a multicast message, the RoCE message is sent to the multicast forwarding module. If it is not a multicast message, the message is discarded.

[0121] The sending data acquisition module is used to acquire the data information to be sent when the application sends data, and send the sending data information to the multicast forwarding module.

[0122] The received data sending module is used to receive the multicast data forwarded locally from the multicast forwarding module, and send the data to each QP under the multicast group.

[0123] The external message receiving module is used to receive RoCE messages from the network and send the RoCE messages to the multicast message parsing module.

[0124] Figure 2 The present invention is a general flow chart of a multicast implementation method of the multicast processing system, which includes the following steps:

[0125] The first step is to build the RDMA multicast processing system in the RoCE network as described above.

[0126] The second step is to initialize the multicast group configuration file.

[0127] In the third step, the initialization module initializes the RoCE network multicast processing system by:

[0128] In the fourth step, the RoCE network multicast processing system realizes the multicast function of RoCE messages by processing the multicast joining and leaving of the application and the messages to be sent and received by the RoCE network card. This step is completed in parallel by the multicast group management module, multicast address calculation module, multicast forwarding module, multicast message parsing module, sending data acquisition module, receiving data uploading module and external message receiving module.

[0129] The present invention combines the Ethernet multicast protocol of the link layer and the Infiniband protocol of the transport layer to design the RDMA multicast protocol in the RoCE network. The multicast message format is designed to be compatible with the Ethernet multicast protocol at the link layer. From the perspective of the Ethernet device on the switching side, the multicast message is an Ethernet multicast message, which is assigned and forwarded using the Ethernet multicast technology; on the end system side, the multicast group structure and the multicast message processing flow are designed. The RDMA multicast in the RoCE network is realized through the cooperation between the end system side and the switching side.

Claims

1. A RDMA multicast processing system in a RoCE network, the system runs on a RoCE network end system and is deployed together with a RoCE network card and a driver; characterized in that: The RDMA multicast processing system consists of a multicast group configuration file, an initialization module, a multicast group management module, a multicast group address calculation module, a multicast group data buffer, a multicast forwarding module, a multicast packet parsing module, a transmitted data acquisition module, a received data forwarding module, and an external packet reception module. The multicast group configuration file is a file of the RoCE network card driver, used to store the maximum number N of multicast groups, the minimum value MIN_ID of the multicast group ID, the maximum value MAX_ID, the maximum number QN of QPs supported by each multicast group, and the maximum number QN of QPs. The initialization module, the multicast group management module, and the multicast group address calculation module are software-implemented modules and are deployed as part of the RoCE network card driver during deployment. The multicast group data buffer exists in the RoCE network card as network card memory. The multicast forwarding module, the multicast packet parsing module, the transmitted data acquisition module, the received data forwarding module, and the external packet reception module are hardware-implemented functional modules and are deployed inside the RoCE network card. The multicast group configuration file stores that the maximum number of multicast groups supported by the RoCE network card is N, where N is a positive integer and N is less than or equal to 8,388,607, i.e., 0X7FFFFF. The minimum value of the multicast group ID is MIN_ID, where MIN_ID is a positive integer and MIN_ID is less than or equal to 8,388,607. The maximum value of the multicast group ID is MAX_ID, where MAX_ID is a positive integer, MAX_ID is greater than or equal to MIN_ID, and MAX_ID is less than MIN_ID + N. The maximum number of QPs supported by each multicast group is QN. When loading the RoCE network card driver, the initialization module reads the maximum number N of multicast groups, the minimum value MIN_ID of the multicast group ID, the maximum value MAX_ID of the multicast group ID, and the maximum number QN of QPs supported by each multicast group from the multicast group configuration file, and writes the read values of N, MIN_ID, MAX_ID, and QN into the multicast group data buffer respectively. The multicast group data buffer exists in the RoCE network card hardware and stores the maximum number N of multicast groups, the minimum value MIN_ID of the multicast group ID, the maximum value MAX_ID of the multicast group ID, the maximum number QN of QPs supported by each multicast group, the current number cn of multicast groups, and an array mcast_array[i] of N possible multicast group structures, where i represents the i-th multicast group structure data and 0 ≤ i < N. Each multicast group structure in the array contains 5 elements, which are respectively: 1) The multicast ID, mid, of the multicast group. 2) The Ethernet multicast IP address, mip, of the multicast group. 3) The Ethernet multicast MAC address, mmac, of the multicast group. 4) The current number qn of valid QPs in the multicast group. 5) An array qp[j] recording the valid QPs in the multicast group, where qp[j] represents the QP number of the j-th QP and 0 ≤ j < qn. When the system is running, the multicast group management module obtains the qpn, operation type opc and multicast ID of the multicast group to be operated from the application, calls the multicast group address calculation module, obtains the Ethernet multicast IP address and Ethernet multicast MAC address corresponding to the multicast group, and updates them to the multicast group data buffer according to the operation type; The multicast group address calculation module is used to receive the call of the multicast group management module, calculate the multicast group Ethernet multicast IP address and the multicast group Ethernet multicast MAC address according to the multicast group ID received from the multicast group management module, and return them to the multicast group management module; The multicast forwarding module is used to obtain the message data and address data to be sent from the sending data acquisition module, and judge whether the message is a multicast message and whether there are multicast members of the multicast group to which the message belongs in the RoCE network card according to the address data and the multicast group structure array. If yes, the message is copied to each multicast member and sent to the receiving data sending module, and then the message data and address data are sent to the receiving data sending module; at the same time, the multicast forwarding module receives the message data and address data received from the outside from the multicast message parsing module, and judges whether the message is a multicast message and whether there are multicast members of the multicast group to which the multicast message belongs in the RoCE network card according to the multicast group structure array. If yes, the message is copied to each multicast member and sent to the receiving data sending module. If not, the message is discarded; The multicast message parsing module is used to receive the external input RoCE message from the external message receiving and processing module, obtain the destination MAC address from it, and determine whether it is a multicast message according to the multicast group structure array. If it is a multicast message, the RoCE message is sent to the multicast forwarding module. If it is not a multicast message, the message is discarded; The sending data acquisition module is used to obtain the data information to be sent when the application sends data, and send the sending data information to the multicast forwarding module; The received data sending module is used to receive the multicast data forwarded locally from the multicast forwarding module, and send the data to each QP under the multicast group; The external message receiving module is used to receive RoCE messages from the network and send the RoCE messages to the multicast message parsing module.

2. A method for implementing RDMA multicast in a RoCE network based on the system of claim 1, characterized in that: The method comprises the following steps: The first step is to construct an RDMA multicast processing system in a RoCE network as described in claim 1; The second step is to initialize the multicast group configuration file: Write the number of multicast groups N supported by the RoCE network card, the minimum multicast ID MIN_ID, the maximum multicast ID MAX_ID, and the maximum number of member QPs QN supported by each multicast group into the multicast group configuration file; In the third step, the initialization module initializes the RoCE network multicast processing system: 3.1 The initialization module reads N, MIN_ID, MAX_ID, and QN from the multicast group configuration file respectively; 3.2 Send N, MIN_ID, MAX_ID, QN to the multicast group data buffer; In the fourth step, the RoCE network multicast processing system realizes the multicast function of RoCE messages by processing the multicast joining and leaving of the application and the messages to be sent and received by the RoCE network card; this step is completed in parallel by the multicast group management module, the multicast address calculation module, the multicast forwarding module, the multicast message parsing module, the sending data acquisition module, the receiving data sending module and the external message receiving module; the specific steps are as follows: 4.1 The multicast address calculation module receives the call of the multicast group management module, receives the multicast group ID input by the multicast group management module, calculates the multicast MAC address according to the Ethernet multicast protocol, and returns it to the multicast group management module; 4.2 The multicast group management module receives the application multicast group management request, processes the multicast group management request, and updates the multicast group data buffer; 4.3 The multicast forwarding module receives the message to be sent to the network from the sending data acquisition module, determines whether it is a multicast message and whether there is a QP belonging to the destination multicast group locally. If yes, the message is forwarded locally, otherwise the message is discarded. At the same time, the external message receiving module receives the RoCE message received from the network, determines whether it is a multicast message and whether there is a QP belonging to the destination multicast group locally. If yes, the message is forwarded locally, otherwise the message is discarded. 4.4 The multicast message parsing module receives the RoCE message from the external message receiving module, determines whether it is a multicast message, and if so, calculates the multicast MAC address and multicast ID, and sends the multicast MAC address and multicast ID as well as the RoCE message to the multicast forwarding module; 4.5 The sending data acquisition module monitors whether the application sends data. When the application sends data, it obtains the information of the data to be sent, and sends the data to be sent and the destination MAC address to the multicast forwarding module; 4.6 The receiving data sending module monitors the multicast forwarding module. When the multicast forwarding module wants to forward a multicast message, it receives the multicast message data and QP number from the multicast forwarding module, and sends the multicast message data to the QP with the specified QP number according to the RoCE protocol; 4.7 The external message receiving module monitors the network card input terminal, receives the RoCE message when a RoCE message arrives on the network, and sends the RoCE message to the multicast message parsing module.

3. The method for implementing RDMA multicast in a RoCE network according to claim 2, wherein: The specific steps of 4.1 are: 4.1.1 The multicast address calculation module monitors whether it is called by the multicast group management module. If it is called, execute 4.1.

2. If it is not called, go to 4.1.1 and continue to monitor; 4.1.2 Input the multicast ID mi_id of the multicast group from the multicast group management module; 4.1.3 Calculation: mac0 = m_id & 0x00000000ff; mac1=(m_id>>8)&0X000000FF; mac2 = (m_id>>16) & 0X00007F; 4.1.4 According to the link layer Ethernet multicast protocol, construct the multicast MAC address m_mac: m_mac=01:00:5e:mac2:mac1:mac0; 4.1.5 Calculation: ip0 = m_id & 0x000000ff; ip1 = (m_id >> 8) & 0X0000FF; ip2 = (m_id>>16) & 0x00FF; ip3 = (m_id>>24) & 0X00E0; 4.1.6 According to the link layer Ethernet multicast protocol, construct the multicast IP address m_ip: m_ip = ip3.ip2.ip1.ip0; 4.1.7 returns the multicast MAC address m_mac and the multicast IP address m_ip to the multicast group management module, and goes to 4.1.1 to continue monitoring.

4. The method for implementing RDMA multicast in a RoCE network according to claim 2, wherein the specific steps of 4.2 are: 4.2.1 The multicast group management module reads N, MIN_ID, MAX_ID, QN from the multicast group configuration file; reads the current multicast group number cn and the multicast group structure array mcast_array[i] from the multicast group data buffer; 4.2.2 The multicast group management module monitors whether it is called by the application; if it is called, execute 4.2.3; if it is not called, go to 4.2.2 and continue to monitor; 4.2.3 The multicast group management module receives a multicast group management command from the application, wherein the multicast group management command is "join" or "leave", as well as the QP number m_qpn of the operation and the multicast ID m_id of the operation multicast group; if cn≥N or m_id<MIN_ID or m_id>MAX_ID, go to step 4.2.2 to continue monitoring, otherwise go to step 4.2.4; 4.2.4 Call the multicast group address calculation module to obtain the Ethernet multicast MAC address m_mac and multicast IP address m_ip; 4.2.5 Determine whether there is an element in the multicast group structure array mcast_array[i] whose mid field value is equal to m_id; if not and the multicast group management command is "join", go to 4.2.6; if not and the multicast group management command is "leave", go to step 4.2.2 to continue monitoring; if so, let the element be the m_indexth element in the multicast group structure array, if the multicast group management command is "join", go to step 4.2.7; if the multicast group management command is "leave", go to step 4.2.8; 4.2.6 Add a new multicast group with multicast ID m_id, multicast MAC address m_mac and multicast IP address m_ip to the multicast group structure array, and add a member with QP number m_qpn to the multicast group; Go to 4.2.9; 4.2.7 Add a member with QP number m_qpn to the multicast group with multicast ID m_id; Go to 4.2.9; 4.2.8 In the multicast group with multicast ID m_id, delete the member with member qpn m_qpn; Go to 4.2.9; 4.2.9 Update the updated cn and mcast_array[i] to the multicast group data buffer; go to 4.2.2 to continue monitoring.

5. The method for implementing RDMA multicast in a RoCE network according to claim 2, wherein: The specific steps of 4.3 are as follows: 4.3.1 The multicast forwarding module monitors whether there is data from the sending data acquisition module. If so, go to 4.3.

3. If not, go to 4.3.2 to continue monitoring; 4.3.2 The multicast forwarding module monitors whether there is data from the multicast message parsing module. If yes, execute 4.3.

4. If not, go to 4.3.1 to continue monitoring. 4.3.3 Multicast forwarding of sent data locally; 4.3.4 Perform multicast forwarding of received data locally.

6. The method for implementing RDMA multicast in a RoCE network according to claim 2, wherein: The specific steps of 4.4 are as follows: 4.4.1 The multicast message parsing module monitors whether RoCE network messages have arrived; if yes, execute 4.4.2; if no, go to 4.4.1 to continue monitoring; 4.4.2 Receive RoCE network messages; 4.4.3 Extract data pkt_data and destination MAC address pkt_mac from the RoCE network message according to the RoCE network message format; 4.4.4 Determine whether pkt_mac is an Ethernet multicast address according to the Ethernet link layer multicast protocol; the method is: If the following conditions are met at the same time: 1) pkt_mac[5] = 0x01; 2) pkt_mac[4] = 0x00; 3) pkt_mac[3] = 0x5E; 4)pkt_mac[2]≤0X7F; If pkt_mac is an Ethernet multicast address, go to step 4.4.5; otherwise, pkt_mac is not an Ethernet multicast address, go to step 4.4.1 to continue monitoring; 4.4.5 Calculate the multicast ID pkt_mid based on the Ethernet multicast protocol and pkt_mac: pkt_mid=pkt_mac[2]<<16+pkt_mac[1]<<8+pkt_mac[0]; 4.4.6 Send pkt_data and pkt_mid to the multicast forwarding module; go to step 4.4.1 to continue monitoring.

7. The method for implementing RDMA multicast in a RoCE network according to claim 5, characterized in that: 4.3.3 Multicast the sent data locally by: 4.3.3.1 The multicast forwarding module receives the data to be sent out_data and the destination MAC address dst_mac from the sending data acquisition module; 4.3.3.2 The multicast forwarding module determines whether dst_mac is an Ethernet multicast address according to the Ethernet multicast protocol; the method is: If the following conditions are met at the same time: 1) dst_mac[5] = 0x01; 2) dst_mac[4] = 0x00; 3) dst_mac[3] = 0x5E; 4)dst_mac[2]≤0X7F; If dst_mac is an Ethernet multicast address, go to step 4.3.3.3; otherwise, dst_mac is not an Ethernet multicast address, go to step 4.3.1 to continue monitoring; 4.3.3.3 Calculate the multicast ID dst_mid of the multicast group based on the Ethernet multicast protocol and dst_mac: dst_mid=dst_mac[2]<<16+dst_mac[1]<<8+dst_mac[0]; 4.3.3.4 Read the number of multicast groups N and N elements of the multicast group structure array mcast_array[i] from the multicast group data buffer; 4.3.3.5 Determine whether there is an element in the multicast group structure array mcast_array[i] with a mid field value equal to dst_mid. If so, set the element to mcast1 and go to step 4.3.3.

6. If not, go to step 4.3.1 and continue monitoring. 4.3.3.6 Copy the qn field value of mcast1 for the data out_data to be sent, and send it to the receiving data uploading module together with each element in the qp array field of mcast1; go to 4.3.1 to continue monitoring.

8. The method for implementing RDMA multicast in a RoCE network according to claim 5, characterized in that: 4.3.4 Multicast the received data locally by: 4.3.4.1 Receive the message data in_data and the received message multicast group ID in_mid from the multicast message parsing module; 4.3.4.2 Read the number of multicast groups N and N elements of the multicast group structure array mcast_array[i] from the multicast group data buffer; 4.3.4.3 Determine whether there is an element in the multicast group structure array mcast_array[i] with a mid field value equal to in_mid. If so, set the element to mcast2 and go to step 4.3.4.

4. If not, go to step 4.3.1 and continue monitoring. 4.3.4.4 Copy the message data in_data to the qn field value of mcast2, and send it to the receiving data uploading module together with each element in the qp array field of mcast2; go to 4.3.1 to continue monitoring.

Images