Gateway switch retransmission method for RDMA data tail packet

Abstract

The invention relates to the technical field of computer data transmission, and provides a gateway switch retransmission method for an RDMA (Remote Direct Memory Access) data tail packet, which is characterized by comprising the following steps: receiving an RDMA message, and adding a tail packet identification field to a user-defined field of a header of the RDMA message; distinguishing the flow type of the RDMA message and identifying a tail packet; storing a tail packet of the RDMA message, calculating a retransmission period, and starting a retransmission timer; when the retransmission timer expires, triggering retransmission of packing and forwarding a tail packet of the stored RDMA message; and confirming that the transmission of the RDMA message is completed, and releasing the tail packet data related to the RDMA message. According to the method and the device, the tail packet of the corresponding flow is identified and then stored in the buffer area connected with the switch for regular retransmission, once the data packet is lost, the corresponding NACK does not need to be waited to arrive at the sender, and the method and the device depend on regular retransmission at the gateway, so that the round-trip time delay is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of computer data transmission, and particularly relates to a method for a gateway switch to retransmit the last packet of RDMA data. Background Art

[0002] In recent years, compared with traditional TCP / IP networks, Remote Direct Memory Access technology (RDMA) has been widely used in data centers due to its high performance and low latency, such as Azure, Amazon Cloud, etc. However, RDMA requires a lossless network state to achieve its maximum performance. The current mainstream solution mainly combines the RDMA over Converged Ethernet (RoCE) protocol with Ethernet, and cooperates with a priority-based flow control algorithm (PFC) and a congestion control algorithm (CC) to achieve a lossless network. However, PFC itself has many problems that cannot be fundamentally changed. Therefore, in recent years, researchers have been constantly reconsidering whether RDMA must be lossless. With the advancement of research, the current second-generation RoCE already supports a lossy network. Therefore, many packet loss recovery algorithms are used to ensure the overall performance.

[0003] However, the current packet loss recovery technology mainly focuses on the network cards of the sender and the receiver, or switches within the same data center also participate in packet loss recovery. Existing technologies have not made new optimizations for the network environment where two data centers are connected. Different data centers are often located in several different cities, and the distance between these cities is very long. The round-trip time (RTT) required for a communication is relatively long. This will result in that once a packet is lost, in addition to the RTT within the data center, it will also pass through the RTT of a long-distance link. This is actually very wasteful of performance. At the same time, most of the traffic within the data center is small traffic, and many flows even have only one packet. Different from TCP, for some RDMA operations, many key control information is stored in the first packet and the last packet of a flow. Especially for the last packet of a flow, since it is the last packet of the flow, once it is lost and no subsequent packets arrive, in addition to facing a long-distance RTT, it also faces timeout retransmission with a long waiting period. Summary of the Invention

[0004] In view of the above-mentioned defects of the prior art, the present invention proposes a method for a gateway switch to retransmit the last packet of RDMA data. The technical solution steps include:

[0005] S1: Receive an RDMA packet, and add a last packet identification field to a custom field in the header of the RDMA packet;

[0006] S2: Distinguish the flow type of the RDMA packet and identify the last packet;

[0007] S3: Store the last packet of the RDMA packet and calculate the retransmission period, and enable the retransmission timer;

[0008] S4: When the retransmission timer expires, trigger the retransmission of packing and forwarding the last packet of the stored RDMA packet;

[0009] S5: Confirm that the transmission of the RDMA packet is completed, and release the last packet data related to the RDMA packet.

[0010] Preferably, the S20 includes:

[0011] S201: Determine whether the speed of the incoming port link of the RDMA packet is a long-distance link with high latency. If so, execute step S202; otherwise, do nothing;

[0012] S202: Parse the Ethernet header, IP header, UDP header, and custom header of the RDMA packet;

[0013] S203: Determine whether the port number of the RDMA packet conforms to the remote direct memory access port number of the first version of converged Ethernet. If so, execute step S205; otherwise, execute S204;

[0014] S204: Determine whether the port number of the RDMA packet conforms to the remote direct memory access port number of the second version of converged Ethernet. If so, execute step S205; otherwise, do nothing;

[0015] S205: Determine whether the last packet identification field of the RDMA packet is enabled. If so, execute S3; otherwise, do nothing.

[0016] Preferably, the S3 includes:

[0017] S301: Generate a flow identifier through hash mapping based on the five-tuple information of the header of the RDMA packet;

[0018] S302: Check whether the last packet of the RDMA packet exists in the buffer. If so, do nothing; otherwise, execute step S303;

[0019] S303: Store the RDMA packet, record the transmission timestamp and the number of retransmissions;

[0020] S304: Calculate the retransmission period;

[0021] S305: Enable the retransmission timer and wait for retransmission.

[0022] Preferably, the S304 includes:

[0023] S3001: Calculate the latency of a single hop within the data center;

[0024] S3002: Calculate the round-trip delay end-to-end inside the data center;

[0025] S3003: Calculate the smoothed round-trip delay;

[0026] S3004: Calculate the variance of the round-trip delay;

[0027] S3005: Calculate the retransmission period.

[0028] Preferably, the S4 includes:

[0029] S401: When the retransmission timer expires, check whether there is corresponding saved information. If yes, execute step S402; otherwise, delete the retransmission timer information and do nothing;

[0030] S402: Trigger the retransmission of packetizing and forwarding the tail packet of the stored RDMA packet. Judge the number of retransmissions. When the number is greater than 5, stop the retransmission, stop and remove the retransmission timer, and remove the relevant information of the flow and tail packet of this RDMA packet. When the number is less than or equal to 5, execute step S4003;

[0031] S403: Generate a new packet and record the timestamp, calculate a new retransmission period and update the number of retransmissions;

[0032] S404: Enable the retransmission timer based on the new retransmission period.

[0033] Preferably, the S5 includes:

[0034] S501: When a new data stream enters, parse the header of the new RDMA packet;

[0035] S502: Judge whether the type of the header of the new RDMA packet is ACK. If yes, execute step S503; otherwise, do nothing;

[0036] S503: According to the five-tuple information of the header of the new RDMA packet, perform reverse processing to generate the corresponding flow identifier according to the hash mapping;

[0037] S504: Search for the corresponding tail packet information based on the flow identifier of the new RDMA packet. Judge whether it is found. If yes, execute step S505; otherwise, do nothing;

[0038] S505: Stop processing and remove the corresponding tail packet information and its retransmission timer.

[0039] Beneficial effects:

[0040] 1. The present invention only saves the tail packet, which occupies less buffer space. Existing solutions either cache all data packets within a period of time or cache the data packets lost by the corresponding switch, which puts too much pressure on the switch buffer.

[0041] 2. This invention saves and retransmits the tail packet through the gateway of the cross-data center network, ensuring that the tail packet can be retransmitted in a very short time if it is lost. However, existing solutions do not make special treatment for the tail packet. Since the tail packet of remote direct memory access can only be retransmitted by relying on the timeout retransmission mechanism of the sender, it has a significant impact on the completion time of the flow.

[0042] 3. The present invention uses a retransmission cycle algorithm to reasonably calculate the round-trip delay of the data center link, derive the retransmission cycle, and increase the retransmission cycle with each retransmission to prevent aggravation of network congestion. However, existing inventions do not consider the increase in retransmitted data packets during retransmission, which increases the network load.

[0043] 4. The present invention determines the gateway switch by the delay of the link, and then saves the tail packet at the gateway switch, so that the tail packet of the traffic between data centers can be retransmitted without waiting for a long round-trip delay. However, the existing solutions are not optimized for the latest cross-data center network environment. They are all based on the improvement work within the internal network of a single data center, ignoring the fact that the ultra-long round-trip delay caused by the long-distance link limits the completion time of the flow. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of a preferred embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of a network topology of a preferred embodiment of the present invention;

[0046] Figure 3 This is an abstract schematic diagram of a switch according to a preferred embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of a remote direct memory access data message header according to a preferred embodiment of the present invention;

[0048] Figure 5 Schematic diagram of five-tuple information of a message in a preferred embodiment of the present invention; DETAILED DESCRIPTION

[0049] The embodiments of the present invention are described in detail below. The following embodiments are implemented based on the technical solutions of the present invention, and provide detailed implementation methods and specific operating procedures. However, the protection scope of the present invention is not limited to the following embodiments.

[0050] The present invention is designed to process the gateway switch for connecting the data center and the long - distance link. When a data packet arrives, it is judged whether it is an RDMA flow, whether it is a RoCEv2 traffic, and the tail packet of the corresponding traffic is identified, and then it is saved in the buffer connected to the switch and re - transmitted regularly. Once a data packet is lost, instead of waiting for the corresponding NACK to reach the sender, it relies on the regular re - transmission at the gateway, which greatly reduces the round - trip delay and reduces the completion time of the flow.

[0051] The present invention proposes a method for re - transmitting the tail packets of RDMA data by a gateway switch. This scheme is based on an RDMA data stream tail packet loss re - transmission system designed for the gateway switch of the data center network. This system runs on the gateway switch, and the overall hardware topology environment of the switch and the server is as Figure 2 shown. The tail packet detection, storage, and re - transmission system designed by this technical scheme mainly judges the data packets entering the inlet pipeline of the switch, then saves the tail packets of the target data stream, and performs regular re - transmission. The application of this method to the abstract model of the switch is as Figure 3 shown.

[0052] The present invention proposes a method for re - transmitting the tail packets of RDMA data by a gateway switch, as Figures 1-5 shown. The technical scheme includes the following steps, specifically including:

[0053] S1: Receive an RDMA message, and add a tail packet identification field to the custom field of the header of the RDMA message;

[0054] S2: Distinguish the flow type of the RDMA message and identify the tail packet;

[0055] S3: Store the tail packet of the RDMA message and calculate the re - transmission period, and start the re - transmission timer;

[0056] S4: When the re - transmission timer expires, trigger the re - transmission of packing and forwarding the stored tail packet of the RDMA message;

[0057] S5: Confirm that the transmission of the RDMA message is completed, and release the tail packet data related to the RDMA message.

[0058] Specifically, for step S1, as Figure 4 shown, it is necessary to briefly supplement the custom field of the header of the current RDMA message, and add a 2 - bit tail packet identification field (LastPktFlag) to the custom header. Although the basic transmission header (BTH) field in the RDMA message contains the identification of the first packet and the tail packet of the data packet, since most current data center switches cannot parse the content of the BTH header, in order for the switch to identify the tail packet of the RDMA flow, it is necessary to add a tail packet identification field to the custom header.

[0059] Preferably, S20 includes:

[0060] S201: Determine whether the speed of the ingress port link of the RDMA packet is a long - distance link with high latency. If so, execute step S202; otherwise, do nothing.

[0061] S202: Parse the Ethernet header, IP header, UDP header, and custom header of the RDMA packet.

[0062] S203: Determine whether the port number of the RDMA packet conforms to the remote direct memory access port number of the first - version converged Ethernet. If so, execute step S205; otherwise, execute S204.

[0063] S204: Determine whether the port number of the RDMA packet conforms to the remote direct memory access port number of the second - version converged Ethernet. If so, execute step S205; otherwise, do nothing.

[0064] S205: Determine whether the tail - packet flag field of the RDMA packet is enabled. If so, execute S3; otherwise, do nothing.

[0065] Specifically, for step S2, the switch enables the remote direct memory access packet detection device and the tail - packet detection device to enter step S201. When the data packet enters the ingress pipeline of the switch, each switch has the same RDMA flow detection device and tail - packet detection device, which are part of the switch. This step is responsible for distinguishing the RDMA flow, RoCE, RoCEv2 traffic from other traffic and identifying the tail - packet. Among them, the enable judgment can be whether its value is 1. The remote direct memory access port numbers of the first - version converged Ethernet and the second - version converged Ethernet are respectively that the remote direct memory access (RoCE, RDMA over Converged Ethernet) of the converged Ethernet is divided into two versions: RoCEv1 and RoCEv2.

[0066] Preferably, S3 includes:

[0067] S301: Generate a flow identifier through hash mapping based on the five - tuple information of the header of the RDMA packet.

[0068] S302: Check whether the tail - packet of the RDMA packet exists in the buffer. If so, do nothing; otherwise, execute step S303.

[0069] S303: Store the RDMA packet, record the transmission timestamp and the re - transmission count.

[0070] S304: Calculate the re - transmission period.

[0071] S305: Enable the retransmission timer and wait for retransmission.

[0072] Preferably, S304 includes:

[0073] S3001: Calculate the latency of a single hop within the data center;

[0074] S3002: Calculate the round-trip time (RTT) end-to-end within the data center;

[0075] S3003: Calculate the smoothed round-trip time;

[0076] S3004: Calculate the variance of the round-trip time;

[0077] S3005: Calculate the retransmission period.

[0078] Specifically, for step S3, where the five-tuple information is as Figure 5 shown, save the qualified RDMA tail packets determined by the RDMA flow detection device and the tail packet detection device to the tail packet memory. For calculating the retransmission period, first calculate the latency of a single hop within the data center according to formula (1), where T ingress，i is the timestamp when the data packet enters the i-th switch, and T egress，i is the timestamp when the data packet leaves the i-th switch;

[0079] Hop latency i = T egress,i - T ingress,i (1)

[0080] Calculate the end-to-end round-trip time within the data center according to formula (2), where N is the number of switch hops in the path and RTT is the round-trip time;

[0081]

[0082] Calculate the smoothed round-trip time according to formula (3). Considering that the RTT itself may fluctuate, especially when the network is not stable, if there is congestion, the RTT of a certain data packet may suddenly become high. To avoid premature or late retransmission due to extremely individual RTTs and waste resources, it is necessary to calculate the smoothed RTT to obtain a more stable estimated value, where RTT sample is the RTT calculated by formula (2), and α is the smoothing factor of SRTT, usually set to 1 / 8;

[0083] SRTT = (1 - α) × SRTT + α × RTT sample (3)

[0084] Calculate the variance of the round-trip delay according to formula (4). When calculating the variance of the round-trip delay, it is necessary to calculate the smoothed SRTT and then measure the variation range of RTT through the variance of RTT. If the variance is large, it indicates that RTT is unstable. Taking the retransmission timeout period RTO as an example, a larger RTO needs to be set at this time to cope with larger fluctuations. On the contrary, if the variance is small, it indicates that RTT is relatively stable, and RTO can be closer to SRTT to reduce unnecessary waiting time. Among them, RTTVAR is the variance of RTT, and β is the smoothing factor of RTTVAR, usually set to 1 / 4.

[0085] RTTVAR = (1 - β) × RTTVAR + β × |SRTT - RTT sample | (4)

[0086] Calculate the retransmission timeout period according to formula (5). RTO is the retransmission timeout period, C is the number of retransmissions, and γ is the backoff factor, usually set to 2. The timer is doubled after each transmission. For example, after the first transmission, the timer New RTO is set to 2 times the Previous RTO. The upper limit is set to 5 transmissions. The final RTO cannot be lower than the RTT of the shortest link in the data center network min and cannot be higher than 16 times the RTT max to prevent infinite waiting. After the calculation is completed, go to step S305;

[0087]

[0088] Preferably, S4 includes:

[0089] S401: When the retransmission timer expires, check whether there is corresponding saved information. If so, execute step S402; otherwise, delete the retransmission timer information and do nothing.

[0090] S402: Trigger the retransmission of the tail packet of the stored RDMA packet by packetizing and forwarding. Judge the number of retransmissions. When the number is greater than 5, stop the retransmission, stop and remove the retransmission timer, and remove the relevant information of the flow and tail packet of the RDMA packet. When the number is less than or equal to 5 times, execute step S4003;

[0091] S403: Generate a new packet and record the timestamp, calculate the new retransmission timeout period and update the number of retransmissions;

[0092] S404: Enable the retransmission timer based on the new retransmission timeout period.

[0093] Specifically, the corresponding saved information is all the associated data maintained for a specific data stream in the tail packet memory, which is used to recover or retransmit the tail packet when the retransmission is triggered, including but not limited to the flow identifier, tail packet data, timestamp, and retransmission timeout period parameters.

[0094] Specifically, after the retransmission timer expires and triggers a retransmission, it enters the tail packet retransmission device. The tail packet retransmission device finds the corresponding packet, performs packet assembly and then forwards it, and updates the necessary information.

[0095] Preferably, S5 includes:

[0096] S501: When a new data stream enters, parse the header of the new RDMA packet;

[0097] S502: Determine whether the type of the header of the new RDMA packet is ACK. If it is, execute step S503; otherwise, do nothing.

[0098] S503: According to the five-tuple information of the header of the new RDMA packet, perform reverse processing to generate the corresponding flow identifier according to the hash mapping.

[0099] S504: Based on the flow identifier of the new RDMA packet, find the corresponding tail packet information and determine whether it is found. If it is, execute step S505; otherwise, do nothing.

[0100] S505: Stop processing and remove the corresponding tail packet information and its retransmission timer.

[0101] Specifically, ACK is the confirmation notice for the corresponding packet. Receiving it indicates that this flow has been completed. At this time, there is no need to retransmit the tail packet anymore. Therefore, in order to consider the space utilization rate, the relevant information of the corresponding flow and tail packet needs to be removed from the tail packet memory.

[0102] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations according to the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field of the present invention through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the protection scope determined by the claims.

Claims

1. A method for a gateway switch to retransmit the tail packet of RDMA data, characterized in that, Including: S1: Receive an RDMA packet, and add a tail packet identification field to the custom field of the header of the RDMA packet; S2: Distinguish the flow type of the RDMA packet and identify the tail packet; S3: Store the tail packet of the RDMA packet, calculate the retransmission period, and enable the retransmission timer; S4: When the retransmission timer expires, trigger the retransmission of assembling and forwarding the stored tail packet of the RDMA packet; S5: Confirm the completion of the RDMA packet transmission, and release the tail packet data related to the RDMA packet.

2. The retransmission method of a gateway switch for the RDMA data tail packet according to claim 1, wherein The S20 includes: S201: Determine whether the speed of the incoming port link of the RDMA packet is a long-distance link with high latency. If so, execute step S202; otherwise, do not process; S202: Parse the Ethernet header, IP header, UDP header, and custom header of the RDMA packet; S203: Determine whether the port number of the RDMA packet conforms to the remote direct memory access port number of the first version of converged Ethernet. If so, execute step S205; otherwise, execute S204; S204: Determine whether the port number of the RDMA packet conforms to the remote direct memory access port number of the second version of converged Ethernet. If so, execute step S205; otherwise, do not process; S205: Determine whether the tail packet identification field of the RDMA packet is enabled. If so, execute S3; otherwise, do not process.

3. A method for retransmitting the last packet of RDMA data by a gateway switch according to claim 1, characterized in that The S3 includes: S301: Generate a flow identifier through hash mapping based on the five-tuple information of the header of the RDMA packet; S302: Check whether the tail packet of the RDMA packet exists in the buffer. If so, do not process; otherwise, execute step S303; S303: Store the RDMA packet, record the send timestamp and the retransmission count; S304: Calculate the retransmission period; S305: Enable the retransmission timer to wait for retransmission.

4. The retransmission method of a gateway switch for the RDMA data tail packet according to claim 3, characterized in that, The S304 includes: S3001: Calculate the latency of a single hop within the data center; S3002: Calculate the end-to-end round-trip delay within the data center; S3003: Calculate the smoothed round-trip delay; S3004: Calculate the variance of the round-trip delay; S3005: Calculate the retransmission period.

5. The retransmission method of a gateway switch for the RDMA data tail packet according to claim 1, wherein The S4 includes: S401: When the retransmission timer expires, check whether there is corresponding saved information. If so, execute step S402; otherwise, delete the retransmission timer information and do not process; S402: Trigger the retransmission of assembling and forwarding the stored tail packet of the RDMA packet, judge the retransmission count. When the count is greater than 5, stop the retransmission, stop and remove the retransmission timer, and remove the relevant information of the flow and tail packet of the RDMA packet. When the count is less than or equal to 5 times, execute step S4003; S403: Generate a new packet and record the timestamp, calculate the new retransmission period and update the retransmission count; S404: Enable the retransmission timer based on the new retransmission period.

6. The retransmission method of a gateway switch for the last packet of RDMA data according to claim 1, wherein The S5 includes: S501: When a new data stream enters, parse the header of the new RDMA packet; S502: Determine whether the type of the header of the new RDMA packet is ACK. If so, execute step S503; otherwise, do not process; S503: Reverse process to generate a corresponding flow identifier according to the hash mapping based on the new five-tuple information of the RDMA packet header; S504: Search for the corresponding tail packet information based on the flow identifier of the new RDMA packet, determine whether it is found. If so, execute step S505; otherwise, do not process; S505: Stop processing and remove the corresponding tail packet information and its retransmission timer.

Images