A load balancing method and related equipment for RDMA network data transmission

CN118317366BActive Publication Date: 2024-11-22XIANGJIANG LAB
View PDF -1 Cites 0 Cited by

Patent Information

Application Number
CN202410538262.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-30
Publication Date
2024-11-22
Estimated Expiration
2044-04-30

Smart Images

  • Figure CN118317366B_ABST
    Figure CN118317366B_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of data transmission, and provides a load balancing method and related equipment for RDMA network data transmission. The method includes: obtaining the packet forwarding rate, the total length of the packet queue, the packet arrival rate of each path, and the path packet queue length of each path; calculating the congestion pause time difference based on the packet forwarding rate and the packet arrival rate of all paths; when the congestion pause time difference is less than the preset time difference, obtaining the path packet queue length threshold based on the total length of the packet queue, and using the path packet queue length threshold to determine the congested path from all paths; obtaining the path delay of each path based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths; determining the target path from all paths using all path delays; and sending a path congestion message to the upstream switch. The method can improve the load balancing performance of data transmission.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to a load balancing method and related equipment for RDMA network data transmission. Background Art

[0002] At present, data center remote direct memory access (RDMA) networks are widely used in distributed storage, high performance computing (HPC), distributed AI training and other scenarios. In order to further improve the transmission performance of lossless networks in data centers, researchers have proposed a series of transmission control and load balancing solutions to optimize the RDMA communication performance between data center servers. Among them, unlike optimizing the end-to-end single-path communication process, the goal of the load balancing solution is to evenly distribute the network traffic between nodes to multiple paths, which is crucial to improving network transmission performance. The current data transmission load balancing method for RDMA data streams in lossless networks in data centers can monitor the congestion level of different paths and reroute the data stream to a path with less congestion. In addition, priority flow control (PFC) is a necessary flow control mechanism in lossless Ethernet. The existing load balancing mechanism predicts the PFC pause / resume frame on the switch to achieve early response to network congestion to avoid performance loss caused by PFC pause due to congestion.

[0003] However, although the existing load balancing mechanism has achieved certain results in improving the transmission performance of lossless networks, it is still unable to quickly perceive the data flow that causes path congestion, which will lead to performance loss, including: 1) Load balancing based on link utilization: The path where PFC is suspended is identified as a non-congested path due to low link utilization, resulting in a large number of data flows being rerouted to the path where PFC is suspended, which will aggravate PFC suspension and spread PFC congestion; 2) Load balancing based on link delay: Since it takes at least one round-trip time to monitor the path delay at the source switch, and the path delay changes quickly, the PFC suspension / resumption The time period is small, resulting in the congested data flow being rerouted to other paths, while the PFC pause of its old path may have ended, which will lead to low link utilization of the old path, PFC pause of the new path and spread of PFC congestion; 3) Load balancing based on downstream switch PFC pause prediction: The PFC pause time is predicted according to the queue length of the downstream switch port queue, and the upstream switch is actively notified that the path is a congested path. However, due to the lack of precise positioning of the data flow that causes the path congestion, other normal data flows are easily rerouted to other paths, affecting transmission efficiency and causing additional RDMA data packet disorder. It can be seen that the existing load balancing mechanism has the problem of low load balancing performance of RDMA network data transmission. Summary of the invention

[0004] The present application provides a load balancing method and related equipment for RDMA network data transmission, which can solve the problem of low load balancing performance of RDMA network data transmission.

[0005] In a first aspect, an embodiment of the present application provides a load balancing method for RDMA network data transmission, the load balancing method comprising:

[0006] Obtain the packet forwarding rate and total length of the packet queue of the downstream switch at the current moment, as well as the packet arrival rate of each path and the length of the path packet queue of each path at the current moment; a path is a link through which the upstream switch sends packets to the downstream switch;

[0007] Based on the packet forwarding rate and the packet arrival rate of all paths, the congestion pause time difference of the downstream switch is calculated;

[0008] When the congestion pause time difference is less than the preset time difference, a path packet queue length threshold is obtained based on the total length of the packet queue, and a plurality of congested paths are determined from all paths using the path packet queue length threshold; the path packet queue length of the congested path is greater than the path packet queue length threshold;

[0009] Based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths, the path delay of each path is obtained; the path delay is used to describe the delay of message transmission and the delay of packet transmission on the path;

[0010] Multiple target paths are determined from all paths using all path delays; the path delay of each target path is smaller than the path delays of all other paths;

[0011] A path congestion message is sent to the upstream switch, so that the upstream switch uses all target paths to transmit data packets sent by all congested paths within a future preset time period according to the identification information of each congested path and the identification information of each target path carried in the path congestion message.

[0012] Optionally, based on the packet forwarding rate and the packet arrival rate of all paths, the congestion pause time difference of the downstream switch is calculated, including:

[0013] By formula:

[0014]

[0015] Calculate the congestion pause time difference Δt of the downstream switch;

[0016] Among them, Q PFCIndicates the total length of the queue pause of packets that trigger congestion pause, t d represents the current delay of transmitting messages to the upstream switch, n represents the number of paths, vi(t) represents the packet arrival rate of the ith path at the current moment, t represents the current moment, and vr(t) represents the packet forwarding rate.

[0017] Optionally, a path packet queue length threshold is obtained based on the total packet queue length, including:

[0018] By formula:

[0019]

[0020] Calculate the path packet queue length threshold F cc ;

[0021] Among them, Q represents the total length of the packet queue, Q T It indicates recording the queue length of all path packets and the list of all paths. qInd indicates the queue index, and fNum indicates the number of paths in the queue index.

[0022] Optionally, based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths, obtain the path delay of each path, including:

[0023] Obtain the congestion pause recovery time based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths;

[0024] Based on the congestion pause recovery time, the path delay of each path is obtained.

[0025] Optionally, based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths, obtain the congestion pause recovery time, including:

[0026] By formula:

[0027]

[0028] Calculate the congestion pause recovery time t re ;

[0029] Among them, Q represents the total length of the packet queue, Indicates the total length of the packet queue restored at the end of the congestion pause, T re_flight The time when the message indicating the end of congestion pause arrives at the upstream switch, and ΔL indicates the gradient of queue length change:

[0030]

[0031] Where n represents the number of paths, vi(t) represents the packet arrival rate of the i-th path at the current moment, t represents the current moment, and vr(t) represents the packet forwarding rate.

[0032] Optionally, based on the congestion pause recovery time, obtain the path delay of each path, including:

[0033] Get the transmission delay of each path;

[0034] For each path, perform the following steps:

[0035] Determine whether the path is a congested path;

[0036] If yes, the sum of the congestion pause recovery time and the transmission delay of the path is taken as the path delay of the path;

[0037] Otherwise, the transmission delay of the path is taken as the path delay of the path.

[0038] Optionally, multiple target paths are determined from all paths using all path delays, including:

[0039] Arrange all path delays from small to large, and use the path delays of a preset number in the arrangement result as the target path delay;

[0040] The path corresponding to each target path delay is respectively taken as a target path.

[0041] In a second aspect, an embodiment of the present application provides a load balancing device for RDMA network data transmission, including:

[0042] The first acquisition module acquires the packet forwarding rate and total length of the packet queue of the downstream switch at the current moment, as well as the packet arrival rate and path packet queue length of each path at the current moment; the path is the link through which the upstream switch sends packets to the downstream switch;

[0043] The calculation module calculates the congestion pause time difference of the downstream switch based on the packet forwarding rate and the packet arrival rate of all paths;

[0044] The first determination module, when the congestion suspension time difference is less than the preset time difference, obtains a path data packet queue length threshold based on the total length of the data packet queue, and determines multiple congested paths from all paths using the path data packet queue length threshold; the path data packet queue length of the congested path is greater than the path data packet queue length threshold;

[0045] The second acquisition module acquires the path delay of each path based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths; the path delay is used to describe the delay of message transmission and the delay of packet transmission on the path;

[0046] The second determination module determines multiple target paths from all paths using all path delays; the path delay of each target path is smaller than the path delays of all other paths;

[0047] The sending module sends a path congestion message to the upstream switch, so that the upstream switch uses all target paths to transmit data packets sent by all congested paths within a future preset time period according to the identification information of each congested path and the identification information of each target path carried in the path congestion message.

[0048] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned load balancing method for RDMA network data transmission when executing the above-mentioned computer program.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned load balancing method for RDMA network data transmission.

[0050] The above solution of the present application has the following beneficial effects:

[0051] In an embodiment of the present application, by obtaining the packet forwarding rate and total length of the packet queue of the downstream switch at the current moment, as well as the packet arrival rate and path packet queue length of each path at the current moment, then based on the packet forwarding rate and the packet arrival rate of all paths, the congestion pause time difference of the downstream switch is calculated, when the congestion pause time difference is less than the preset time difference, the path packet queue length threshold is obtained based on the total length of the packet queue, and the path packet queue length threshold is used to determine multiple congested paths from all paths, then based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths, the path delay of each path is obtained, and then all path delays are used to determine multiple target paths from all paths, and finally a path congestion message is sent to the upstream switch, so that the upstream switch uses all target paths to transmit data packets sent by all congested paths within a preset time period in the future according to the identification information of each congested path and the identification information of each target path carried in the path congestion message. Among them, the congestion pause time difference is calculated based on the data packet forwarding rate and data packet arrival rate at the current moment, so that the congestion situation can be monitored in real time, and the real-time and accuracy of the congestion pause time difference can be improved. The target path is determined based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate. The change of the data packet queue of the path is taken into account, so that the performance of the obtained target path is better than that of other paths. At the same time, the path congestion message carrying the identification information of the target path and the identification information of the congested path is sent to the upstream switch, so that the upstream switch can simultaneously identify the congested path and the target path, which is convenient for the upstream switch to perform accurate load balancing, thereby improving the load balancing performance of data transmission.

[0052] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 A flow chart of a load balancing method for RDMA network data transmission provided in an embodiment of the present application;

[0055] Figure 2 A schematic diagram of a load balancing system for RDMA network data transmission provided by an embodiment of the present application;

[0056] Figure 3A schematic diagram of the operation of a load balancing system for RDMA network data transmission provided by an embodiment of the present application;

[0057] Figure 4 A schematic diagram of the structure of a load balancing device for RDMA network data transmission provided in one embodiment of the present application;

[0058] Figure 5 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0059] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0060] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0061] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0062] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0063] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0064] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0065] In view of the problem of low load balancing performance of existing RDMA network data transmission, an embodiment of the present application provides a load balancing method for RDMA network data transmission. The load balancing method obtains the packet forwarding rate and the total length of the packet queue of the downstream switch at the current moment, as well as the packet arrival rate and the path packet queue length of each path at the current moment, and then calculates the congestion pause time difference based on the packet forwarding rate and the packet arrival rate of all paths. When the congestion pause time difference is less than the preset time difference, the path packet queue length threshold is obtained based on the total length of the packet queue, and multiple congested paths are determined from all paths using the path packet queue length threshold. Then, based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths, the path delay of each path is obtained, and then multiple target paths are determined from all paths using all path delays. Finally, a path congestion message is sent to the upstream switch, so that the upstream switch uses all target paths to transmit the packets sent by all congested paths within a preset time period in the future according to the identification information of each congested path and the identification information of each target path carried in the path congestion message. Among them, the congestion pause time difference is calculated based on the data packet forwarding rate and data packet arrival rate at the current moment, so that the congestion situation can be monitored in real time, and the real-time and accuracy of the congestion pause time difference can be improved. The target path is determined based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate. The change of the data packet queue of the path is taken into account, so that the performance of the obtained target path is better than that of other paths. At the same time, the path congestion message carrying the identification information of the target path and the identification information of the congested path is sent to the upstream switch, so that the upstream switch can simultaneously identify the congested path and the target path, which is convenient for the upstream switch to perform accurate load balancing, thereby improving the load balancing performance of data transmission.

[0066] Next, the load balancing method for RDMA network data transmission provided by the present application is exemplified.

[0067] like Figure 1As shown, the load balancing method for RDMA network data transmission provided by the present application includes the following steps:

[0068] Step 11, obtaining the packet forwarding rate and total length of the packet queue of the downstream switch at the current moment, as well as the packet arrival rate and path packet queue length of each path at the current moment.

[0069] The above path is the link for the upstream switch to send data packets to the downstream switch.

[0070] In some embodiments of the present application, a transmission performance testing tool such as iperf can be used to obtain the packet forwarding rate of the downstream switch, the total length of the packet queue, the packet arrival rate of each path, and the path packet queue length of each path.

[0071] It should be noted that the above-mentioned data packet forwarding rate is the rate at which the downstream switch forwards data packets to other devices, the above-mentioned total length of the data packet queue is the queue length of the data packets received and not forwarded by the downstream switch, the above-mentioned data packet arrival rate is the rate at which the data packets in the path reach the downstream switch, and the above-mentioned path data packet queue length is the queue length of the data packets in the path transmitted to the downstream switch and not forwarded.

[0072] Step 12: Based on the data packet forwarding rate and the data packet arrival rate of all paths, the congestion pause time difference of the downstream switch is calculated.

[0073] Specifically, through the formula:

[0074]

[0075] Calculate the congestion pause time difference Δt of the downstream switch.

[0076] Among them, Q PFC Indicates the total length of the queue pause of packets that trigger congestion pause, t d represents the current delay of transmitting messages to the upstream switch, n represents the number of paths, vi(t) represents the packet arrival rate of the ith path at the current moment, t represents the current moment, and vr(t) represents the packet forwarding rate.

[0077] It should be noted that the above congestion suspension time difference is the time difference from the current time to triggering the flow control algorithm (such as PFC suspension). For example, if the current time is 9 o'clock and the congestion suspension time difference is 10 minutes, the PFC suspension will be triggered at 9:10.

[0078] Exemplarily, the congestion pause time difference may be calculated using computer software for mathematical calculations such as MATLAB and Mathematica.

[0079] It is worth mentioning that the congestion pause time difference is calculated based on the current packet forwarding rate and packet arrival rate, which can monitor the congestion situation in real time and improve the real-time and accuracy of the congestion pause time difference.

[0080] Step 13: when the congestion pause time difference is less than the preset time difference, a path packet queue length threshold is obtained based on the total length of the packet queue, and a plurality of congested paths are determined from all paths using the path packet queue length threshold.

[0081] The path packet queue length of the congested path is greater than the path packet queue length threshold. The preset time difference may be a time difference set in the PFC mechanism.

[0082] In some embodiments of the present application, when the congestion suspension time difference is greater than or equal to the preset time difference, it indicates that the triggering of the PFC mechanism has not yet been determined. For example, if the preset time difference is 10 seconds, the PFC mechanism will be triggered after 10 seconds. However, at this time, the congestion suspension time difference is 30 seconds, which is greater than the preset time difference, and the PFC mechanism does not need to be triggered.

[0083] Specifically, through the formula:

[0084]

[0085] Calculate the path packet queue length threshold F cc ;

[0086] Among them, Q represents the total length of the packet queue, Q T It indicates recording the queue length of all path packets and the list of all paths. qInd indicates the queue index, and fNum indicates the number of paths in the queue index.

[0087] It should be noted that the above step of determining multiple congested paths from all paths using the path packet queue length threshold is specifically as follows: for each path, determine whether the path packet queue length of the path is greater than the path packet queue length threshold, and if so, take the path as a congested path. The above list is a data flow table flow-table (flowID, qInd, buff_pkts, sendRate, receiveRate, curr_time, td, Td) maintained by the downstream switch, where flowID is an identifier used to distinguish different paths, qInd is the index of the allocated queue, buff_pkts is the queue capacity occupied in the flowID path, sendRate is the arrival rate of the data packets in the path to the downstream switch inlet port, receiveRate is the forwarding rate of the data packets in the downstream switch inlet port queue, curr_time is the current time, td is the transmission delay of the path, and Td is the path delay of the path. The above queue index includes the identification information of all paths that are currently transmitting data packets.

[0088] Exemplarily, the path data packet queue length threshold may be calculated using computer software for mathematical calculations such as MATLAB and Mathematica.

[0089] It is worth mentioning that by obtaining the congested path according to the path packet queue length of each path, the path that needs to be load balanced at the current moment can be accurately located to avoid load balancing on normal paths.

[0090] Step 14: Obtain the path delay of each path based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths.

[0091] The above path delay is used to describe the delay of message transmission and the delay of packet transmission along the path.

[0092] In some embodiments of the present application, the step of obtaining the path delay of each path based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths is specifically as follows:

[0093] In the first step, the congestion pause recovery time is obtained based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths.

[0094] Specifically, through the formula:

[0095]

[0096] Calculate the congestion pause recovery time t re .

[0097] Among them, Q represents the total length of the packet queue, Indicates the total length of the packet queue restored at the end of the congestion pause, T re_flight The time when the message indicating the end of congestion pause arrives at the upstream switch, and ΔL indicates the gradient of queue length change:

[0098]

[0099] Where n represents the number of paths, vi(t) represents the packet arrival rate of the i-th path at the current moment, t represents the current moment, and vr(t) represents the packet forwarding rate.

[0100] The second step is to obtain the path delay of each path based on the congestion pause recovery time.

[0101] First, obtain the transmission delay of each path.

[0102] Exemplarily, the transmission delay of each path may be obtained by using a transmission performance testing tool such as iperf.

[0103] Then, for each path, perform the following steps:

[0104] Determine whether the path is a congested path. Specifically, if the path is determined to be a congested path in the above steps, then the path is a congested path.

[0105] If so, the sum of the congestion pause recovery time and the transmission delay of the path is taken as the path delay of the path.

[0106] Otherwise, the transmission delay of the path is taken as the path delay of the path.

[0107] Exemplarily, if the first path is a congested path, the path delay of the path is 2 seconds of the transmission delay of the path plus 3 seconds of congestion suspension recovery time, a total of 5 seconds. If the second path is not a congested path, the path delay of the path is equal to the transmission delay of the path, which is 3 seconds.

[0108] It should be noted that the above congestion pause recovery time is used to describe the time when the path can work normally after the congestion ends, that is, the total length of the packet queue of the ingress port of the downstream switch reaches the total length of the packet queue recovery, and the total time for the message of the end of the congestion pause to be transmitted to the upstream switch. The above total length of the packet queue recovery can be configured by the switch supporting PFC, and the time when the message of the end of the congestion pause arrives at the upstream switch is related to the hardware devices such as the forwarding rate and processing delay of the downstream switch.

[0109] It is worth mentioning that when obtaining the path delay of each path, the congestion condition of the path and the transmission delay of the path are taken into consideration, so that the path delay conforms to the actual condition of the path and improves the accuracy of the path delay.

[0110] Step 15: Determine multiple target paths from all paths using all path delays.

[0111] The path delay of each of the above target paths is smaller than the path delays of all other paths.

[0112] In some embodiments of the present application, the above step of determining multiple target paths from all paths using all path delays is specifically as follows:

[0113] In the first step, all path delays are arranged from small to large, and a preset number of path delays in the arrangement result are used as target path delays.

[0114] In the second step, the path corresponding to each target path delay is taken as a target path.

[0115] Exemplarily, three path delays are sorted: 3 seconds, 5 seconds, and 6 seconds, the first two path delays are taken as target path delays, and the path corresponding to the target path delay is taken as the target path, that is, the paths corresponding to 3 seconds and 5 seconds are both target paths.

[0116] It should be noted that the number of target paths can be set according to the number of congested paths.

[0117] It is worth mentioning that the target path is determined by using the path delay, and the changes in the packet queue of the path are taken into account, so that the obtained target path has better performance than other paths.

[0118] Step 16, sending a path congestion message to the upstream switch, so that the upstream switch uses all target paths to transmit data packets sent by all congested paths within a future preset time period according to the identification information of each congested path and the identification information of each target path carried in the path congestion message.

[0119] The above-mentioned path congestion message carries the identification information of each congested path and the identification information of each target path. The above-mentioned preset time period is the time when all congested paths return to normal. For example, at 9:40, the upstream switch receives the path congestion message and uses all target paths to transmit the data packets that need to be sent by all congested paths. At 10 o'clock, the path data packet queue lengths of all congested paths are less than or equal to the path data packet queue length threshold, indicating that all congested paths have returned to normal at this time and data transmission can continue. Then, the 20 minutes between 9:40 and 10 o'clock is the preset time period.

[0120] In some embodiments of the present application, after the downstream switch sends a path congestion message, the upstream switch receives the path congestion message and identifies all congested paths and all target paths in all paths based on the identification information of each congested path and the identification information of each target path, and then uses all target paths to transmit data packets sent to the downstream switch by all congested paths within a future preset time period.

[0121] It should be noted that the PFC mechanism can be used to implement the step of using all target paths to transmit data packets sent to downstream switches by all congested paths within a preset time period in the future, such as allocating target paths with small path delays to congested paths with high priorities, and using the target paths to transmit data packets that need to be sent by the corresponding congested paths until the upstream switch receives a message that the congested path has returned to normal and the pause has ended.

[0122] The load balancing method for RDMA network data transmission of the present application is exemplarily described below with reference to a specific example.

[0123] The load balancing system for RDMA network data transmission is as follows: Figure 2 As shown, the data flow is sent to the upstream source switch (i.e., the upstream switch mentioned above), the output port of the upstream source switch is connected to the input port of the downstream switch, the upstream source switch includes a rerouting module for receiving congestion notifications (i.e., the path congestion message mentioned above) and rerouting to the specified path (i.e., the target path mentioned above), the downstream destination switch includes a traffic prediction and monitoring module for predicting PFC suspension and recovery, identifying congested flows (i.e., the congested path mentioned above) and the optimal path (i.e., the target path mentioned above), and sending congestion notifications, where FCN is the congestion notification.

[0124] The data transmission process when the above load balancing system is working is as follows Figure 3 As shown in the figure, Spine is the upstream switch layer, Leaf is the downstream switch layer, the straight line between the upstream switch and the downstream switch represents the data transmission path, the arrow represents the direction of data transmission or message sending, f1 represents the first path, f2 represents the second path, and f n-1 represents the n-1th path, f n represents the nth path, Q PFC Indicates the total length of the queue pause for packets that trigger congestion pause. The formula for calculating the path packet queue length threshold in the above text is that the downstream switch ingress port sends FCN to the upstream switch egress port. FCN is a congestion notification. The upstream switch switches the congested flow (i.e., the congested path in the above text) to a new path (i.e., the target path in the above text) according to FCN.

[0125] It is worth mentioning that the congestion pause time difference is calculated based on the current packet forwarding rate and packet arrival rate, which can monitor the congestion situation in real time, improve the real-time and accuracy of the congestion pause time difference, and determine the target path based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate. The change of the packet queue of the path is taken into account, so that the performance of the obtained target path is better than that of other paths. At the same time, the path congestion message carrying the identification information of the target path and the identification information of the congested path is sent to the upstream switch, so that the upstream switch can simultaneously identify the congested path and the target path, which is convenient for the upstream switch to perform accurate load balancing, thereby improving the load balancing performance of data transmission.

[0126] The following is an exemplary description of the load balancing device for RDMA network data transmission provided by the present application.

[0127] like Figure 4 As shown, an embodiment of the present application provides a load balancing device for RDMA network data transmission, and the load balancing device 400 for RDMA network data transmission includes:

[0128] The first acquisition module 401 acquires the packet forwarding rate and total length of the packet queue of the downstream switch at the current moment, as well as the packet arrival rate and path packet queue length of each path at the current moment; a path is a link through which the upstream switch sends packets to the downstream switch;

[0129] A calculation module 402 calculates the congestion pause time difference of the downstream switch based on the packet forwarding rate and the packet arrival rate of all paths;

[0130] The first determination module 403, when the congestion suspension time difference is less than the preset time difference, obtains a path packet queue length threshold based on the total length of the packet queue, and determines multiple congested paths from all paths using the path packet queue length threshold; the path packet queue length of the congested path is greater than the path packet queue length threshold;

[0131] The second acquisition module 404 acquires the path delay of each path based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths; the path delay is used to describe the delay of message transmission and the delay of the path in data packet transmission;

[0132] The second determination module 405 determines multiple target paths from all paths using all path delays; the path delay of each target path is smaller than the path delays of all other paths;

[0133] The sending module 406 sends a path congestion message to the upstream switch, so that the upstream switch uses all target paths to transmit data packets sent by all congested paths within a future preset time period according to the identification information of each congested path and the identification information of each target path carried in the path congestion message.

[0134] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0135] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0136] like Figure 5 As shown, an embodiment of the present application provides a terminal device. The terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 5 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 implements the steps of any of the above-mentioned method embodiments when executing the computer program D102.

[0137] Specifically, when the processor D100 executes the computer program D102, the processor D100 obtains the packet forwarding rate and the total length of the packet queue of the downstream switch at the current moment, as well as the packet arrival rate and the path packet queue length of each path at the current moment, and then calculates the congestion pause time difference of the downstream switch based on the packet forwarding rate and the packet arrival rate of all paths. When the congestion pause time difference is less than the preset time difference, the path packet queue length threshold is obtained based on the total length of the packet queue, and multiple congested paths are determined from all paths using the path packet queue length threshold. Then, based on the total length of the packet queue, the packet forwarding rate, and the packet arrival rate of all paths, the path delay of each path is obtained, and then multiple target paths are determined from all paths using all path delays. Finally, a path congestion message is sent to the upstream switch, so that the upstream switch uses all target paths to transmit the data packets sent by all congested paths within a preset time period in the future according to the identification information of each congested path and the identification information of each target path carried in the path congestion message. Among them, the congestion pause time difference is calculated based on the data packet forwarding rate and data packet arrival rate at the current moment, so that the congestion situation can be monitored in real time, and the real-time and accuracy of the congestion pause time difference can be improved. The target path is determined based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate. The change of the data packet queue of the path is taken into account, so that the performance of the obtained target path is better than that of other paths. At the same time, the path congestion message carrying the identification information of the target path and the identification information of the congested path is sent to the upstream switch, so that the upstream switch can simultaneously identify the congested path and the target path, which is convenient for the upstream switch to perform accurate load balancing, thereby improving the load balancing performance of data transmission.

[0138] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0139] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0140] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0141] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0142] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium that can carry computer program code to the load balancing method device / terminal device of RDMA network data transmission. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0143] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0144] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0145] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A load balancing method for RDMA network data transmission, characterized in that: Applicable to downstream switches, including: Obtaining the packet forwarding rate and total length of the packet queue of the downstream switch at the current moment, as well as the packet arrival rate and path packet queue length of each path at the current moment; the path is a link through which the upstream switch sends packets to the downstream switch; Based on the data packet forwarding rate and the data packet arrival rate of all paths, the congestion pause time difference of the downstream switch is calculated; When the congestion pause time difference is less than the preset time difference, a path data packet queue length threshold is obtained based on the total length of the data packet queue, and a plurality of congested paths are determined from all paths using the path data packet queue length threshold; the path data packet queue length of the congested path is greater than the path data packet queue length threshold; Based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths, the path delay of each path is obtained; the path delay is used to describe the delay of message transmission and the delay of data packet transmission by the path; Determine multiple target paths from all paths using all path delays; the path delay of each of the target paths is less than the path delays of all other paths; Sending a path congestion message to an upstream switch, so that the upstream switch uses all target paths to transmit data packets sent by all congested paths within a future preset time period according to identification information of each congested path and identification information of each target path carried in the path congestion message; The step of calculating the congestion pause time difference of the downstream switch based on the data packet forwarding rate and the data packet arrival rate of all paths includes: By formula: Calculating the congestion pause time difference Δt of the downstream switch; Among them, Q PFC Indicates the total length of the queue pause of packets that trigger congestion pause, t d represents the current delay of transmitting messages to the upstream switch, n represents the number of paths, vi(t) represents the packet arrival rate of the ith path at the current moment, t represents the current moment, and vr(t) represents the packet forwarding rate; The acquiring a path data packet queue length threshold based on the total length of the data packet queue comprises: By formula: Calculate the path packet queue length threshold F cc ; Wherein, Q represents the total length of the data packet queue, QT represents a list recording the lengths of all path data packet queues and all paths, qInd represents a queue index, and fNum represents the number of paths in the queue index; The acquiring the path delay of each path based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths includes: Obtaining a congestion pause recovery time based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths; Based on the congestion pause recovery time, obtaining a path delay of each path; The acquiring the congestion suspension recovery time based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths includes: By formula: Calculate the congestion pause recovery time t re ; Wherein, Q represents the total length of the packet queue, Indicates the total length of the packet queue restored at the end of the congestion pause, T re_flight The time when the message indicating the end of congestion pause arrives at the upstream switch, and ΔL indicates the gradient of queue length change: Wherein, n represents the number of paths, vi(t) represents the packet arrival rate of the i-th path at the current moment, t represents the current moment, and vr(t) represents the packet forwarding rate; The acquiring the path delay of each path based on the congestion suspension recovery time includes: Get the transmission delay of each path; For each path, perform the following steps: Determining whether the path is a congested path; If yes, taking the sum of the congestion suspension recovery time and the transmission delay of the path as the path delay of the path; Otherwise, the transmission delay of the path is taken as the path delay of the path.

2. The load balancing method according to claim 1, characterized in that: The method of determining multiple target paths from all paths by utilizing all path delays includes: Arrange all path delays from small to large, and use the path delays of a preset number in the arrangement result as the target path delay; A path corresponding to each target path delay is respectively used as a target path.

3. A load balancing device for RDMA network data transmission, characterized in that: include: The first acquisition module acquires the packet forwarding rate and the total length of the packet queue of the downstream switch at the current moment, as well as the packet arrival rate and the path packet queue length of each path at the current moment; the path is a link through which the upstream switch sends a packet to the downstream switch; A calculation module, based on the data packet forwarding rate and the data packet arrival rate of all paths, calculates the congestion pause time difference of the downstream switch; A first determination module, when the congestion pause time difference is less than a preset time difference, obtains a path data packet queue length threshold based on the total length of the data packet queue, and determines multiple congested paths from all paths using the path data packet queue length threshold; the path data packet queue length of the congested path is greater than the path data packet queue length threshold; A second acquisition module, based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths, acquires the path delay of each path; the path delay is used to describe the delay of message transmission and the delay of path data packet transmission; A second determination module determines a plurality of target paths from all paths using all path delays; the path delay of each of the target paths is smaller than the path delays of all other paths; A sending module, which sends a path congestion message to an upstream switch, so that the upstream switch uses all target paths to transmit data packets sent by all congested paths within a future preset time period according to the identification information of each congested path and the identification information of each target path carried in the path congestion message; Wherein, the calculation module is specifically used for: By formula: Calculating the congestion pause time difference Δt of the downstream switch; Among them, Q PFC Indicates the total length of the queue pause of packets that trigger congestion pause, t d represents the current delay of transmitting messages to the upstream switch, n represents the number of paths, vi(t) represents the packet arrival rate of the ith path at the current moment, t represents the current moment, and vr(t) represents the packet forwarding rate; The first determination module is specifically used to implement: By formula: Calculate the path packet queue length threshold F cc ; Wherein, Q represents the total length of the packet queue, Q T It indicates recording the queue length of all path data packets and the list of all paths, qInd indicates the queue index, and fNum indicates the number of paths in the queue index; The second acquisition module is specifically used for: Obtaining a congestion pause recovery time based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths; Based on the congestion pause recovery time, obtaining a path delay of each path; The acquiring the congestion suspension recovery time based on the total length of the data packet queue, the data packet forwarding rate, and the data packet arrival rate of all paths includes: By formula: Calculate the congestion pause recovery time t re ; Wherein, Q represents the total length of the packet queue, Indicates the total length of the packet queue restored at the end of the congestion pause, T re_flight The time when the message indicating the end of congestion pause arrives at the upstream switch, and ΔL indicates the gradient of queue length change: Wherein, n represents the number of paths, vi(t) represents the packet arrival rate of the i-th path at the current moment, t represents the current moment, and vr(t) represents the packet forwarding rate; The acquiring the path delay of each path based on the congestion suspension recovery time includes: Get the transmission delay of each path; For each path, perform the following steps: Determining whether the path is a congested path; If yes, taking the sum of the congestion suspension recovery time and the transmission delay of the path as the path delay of the path; Otherwise, the transmission delay of the path is taken as the path delay of the path.

4. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the load balancing method for RDMA network data transmission according to any one of claims 1 to 2 is implemented.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the load balancing method for RDMA network data transmission as described in any one of claims 1 to 2 is implemented.