RDMA rapid congestion control method and system

By generating target congestion notification messages on network node devices and performing congestion control on source servers, the problem of low congestion control efficiency in the Ethernet environment of RDMA protocol is solved, and the support for SOCKET mode and the reduction of resource overhead are achieved, improving the improvement of The system's universality and throughput efficiency are achieved.

CN119071229AActive Publication Date: 2024-12-03FIBERHOME TELECOMMUNICATION TECHNOLOGIES CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202411078157.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-12-03
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

The existing RDMA protocol has low congestion control efficiency in Ethernet environments, and the fast CNP solution only supports CM mode and cannot support SOCKET mode, resulting in poor generalization and large resource overhead.

Method used

When the network node device receives the explicit congestion notification message ECN, the target congestion notification message CNP is generated, and the congestion control speed regulation algorithm is executed on the source server, and the WRED sampling probability is used for ECN marking, reducing the conversation of the network node device. Information extraction and information flow table creation support congestion control in SOCKET mode.

Benefits of technology

It improves the efficiency of congestion notification, reduces resource overhead, enhances the generality of the system, avoids the deterioration of congestion, and ensures the efficiency of the entire network throughput.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071229A_ABST
    Figure CN119071229A_ABST
Patent Text Reader

Abstract

An RDMA rapid congestion control method and system relates to the technical field of networks, and comprises the following steps: when a network node device receives a target explicit congestion notification message ECN, generating a target congestion notification message CNP based on a source IP, a destination IP and a destination queue number QP or a source QP number in the target ECN message, and sending the target congestion notification message CNP to a source server; when a source end server determines that a target CNP message is a host end CNP message based on a notification type field in the target CNP message, if a target source end QP number corresponding to a source IP, a target IP and a host end QP number in the target CNP message is inquired from a preset source and host end QP mapping table, the host end QP number in the host end CNP message is replaced with the target source end QP number, a new target CNP message is formed, and the target CNP message is sent to the source end server. And executing an original congestion control speed regulation algorithm based on the new target CNP message. According to the invention, the efficiency of congestion notification can be improved, the resource overhead is reduced, and the universality is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of network technology, and in particular to an RDMA fast congestion control method and system. Background Art

[0002] With the rapid development of information and communication technologies such as cloud computing, big data, artificial intelligence, and 5G, the trend of digital transformation in all walks of life has become increasingly obvious, and the wave of digital economy has become unstoppable. As the core resource in the digital economy era, computing power has become an important driving force for promoting high-quality economic development, and its role has become increasingly prominent. However, the explosive growth of data volume has undoubtedly put forward more stringent requirements for the construction of computing power networks. The traditional TCP / IP network architecture has problems such as low transmission performance and high latency, and can no longer meet the needs of computing power network construction. RDMA (Remote Direct Memory Access) is a network data transmission technology based on the IBA (InfiniBand Architecture) protocol stack architecture. It can complete direct access to remote memory without CPU (Central Processing Unit) intervention, and realizes high-throughput and low-latency network transmission. Therefore, it has gradually become the preferred technology for building high-performance computing power networks.

[0003] However, the RDMA protocol itself is developed from the InfiniBand architecture released by the InfiniBand Trade Association (IBTA), and the architecture is based on a hardware-supported lossless network, which results in relatively limited support for packet loss retransmission and congestion control in the design of the RDMA protocol; therefore, when the RDMA protocol runs on a lossy network such as Ethernet, once packet loss occurs, there will be a significant drop in transmission performance. Therefore, how to achieve efficient congestion control in the Ethernet-carrying RDMA solution to minimize packet loss is currently the main research direction in the industry.

[0004] In the related technology, the traditional CNP (Congestion Notification Packet) congestion control method in which congestion notification and congestion processing are implemented on the server side and the fast CNP congestion control method in which congestion notification is processed at the point where network congestion occurs are proposed. Among them, since the congestion notification in the traditional CNP congestion control method is implemented on the source server, as the network scale expands, the network congestion information feedback path will be too long, resulting in the failure of the congestion control strategy, and then the source server traffic cannot be slowed down in time, which leads to further congestion deterioration and causes the whole network throughput efficiency to decrease. The existing fast CNP method needs to extract RDMA session information in the RDMA link establishment stage and create an RDMA session information flow table in the network device nodes in the RDMA transmission path. However, for RDMA link establishment in CM mode and SOCKET mode, only the message format of CM mode is fixed, that is, session information extraction can only be performed in CM mode. Therefore, the existing fast CNP solution only supports CM mode but not SOCKET mode, which shows that its versatility has problems. At the same time, the RDMA session information flow table created in the network device node needs to be replicated on the entire RDMA transmission path. As the network node level increases, the number of information flow table entries that need to be saved will increase exponentially, which will greatly occupy the table hardware resources of the entire RDMA network, thereby increasing the overall deployment cost. Summary of the invention

[0005] The present application provides an RDMA fast congestion control method and system, which can improve the efficiency of congestion notification while reducing resource overhead and improving versatility.

[0006] In a first aspect, an embodiment of the present application provides an RDMA fast congestion control method, comprising the following steps:

[0007] When the network node device receives the target explicit congestion notification message ECN, it generates a target congestion notification message CNP based on the source IP, destination IP and sink queue number QP or source QP number in the target ECN message and sends it to the source server;

[0008] When the source server determines that the target CNP message is a destination CNP message based on the notification type field in the target CNP message, if the target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message is queried from the preset source-destination QP mapping table, the destination QP number in the target CNP message is replaced with the target source QP number to form a new target CNP message, and the original congestion control speed regulation algorithm is executed based on the new target CNP message.

[0009] In combination with the first aspect, in one implementation, before the step of when the network node device receives the target explicit congestion notification message ECN, it also includes:

[0010] Determine the ECN marking period based on the minimum ECN message sending period and the maximum transmission message length of the network node device under the remote direct memory access RDMA network congestion state;

[0011] The ECN marking period is used as the weighted random early discard (WRED) sampling probability;

[0012] When congestion occurs in the egress port queue of the network node device, ECN marking is performed on the congested messages in the egress port queue based on the WRED sampling probability to generate a target ECN message.

[0013] In combination with the first aspect, in one implementation, determining the ECN marking period based on the minimum ECN message sending period and the maximum transmission message length of the network node device in a remote direct memory access RDMA network congestion state includes:

[0014] Calculate the number of ECN messages that can be marked per second based on the minimum ECN message sending period;

[0015] Calculate the port message forwarding rate of the egress port when sending the maximum transmission message length according to the port rate of the egress port and the maximum transmission message length;

[0016] The ECN marking period is determined by the number of ECN packets that can be marked per second and the port packet forwarding rate.

[0017] In combination with the first aspect, in one implementation, generating the target congestion notification message CNP based on the source IP, destination IP and sink queue number QP or source QP number in the target ECN message includes:

[0018] When the target ECN message includes a source IP, a destination IP, and a sink QP number, a target CNP message is generated based on the source IP, the destination IP, and the sink QP number in the target ECN message, wherein the destination IP in the target CNP message is the source IP in the target ECN message, the source IP in the target CNP message is the destination IP in the target ECN message, the notification type field in the target CNP message is set to a first preset value, and the QP field is set to the sink QP number, and the first preset value is used to characterize that the target CNP message is a sink CNP message;

[0019] When the target ECN message includes a source IP, a destination IP and a source QP number, a target CNP message is generated based on the source IP, the destination IP and the source QP number in the target ECN message, wherein the notification type field in the target CNP message is set to a second preset value and the QP field is set to the source QP number, and the second preset value is used to characterize that the target CNP message is a source CNP message.

[0020] In combination with the first aspect, in one implementation, the method further includes:

[0021] When the source server determines that the target CNP message is a source CNP message based on the notification type field in the target CNP message, the original congestion control speed adjustment algorithm is executed based on the target CNP message.

[0022] In combination with the first aspect, in an implementation manner, after the step of determining that the target CNP message is a sink CNP message based on the notification type field in the target CNP message, the method further includes:

[0023] If the source server fails to find the target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message from the preset source-destination QP mapping table, the target CNP message is discarded.

[0024] In a second aspect, an embodiment of the present application provides an RDMA fast congestion control system, including:

[0025] A congestion notification unit, which is used to generate a target congestion notification message CNP based on the source IP, destination IP and sink queue number QP or source QP number in the target ECN message and send it to the source server when the network node device receives the target explicit congestion notification message ECN;

[0026] A congestion processing unit, which is used to replace the destination QP number in the target CNP message with the target source QP number to form a new target CNP message when the source server determines that the target CNP message is a destination CNP message based on the notification type field in the target CNP message, if a target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message is queried from a preset source-destination QP mapping table, to form a new target CNP message, and execute the original congestion control speed regulation algorithm based on the new target CNP message.

[0027] In conjunction with the second aspect, in one implementation, the system further includes a congestion marking unit, which is used to:

[0028] Determine the ECN marking period based on the minimum ECN message sending period and the maximum transmission message length of the network node device under the remote direct memory access RDMA network congestion state;

[0029] The ECN marking period is used as the weighted random early discard (WRED) sampling probability;

[0030] When congestion occurs in the egress port queue of the network node device, the congested packets in the egress port queue are ECN marked based on the WRED sampling probability to generate a target ECN packet.

[0031] In conjunction with the second aspect, in one implementation, the congestion marking unit is specifically configured to:

[0032] Calculate the number of ECN messages that can be marked per second based on the minimum ECN message sending period;

[0033] Calculate the port message forwarding rate of the egress port when sending the maximum transmission message length according to the port rate of the egress port and the maximum transmission message length;

[0034] The ECN marking period is determined by the number of ECN packets that can be marked per second and the port packet forwarding rate.

[0035] In conjunction with the second aspect, in one implementation, the congestion notification unit is specifically configured to:

[0036] When the target ECN message includes a source IP, a destination IP, and a sink QP number, a target CNP message is generated based on the source IP, the destination IP, and the sink QP number in the target ECN message, wherein the destination IP in the target CNP message is the source IP in the target ECN message, the source IP in the target CNP message is the destination IP in the target ECN message, the notification type field in the target CNP message is set to a first preset value, and the QP field is set to the sink QP number, and the first preset value is used to characterize that the target CNP message is a sink CNP message;

[0037] When the target ECN message includes a source IP, a destination IP and a source QP number, a target CNP message is generated based on the source IP, the destination IP and the source QP number in the target ECN message, wherein the notification type field in the target CNP message is set to a second preset value and the QP field is set to the source QP number, and the second preset value is used to characterize that the target CNP message is a source CNP message.

[0038] In combination with the second aspect, in one implementation, the congestion processing unit is further used to: when the source server determines that the target CNP message is a source CNP message based on the notification type field in the target CNP message, execute an original congestion control speed adjustment algorithm based on the target CNP message.

[0039] In combination with the second aspect, in one embodiment, the congestion processing unit is also used to: if the source server fails to query the target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message from the preset source-destination QP mapping table, then the target CNP message is discarded.

[0040] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0041] When the network node device receives the target explicit congestion notification message ECN, a target congestion notification message CNP is generated based on the source IP, destination IP and destination queue number QP or source QP number in the target ECN message and sent to the source server; when the source server determines that the target CNP message is a destination CNP message based on the notification type field in the target CNP message, if the target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message is queried from the preset source-destination QP mapping table, the destination QP number in the target CNP message is replaced with the target source QP number to form a new target CNP message, and the original congestion control speed regulation algorithm is executed based on the new target CNP message. The present application improves the efficiency of congestion notification by performing congestion notification in the network node device, and does not require the extraction of session information and the creation of RDMA information flow tables in the network node device, which effectively improves versatility and reduces resource overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of an embodiment of the RDMA fast congestion control method of the present application;

[0043] Figure 2 A schematic diagram of the Xwred ECN marking probability involved in the embodiment of the present application;

[0044] Figure 3 This is a schematic diagram of the ECN marking algorithm involved in the embodiment of the present application;

[0045] Figure 4 This is a schematic diagram of the source-destination CNP notification message format involved in the embodiment of the present application;

[0046] Figure 5 This is a schematic diagram of the source-destination QP mapping involved in the embodiment of the present application;

[0047] Figure 6 A schematic diagram of the congestion handling process involved in the embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0050] In a first aspect, an embodiment of the present application provides an RDMA fast congestion control method.

[0051] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the RDMA fast congestion control method of the present application. Figure 1 As shown, the RDMA fast congestion control method includes:

[0052] Step S10: When the network node device receives the target explicit congestion notification message ECN, it generates a target congestion notification message CNP based on the source IP, destination IP and sink queue number QP or source QP number in the target ECN message and sends it to the source server.

[0053] For example, it is understandable that, at present, the RoCE (RDMA over Converged Ethernet) protocol solution based on RDMA migration to Ethernet based on hardware lossless network design has the characteristics of low cost and wide versatility, but the inherent unreliable transmission characteristics of Ethernet will cause the Ethernet packet loss rate to be much higher than that of InfiniBand network. Therefore, the packet loss retransmission and congestion control mechanism of the native RDMA protocol is only suitable for InfiniBand network, and when the Ethernet packet loss rate increases, the transmission performance will be greatly reduced. In order to solve the related problems, the industry has proposed the traditional CNP congestion control method and the fast CNP congestion control method, but the traditional CNP congestion control method has the problem of low congestion notification efficiency, and the fast CNP congestion control method has the problem of poor versatility and high resource overhead.

[0054] In this embodiment, in order to solve the problem of low efficiency of congestion notification, the CNP message will be assembled through the network node device and sent directly to the source server to realize congestion notification, without the need to transmit the CNP message to the source server through the destination server, thereby shortening the feedback path of network congestion information and preventing the failure of the congestion control strategy, so that the source server traffic can be slowed down in time to avoid congestion deterioration and ensure the throughput efficiency of the entire network. Specifically, when the network node device receives the target ECN message marked with ECN, the target CNP message will be directly generated and sent to the source server through the source IP, destination IP and destination QP (Queue Pair) number or source QP number in the target ECN message, without extracting session information and creating an RDMA information flow table on the network node device. It can be understood that the target CNP message will integrate the source IP, destination IP and destination QP, or integrate the source IP, destination IP and source QP number.

[0055] Furthermore, in one embodiment, before the step of when the network node device receives the target explicit congestion notification message ECN, it also includes:

[0056] Determine the ECN marking period based on the minimum ECN message sending period and the maximum transmission message length of the network node device under the remote direct memory access RDMA network congestion state;

[0057] The ECN marking period is used as the weighted random early discard (WRED) sampling probability;

[0058] When congestion occurs in the egress port queue of the network node device, ECN marking is performed on the congested messages in the egress port queue based on the WRED sampling probability to generate a target ECN message.

[0059] Exemplarily, in this embodiment, on each network node device of the RDMA transmission network, the ECN marking interval (i.e., the ECN marking period Tecn) is calculated based on the shortest time interval for sending the congestion control notification CNP message (i.e., the minimum ECN message sending period Tmin in the RDMA network congestion state, such as 50ms) and the maximum transmission message length MTU of the egress port of the network node device; then the ECN marking period Tecn is converted into a WRED (Weighted Random Early Detection) sampling probability Xwred (i.e., Figure 2The marking rate in the WRED sampling probability Xwred is sent down through the port for hardware configuration to implement the congestion marking ECN function with the shortest time interval for sending CNP messages being Tmin. That is, when congestion occurs in the egress port queue of the network node device, the congested messages in the egress port queue are ECN marked based on the WRED sampling probability to generate a target ECN message.

[0060] Among them, see Figure 2 As shown, it can be understood that when the cache utilization is less than or equal to the preset low threshold, it means that there is enough cache for data transmission, that is, there is no congestion problem in the outbound port queue, and the marking rate is 0 at this time. In other words, it is not necessary to perform ECN marking on the outbound port message; and if the cache utilization is between the preset low threshold and the preset high threshold, it means that although there is a certain amount of cache for data transmission, the remaining cache is insufficient, that is, there is a partial congestion problem in the outbound port queue, and some congested messages can be marked with ECN according to the calculated marking rate; in addition, if the cache utilization is equal to the high threshold, it means that the current remaining cache is far from meeting the demand for data transmission, that is, the outbound port queue is seriously congested, and it is necessary to perform ECN marking on the congested message according to the maximum marking rate; if the cache utilization continues to be greater than the high threshold, all congested files are marked with ECN according to the marking rate of 100%. It should be noted that the specific values ​​of the low threshold and the high threshold can be determined according to actual needs and are not limited here.

[0061] Further, in one embodiment, the determining of the ECN marking period based on the minimum ECN message sending period and the maximum transmission message length of the network node device in the remote direct memory access RDMA network congestion state includes:

[0062] Calculate the number of ECN messages that can be marked per second based on the minimum ECN message sending period;

[0063] Calculate the port message forwarding rate of the egress port when sending the maximum transmission message length according to the port rate of the egress port and the maximum transmission message length;

[0064] The ECN marking period is determined by the number of ECN packets that can be marked per second and the port packet forwarding rate. For example, see Figure 3As shown, first, the minimum ECN message sending period Tecn under the RDMA network congestion state is configured, and the number of ECN messages that can be marked per second PPSecn is calculated based on the minimum ECN message sending period Tecn (i.e., PPSecn=1 / Tecn); then the port rate of the network node device egress port and the maximum transmission message length MTU are obtained to calculate the port message forwarding rate PPSmtu corresponding to the port when sending a message with a length of MTU; then the WRED function is enabled on the egress port, and the ECN marking period is determined by PPSecn and PPSmtu, and the ECN marking period is used as the WRED sampling probability Xwred (i.e., Xwred=PPSecn / PPSmtu); finally, the congestion coloring ratio of the egress port related queue ECN is set to Xwred, so as to achieve the purpose of converting the congestion marking time interval into the WRED coloring ratio, so that when the egress port queue is congested, the WRED function can perform ECN coloring on the congested message according to the WRED sampling probability setting to obtain the target ECN message, and send the target ECN message to the next hop.

[0065] It can be understood that when the existing fast CNP congestion control method performs congestion notification through the network node device, it is not only necessary to query the local QP flow table before packet forwarding, but also to start the hardware timer for each QP to calculate the CNP packet interval; in this embodiment, there is no need to extract session information and create an RDMA information flow table on the network node device, and because the CNP message sending interval is converted into an ECN marking interval, the network node device can perform CNP message packetization and forwarding after receiving the ECN message without being restricted by the hardware timer, that is, there is no need to set the CNP message time interval timer to calculate the CNP packet interval, which effectively simplifies the CNP message sending process and can reach the line speed; and when a congestion state occurs, the ECN marking interval can be achieved through the ECN coloring ratio corresponding to WRED, and the processing remains unchanged when there is no congestion or the congestion reaches the upper limit, so that the ECN marking does not need to be restricted by the QP type.

[0066] Step S20: When the source server determines that the target CNP message is a destination CNP message based on the notification type field in the target CNP message, if the target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message is queried from the preset source-destination QP mapping table, the destination QP number in the target CNP message is replaced with the target source QP number to form a new target CNP message, and the original congestion control speed regulation algorithm is executed based on the new target CNP message.

[0067] For example, in this embodiment, in order to be compatible with and hybrid deploy existing congestion control solutions, a compatible mode will be adopted in the message format, that is, the source / sink RDMA congestion notification message format is improved to achieve compatibility. In the original congestion control process, only the source server will parse the 8-11 bytes of the Reserved field in the BTH message header format of the RDMA ROCEv2 message, while the intermediate node will not parse it. Therefore, this embodiment will Figure 4 The 8-11-byte Reserved field in the BTH message header format of the RDMA ROCEv2 message shown is extended to achieve compatibility.

[0068] Specifically, the 4 bits in the Reserved field are used as the identification field of the source and sink CNP notification type. The notification type field is used to indicate the CNP notification type. For example, the source CNP notification is set to 0, and the sink CNP notification is set to 1. The remaining types are reserved and expandable to maintain compatibility with the existing congestion control scheme field definition. It should be noted that this field can also be used to identify the ECN mark, etc., which is not limited here. It can be understood that if the notification type field is set to 0, it means that the corresponding CNP message is the source CNP message, and if the notification type field is set to 1, it means that the corresponding CNP message is the sink CNP message.

[0069] In addition, the 3 bits in the Reserved field can also be used as an identification field of the CNP service message type. The service message type field is used to indicate the congestion control algorithm type of the CNP. For example, the existing CNP scheme type is set to 0, and the CNP scheme provided in this embodiment is set to 1. The remaining types are reserved for expansion and remain compatible with the existing congestion control scheme field definition.

[0070] Therefore, after receiving the target CNP message sent by the network node device, the remote server can determine whether the target CNP message is a destination CNP message or a source CNP message based on the notification type field in the target CNP message; if the target CNP message is a destination CNP message, the preset source destination QP mapping table is queried according to the source IP, destination IP and destination QP number in the target CNP message; when the target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message is queried, the destination QP number in the target CNP message is replaced with the target source QP number to form a new target CNP message; then, based on the target source QP number in the new target CNP message, the original congestion control speed regulation algorithm can be executed, that is, the network card end RP (Rendezvous Point) node congestion control speed regulation algorithm is executed. It should be noted that the principle and process of the original congestion control speed regulation algorithm are common knowledge in the art, and for the sake of brevity of description, they will not be repeated here.

[0071] In addition, it can be understood that the initiation and acceptance of the RDMA transmission task are both on the source server network card, and the source server network card stores the source IP address (i.e., source IP), sink IP address (i.e., destination IP), sink QP number, source QP number and other information of the RDMA session. Therefore, this embodiment will create a mapping table (i.e., source QP and sink QP number) between the source QP and sink QP of the RDMA transmission drawing on the source server network card based on the above information. Figure 5 The source-destination QP mapping table shown in the figure is used to convert the destination CNP message into the source CNP message; it can be understood that the table entry fields are the source IP address, the source QP number, the destination IP address, and the destination QP number, wherein the source IP address, the destination IP address, and the destination QP are search keywords.

[0072] It can be seen that this embodiment improves the efficiency of congestion notification by performing congestion notification in the network node device, and there is no need to extract session information and create RDMA information flow tables in the network node device. It not only solves the problem that the existing solution does not support fast CNP message generation under the RDMA SOCKET link building mode, effectively improving the versatility, but also solves the problem that the flow table specifications required for intermediate network nodes to support fast CNP increase exponentially with the increase of network levels, and the huge resource overhead problem caused by the need to store the RDMA flow table of the same end-side node in the network device of the entire link, effectively reducing resource overhead.

[0073] In addition, in the existing fast CNP congestion control method, congestion processing needs to be implemented jointly by the network node device and the end-side device, while the congestion processing in this embodiment only needs to be implemented on the end-side device alone, which greatly reduces the table item requirements for the network node device, thereby greatly reducing the network device specification requirements and reducing the networking cost. For example, taking a typical small and medium-sized spine-leaf architecture data center (for example, including 4 spines (core layers), 32 leafs (access layers), each leaf is connected to 40 servers, a total of 1280 servers, and each server has 2000 QPs) as an example, a comparative analysis of this embodiment and the existing fast CNP congestion control method is conducted from the following three aspects: (1) For the required table item specifications, this embodiment has 2000 end nodes, while the existing fast CNP congestion control method corresponds to 2000*40=80000 Leaf nodes and 80000 spine nodes. *32=2560000; (2) For the required table entry learning time, this implementation is based on existing information creation and does not require learning. According to the learning rate of 4000 items / s, the table entry learning time corresponding to the existing fast CNP congestion control method is 20s to 640s; (3) For the table entry query time, taking the balanced binary tree storage scheme with a complexity of log2n as an example, the table entry query time required by this embodiment is Log2(2000)=11s, while the table entry query time corresponding to the existing fast CNP congestion control method is log2(80000)=16s and log2(2560000)=21s. Therefore, this embodiment greatly reduces the table entry requirements for network node devices.

[0074] In summary, this embodiment improves the efficiency of congestion notification and reduces the resource overhead of network intermediate node devices, while being compatible with the two existing CNP notification methods, thereby meeting the demand for smooth network upgrade.

[0075] Further, in one embodiment, the generating of the target congestion notification message CNP based on the source IP, the destination IP and the destination queue number QP or the source queue number QP in the target ECN message includes:

[0076] When the target ECN message includes a source IP, a destination IP, and a sink QP number, a target CNP message is generated based on the source IP, the destination IP, and the sink QP number in the target ECN message, wherein the destination IP in the target CNP message is the source IP in the target ECN message, the source IP in the target CNP message is the destination IP in the target ECN message, the notification type field in the target CNP message is set to a first preset value, and the QP field is set to the sink QP number, and the first preset value is used to characterize that the target CNP message is a sink CNP message;

[0077] When the target ECN message includes a source IP, a destination IP and a source QP number, a target CNP message is generated based on the source IP, the destination IP and the source QP number in the target ECN message, wherein the notification type field in the target CNP message is set to a second preset value and the QP field is set to the source QP number, and the second preset value is used to characterize that the target CNP message is a source CNP message.

[0078] By way of example, it can be understood that a congestion notification unit is deployed on each network node device of the RDMA transmission network. Since the CNP message sending interval has been converted into the WRED sampling probability Xwred in the congestion notification unit, for the received RDMA service ECN message (i.e., the target ECN message), the network node device where the congestion occurs can directly perform CNP packetization and sending processing.

[0079] It should be noted that the specific values ​​of the first preset value and the second preset value can be determined according to actual needs and are not limited here. Specifically, taking the first preset value as 1 and the second preset value as 0 as an example, the network node device deployed with the congestion notification unit will extract the source IP, destination IP, destination QP number or source QP number in the target ECN message and generate a target CNP message according to the standard CNP message format; wherein, if the target ECN message only contains the source IP, destination IP and destination QP number, the target CNP message is composed according to the source IP, destination IP and destination QP number, and the notification type field in the target CNP message is set to 1 to indicate that the notification type of the message is the destination CNP type; at the same time, the QP field in the BTH header of the target CNP message is set to the destination QP number, and the source IP in the IP header of the target CNP message is filled in with the destination IP in the target ECN message, and the destination IP in the IP header of the target CNP message is filled in with the source IP in the target ECN message, and then the generated target CNP message is sent to the source server.

[0080] If the target ECN message only includes the source IP, destination IP and source QP number, the target CNP message is composed according to the source IP, destination IP and source QP number, and the announcement type field in the target CNP message is set to 0 to indicate that the announcement type of the message is the source CNP type; at the same time, the QP field in the BTH header of the target CNP message is set to the source QP number, while the source IP and destination IP in the IP header of the target CNP message remain unchanged, and then the generated target CNP message is sent to the source server.

[0081] Furthermore, in one embodiment, the method further includes:

[0082] When the source server determines that the target CNP message is a source CNP message based on the notification type field in the target CNP message, the original congestion control speed adjustment algorithm is executed based on the target CNP message.

[0083] For example, it should be understood that in this embodiment, a congestion processing unit is deployed on the source server network card, which is mainly used to process the target CNP message. Figure 6 As shown, when the source server receives the target CNP message, the notification type flag is used to judge the notification type of the target CNP message. If the target CNP message is the source CNP message, that is, the QP number contained in the target CNP message is the source QP number, the regular speed reduction process is entered and the source QP regular congestion processing process is executed.

[0084] If the target CNP message is a destination CNP message, that is, the QP number contained in the target CNP message is the destination QP number, it is necessary to enter the QP pair conversion process, that is, use the destination IP (that is, the source IP in the destination CNP message), the source IP (that is, the destination IP in the destination CNP message) and the destination CNP (that is, the destination QP number) as keywords to find out whether there is a matching entry in the source-destination QP mapping table; and after finding the target source QP number corresponding to the destination QP number, that is, there is a matching entry, the destination QP number in the target CNP message is replaced with the target source QP number in the matching entry, and then enter the regular speed reduction process and execute the source QP regular congestion processing process, so as to achieve the purpose of compatibility with the existing congestion control message processing process.

[0085] Further, in one embodiment, after the step of determining that the target CNP message is a sink CNP message based on the notification type field in the target CNP message, the step further includes:

[0086] If the source server fails to find the target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message from the preset source-destination QP mapping table, the target CNP message is discarded.

[0087] Exemplarily, in this embodiment, if there is no matching entry corresponding to the destination QP number in the target CNP message in the source-destination QP mapping table on the source server network card, that is, no relevant source QP number information is found, then the target CNP message is identified as an illegal destination CNP message and needs to be discarded.

[0088] In summary, this embodiment, based on the newly added source-destination congestion notification message format, proposes a new congestion marking and congestion notification implementation method on the network node device where congestion occurs, so as to ensure the timeliness of congestion marking and notification while eliminating the need to create an RDMA information flow table on the network transmission node device; and proposes a new congestion processing implementation method compatible with the source-destination congestion notification message format on the source-end server node device, thereby being able to adapt to all two RDMA transmission link establishment modes while ensuring the timeliness of congestion marking, congestion notification, and congestion processing in RDMA transmission networks of various sizes, so as to reduce the hardware table entry requirements for intermediate node devices in the RDMA transmission network and be compatible with various existing congestion control schemes, so that they can be mixed and deployed on the same network with maximum versatility.

[0089] In a second aspect, an embodiment of the present application also provides an RDMA fast congestion control system.

[0090] In one embodiment, the RDMA fast congestion control system includes:

[0091] A congestion notification unit, which is used to generate a target congestion notification message CNP based on the source IP, destination IP and sink queue number QP or source QP number in the target ECN message and send it to the source server when the network node device receives the target explicit congestion notification message ECN;

[0092] A congestion processing unit, which is used to replace the destination QP number in the target CNP message with the target source QP number to form a new target CNP message when the source server determines that the target CNP message is a destination CNP message based on the notification type field in the target CNP message, if a target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message is queried from a preset source-destination QP mapping table, to form a new target CNP message, and execute the original congestion control speed regulation algorithm based on the new target CNP message.

[0093] Furthermore, in one embodiment, the system further includes a congestion marking unit, which is used to:

[0094] Determine the ECN marking period based on the minimum ECN message sending period and the maximum transmission message length of the network node device under the remote direct memory access RDMA network congestion state;

[0095] The ECN marking period is used as the weighted random early discard (WRED) sampling probability;

[0096] When congestion occurs in the egress port queue of the network node device, the congested packets in the egress port queue are ECN marked based on the WRED sampling probability to generate a target ECN packet.

[0097] Furthermore, in one embodiment, the congestion marking unit is specifically used to:

[0098] Calculate the number of ECN messages that can be marked per second based on the minimum ECN message sending period;

[0099] Calculate the port message forwarding rate of the egress port when sending the maximum transmission message length according to the port rate of the egress port and the maximum transmission message length;

[0100] The ECN marking period is determined by the number of ECN packets that can be marked per second and the port packet forwarding rate.

[0101] Furthermore, in one embodiment, the congestion notification unit is specifically used to:

[0102] When the target ECN message includes a source IP, a destination IP, and a sink QP number, a target CNP message is generated based on the source IP, the destination IP, and the sink QP number in the target ECN message, wherein the destination IP in the target CNP message is the source IP in the target ECN message, the source IP in the target CNP message is the destination IP in the target ECN message, the notification type field in the target CNP message is set to a first preset value, and the QP field is set to the sink QP number, and the first preset value is used to characterize that the target CNP message is a sink CNP message;

[0103] When the target ECN message includes a source IP, a destination IP and a source QP number, a target CNP message is generated based on the source IP, the destination IP and the source QP number in the target ECN message, wherein the notification type field in the target CNP message is set to a second preset value and the QP field is set to the source QP number, and the second preset value is used to characterize that the target CNP message is a source CNP message.

[0104] Furthermore, in one embodiment, the congestion processing unit is further configured to: when the source server determines that the target CNP message is a source CNP message based on the notification type field in the target CNP message, execute an original congestion control speed adjustment algorithm based on the target CNP message.

[0105] Furthermore, in one embodiment, the congestion processing unit is also used to: if the source server fails to find a target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message from a preset source-destination QP mapping table, then discard the target CNP message.

[0106] Among them, the functional implementation of each module in the above-mentioned RDMA fast congestion control system corresponds to each step in the above-mentioned RDMA fast congestion control method embodiment, and its functions and implementation processes are no longer repeated here.

[0107] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0108] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0109] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0110] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0111] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0113] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A fast congestion control method for RDMA, It is characterized in that The following steps are involved: When the network node device receives the target explicit congestion notification message ECN, it generates a target congestion notification message CNP based on the source IP, destination IP and sink queue number QP or source QP number in the target ECN message and sends it to the source server; When the source server determines that the target CNP message is a destination CNP message based on the notification type field in the target CNP message, if the target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message is queried from the preset source-destination QP mapping table, the destination QP number in the target CNP message is replaced with the target source QP number to form a new target CNP message, and the original congestion control speed regulation algorithm is executed based on the new target CNP message.

2. The RDMA fast congestion control method according to claim 1, It is characterized in that Before the step of when the network node device receives the target explicit congestion notification message ECN, it also includes: Determine the ECN marking period based on the minimum ECN message sending period and the maximum transmission message length of the network node device under the remote direct memory access RDMA network congestion state; The ECN marking period is used as the weighted random early discard (WRED) sampling probability; When congestion occurs in the egress port queue of the network node device, ECN marking is performed on the congested messages in the egress port queue based on the WRED sampling probability to generate a target ECN message.

3. The RDMA fast congestion control method as claimed in claim 2, It is characterized in that The method of determining the ECN marking period based on the minimum ECN message sending period and the maximum transmission message length of the network node device in the remote direct memory access RDMA network congestion state includes: Calculate the number of ECN messages that can be marked per second based on the minimum ECN message sending period; Calculate the port message forwarding rate of the egress port when sending the maximum transmission message length according to the port rate of the egress port and the maximum transmission message length; The ECN marking period is determined by the number of ECN packets that can be marked per second and the port packet forwarding rate.

4. The RDMA fast congestion control method according to claim 1, It is characterized in that The generating of the target congestion notification message CNP based on the source IP, the destination IP and the destination queue number QP or the source queue number QP in the target ECN message includes: When the target ECN message includes a source IP, a destination IP, and a sink QP number, a target CNP message is generated based on the source IP, the destination IP, and the sink QP number in the target ECN message, wherein the destination IP in the target CNP message is the source IP in the target ECN message, the source IP in the target CNP message is the destination IP in the target ECN message, the notification type field in the target CNP message is set to a first preset value, and the QP field is set to the sink QP number, and the first preset value is used to characterize that the target CNP message is a sink CNP message; When the target ECN message includes a source IP, a destination IP and a source QP number, a target CNP message is generated based on the source IP, the destination IP and the source QP number in the target ECN message, wherein the notification type field in the target CNP message is set to a second preset value and the QP field is set to the source QP number, and the second preset value is used to characterize that the target CNP message is a source CNP message.

5. The RDMA fast congestion control method according to claim 1, It is characterized in that The method further comprises: When the source server determines that the target CNP message is a source CNP message based on the notification type field in the target CNP message, the original congestion control speed adjustment algorithm is executed based on the target CNP message.

6. The RDMA fast congestion control method according to claim 1, It is characterized in that After the step of determining that the target CNP message is a sink CNP message based on the notification type field in the target CNP message by the source server, the method further includes: If the source server fails to find the target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message from the preset source-destination QP mapping table, the target CNP message is discarded.

7. A RDMA fast congestion control system, It is characterized in that include: A congestion notification unit, which is used to generate a target congestion notification message CNP based on the source IP, destination IP and sink queue number QP or source QP number in the target ECN message and send it to the source server when the network node device receives the target explicit congestion notification message ECN; A congestion processing unit, which is used to replace the destination QP number in the target CNP message with the target source QP number to form a new target CNP message when the source server determines that the target CNP message is a destination CNP message based on the notification type field in the target CNP message, if a target source QP number corresponding to the source IP, destination IP and destination QP number in the target CNP message is queried from a preset source-destination QP mapping table, to form a new target CNP message, and execute the original congestion control speed regulation algorithm based on the new target CNP message.

8. The RDMA fast congestion control system as claimed in claim 7, It is characterized in that The system further comprises a congestion marking unit, which is configured to: Determine the ECN marking period based on the minimum ECN message sending period and the maximum transmission message length of the network node device under the remote direct memory access RDMA network congestion state; The ECN marking period is used as the weighted random early discard (WRED) sampling probability; When congestion occurs in the egress port queue of the network node device, the congested packets in the egress port queue are ECN marked based on the WRED sampling probability to generate a target ECN packet.

9. The RDMA fast congestion control system as claimed in claim 8, It is characterized in that The congestion marking unit is specifically used for: Calculate the number of ECN messages that can be marked per second based on the minimum ECN message sending period; Calculate the port message forwarding rate of the egress port when sending the maximum transmission message length according to the port rate of the egress port and the maximum transmission message length; The ECN marking period is determined by the number of ECN packets that can be marked per second and the port packet forwarding rate.

10. The RDMA fast congestion control system according to claim 7, It is characterized in that The congestion notification unit is specifically used for: When the target ECN message includes a source IP, a destination IP, and a sink QP number, a target CNP message is generated based on the source IP, the destination IP, and the sink QP number in the target ECN message, wherein the destination IP in the target CNP message is the source IP in the target ECN message, the source IP in the target CNP message is the destination IP in the target ECN message, the notification type field in the target CNP message is set to a first preset value, and the QP field is set to the sink QP number, and the first preset value is used to characterize that the target CNP message is a sink CNP message; When the target ECN message includes a source IP, a destination IP and a source QP number, a target CNP message is generated based on the source IP, the destination IP and the source QP number in the target ECN message, wherein the notification type field in the target CNP message is set to a second preset value and the QP field is set to the source QP number, and the second preset value is used to characterize that the target CNP message is a source CNP message.

Citation Information

Patent Citations

  • ECN quick response method and device

    CN110784415A

  • Network congestion notification method, proxy node, network node and computer equipment

    CN113709057A

  • Improved method for fast CNP congestion notification in network

    CN115695314A

  • Method and device for marking ECN in RoCE network, electronic equipment and storage medium

    CN116318626A

  • Network congestion notification method and device and storage medium

    CN117221225A