RoCE network congestion control method and related device
By aggregating the indications and congestion information in the acknowledgement message in the RoCE network, the problem of congestion notification delay in the traditional RoCE network is solved, and faster congestion control and bandwidth utilization are achieved.
Patent Information
- Application Number
- CN202010915720.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-03
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-09-03
AI Technical Summary
When a RoCE network is congested, the traditional method of single CNP message notification causes delay in congestion notification, the source device responds slowly, and the transmission rate cannot be restored in time, affecting the network bandwidth utilization rate.
By aggregating indication information and congestion information in the confirmation message to avoid sending independent CNP messages, the source device can obtain the network status in time and perform congestion control, using detailed congestion information (such as congestion degree, location, queue depth, time Delay) to diversify the transmission rate.
It reduces the delay of congestion notification, improves the response speed and network bandwidth utilization of source devices, and achieves faster congestion control and rate convergence.
Smart Images

Figure CN114143827B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communication technology, and in particular to a method and related device for controlling RoCE network congestion. Background Art
[0002] In data communication systems, in order to increase the speed of message transmission between computing devices, Remote Direct Memory Access (RDMA) technology is usually used for connection. RDMA technology directly transfers data to the computer's storage area through the network, quickly moving data from one system to the remote system memory without the intervention of the operating system or kernel of the two computing devices. RDMA eliminates the overhead of external memory copying and context switching, thereby freeing up memory bandwidth and CPU cycles to improve application system performance.
[0003] Remote Direct Memory Access over Converged Ethernet (RDMA over Converged Ethernet, RoCE) is a type of RDMA technology that allows servers to perform remote direct memory access over Ethernet. Although the advantages of the RoCE protocol are mainly based on the characteristics of Converged Ethernet, the RoCE protocol can also be applied to traditional Ethernet networks or non-converged Ethernet networks.
[0004] When the traffic in the network is too large and congestion occurs (the message traffic sent from a certain source port may be relatively large in a certain period of time), DCQCN stipulates that the congestion point (CP) device performs random early detection (RED) and explicit congestion notification (ECN) marking on the message. For the receiving end that supports the RoCE protocol, when the receiving end receives a message carrying the ECN mark, it sends an independent congestion notification (CNP) message to the source end to notify the network congestion. Based on the CNP message, the source end reduces the sending rate of subsequent messages to a certain value to eliminate congestion.
[0005] The RoCE protocol uses a separate CNP message notification method to notify network congestion, so when network congestion occurs, the network card at the receiving end needs to continuously send ACK confirmation messages and CNP messages. However, since the message sending rate of the network card at the receiving end has an upper limit, it may cause a delay in sending congestion notifications, which will cause a slow response to congestion control at the source end.
[0006] In addition, CNP messages can only inform the source end that network congestion has occurred, but cannot inform the source end that the network congestion has been eliminated. The source end can only periodically detect whether the network congestion has been eliminated. This also causes the source end to be unable to restore the message sending rate in time, affecting the effective utilization of network bandwidth. Summary of the invention
[0007] The embodiments of the present application provide a RoCE network congestion control method and related devices, which can promptly notify the occurrence and elimination of network congestion, improve the congestion control response speed of the source end, and improve the network bandwidth utilization.
[0008] In a first aspect, the present application provides a method for RoCE network congestion control, the method comprising: a first network device sends a RoCE protocol message to a second network device; the first network device receives a confirmation message from the second network device, the confirmation message comprising confirmation information and indication information for the RoCE protocol message, the indication information being used to indicate whether congestion occurs in a network path between the first network device and the second network device; the first network device performs congestion control based on the confirmation message.
[0009] Wherein, the first network device and the second network device are both hardware designed to allow computing devices to communicate on a network, and both support network communication of the RoCE protocol, RoCE (RDMA over Converged Ethernet) is a network protocol that allows the use of remote direct memory access (RDMA) via Ethernet. The first network device and the second network device can be, for example, a network interface controller (RDMA network interface controller, RNIC), a network interface controller, a network adapter (network adapter), a network interface card (network interface card), or a local area network receiver (LAN adapter). In a possible implementation, at least one of the first network device and the second network device can also be a switch device.
[0010] The first network device can be set at the source device, and the second network device can be set at the destination device, so the source device and the destination device can realize remote data reading, writing and transmission based on the communication interaction between the first network device and the second network device.
[0011] It can be seen that, in the implementation of the present application, the source device can send a message through the first network device, and the destination device can reply to the ACK confirmation information aggregated with the indication information through the second network device, and the indication information is used to inform the first network device whether there is network congestion on the current network path. In this way, the first network device can obtain the state of the current network, that is, whether the current network is congested or not, and the first network device can perform corresponding congestion control operations based on the state of the current network. For example, when the network is not congested, the first network device can maintain or restore the high-speed transmission rate in time. Therefore, on the one hand, through the aggregation bearing of the indication information and ACK, the disadvantage of sending independent CNPs in the traditional solution is avoided, the overhead of the notification is reduced, and the delay of the congestion notification in the large traffic scenario is reduced, which is beneficial to reduce the reaction speed of the destination device. On the other hand, through the indication information, the source device can obtain the network congestion situation at the first time, trigger the congestion control earlier to realize the regulation of the transmission rate, and improve the response speed of the source device. When the network congestion is relieved, the source device can also be informed through the indication information, and the transmission rate can be restored in time, which improves the network bandwidth utilization.
[0012] Based on the first aspect, in a specific embodiment, when the indication information indicates that congestion occurs in the network path, the confirmation message also includes congestion information, and the congestion information specifically includes at least one of the congestion degree, congestion location, message queue length, and network delay of the network path; the first network device performs congestion control based on the confirmation message, specifically including: the first network device performs congestion control according to the congestion information.
[0013] It can be seen that, by implementing the embodiment of the present application, when there is network congestion on the current network path, the confirmation message can also carry congestion information, and the congestion information represents the detailed network status content. In this way, the first network device of the source device can extract the congestion information to perform quantitative and diversified congestion control actions. Therefore, on the one hand, by the aggregation of indication information, congestion information and ACK, the sending of independent CNPs is avoided, the overhead of the notification is reduced, and the delay of congestion notification in large traffic scenarios is reduced, which is beneficial to reduce the delay of congestion notification in large traffic scenarios and improve the response speed of the destination device. On the other hand, the existing RDMA network congestion control has little notification information and slow network congestion convergence speed, while the present application uses aggregated messages to carry detailed congestion information, such as information of different dimensions such as congestion degree, congestion point, queue depth, network delay, etc., which is beneficial to the first network device to perform diversified, differentiated and specific congestion control according to the detailed congestion information, such as adjusting the sending rate of different levels, and realizing diversified adjustments such as the number of messages and sending time, thereby greatly improving the effect of congestion control.
[0014] Based on the first aspect, in a possible embodiment, the first network device performs congestion control according to the congestion information including at least one of the following methods:
[0015] (1) The first network device can quantitatively adjust the message sending rate within the next time window of the first network device according to the congestion level, wherein in a possible embodiment, the congestion level belongs to one of multiple different levels of congestion levels, that is, different levels of congestion levels correspond to different sending rates. There is a corresponding relationship between the congestion level and the sending rate. For example, for multiple levels such as "no congestion, light congestion, moderate congestion, and heavy congestion", the first network device can decide how to implement the speed reduction processing according to the difference in congestion levels. Different levels can correspond to different message sending rates, thereby realizing the adjustment of message sending rates of different grades, thereby achieving faster rate convergence.
[0016] (2) The first network device may determine the number of packets to be sent in the next time window according to at least one of the congestion position and the packet queue depth. The RNIC of the source device may determine how many data packets can be carried on the network path without causing packet loss and the like according to the congestion position and / or the packet queue depth, thereby determining the number of packets that can continue to be sent, which is more friendly to network applications with high bandwidth requirements.
[0017] (3) The first network device may adjust the sending rate of the first network device or the number of messages to be sent in the next time window according to the network delay.
[0018] It can be seen that by implementing the embodiments of the present application, the present application uses aggregated messages to carry detailed congestion information, such as congestion degree, congestion point, queue depth, network delay and other information of different dimensions, which is beneficial for the first network device to perform diversified, differentiated and specific congestion control according to the detailed congestion information, thereby greatly improving the effect of congestion control.
[0019] Based on the first aspect, in a possible embodiment, the confirmation message specifically includes a basic transport header (BTH) field and an extension field, the confirmation information and the indication information are carried in the BTH field, and the congestion information is carried in the extension field.
[0020] The extended field is, for example, the congestion control extended transport header (CETH) described in this document, that is, the congestion information can be carried by the extended CETH header.
[0021] For example, the CETH includes two parts: standard definition and manufacturer-defined information. The standard definition part can be used for compatibility docking in hybrid networking scenarios. The standard definition part may include the following fields: version number (Ver), CETH header length (Length). Among them:
[0022] The manufacturer-defined information field is used to support each manufacturer's customized congestion control notification information, and the total length is, for example, (Length*4-1) bytes. For example, the manufacturer can design at least one of the following information: the congestion level of the bearer network path, the congestion location, the message queue length, and the network delay. For example, the congestion level of the network path can be represented by a 2-bit ratio field, which is used to identify the congestion level. In the application scenario, the ratio field can indicate the congestion level by weight, such as: no congestion, light congestion, moderate congestion, heavy congestion, and so on. In addition, more other content can be designed for the manufacturer-defined information, such as a designed 1-bit field to indicate whether the current congestion notification is a normal CNP type or an enhanced CNP type. For another example, a 4-bit field can be designed to identify a business scenario, such as an RC / XRC write / send scenario, an RC / XRC read response scenario, or an UD send scenario.
[0023] The Ver field is used to indicate the CETH version number, for example, it may occupy 4 bits, and is used to support the upgrade of the congestion control algorithm and compatibility docking.
[0024] The Length field is used to indicate the length of the CETH header, for example, occupies 4 bits and supports variable length of the CETH header to reduce fixed overhead.
[0025] It can be seen that by designing the extended field CETH, it is possible to ensure the carrying of congestion information without occupying the space of existing fields, and it can also be used for compatibility docking in hybrid networking scenarios, and support each manufacturer to customize congestion control notification information, which is conducive to meeting the needs of different manufacturers.
[0026] Based on the first aspect, in a possible embodiment, the confirmation message specifically includes a BTH field, and the confirmation information, the indication information and the congestion information are all carried in the BTH field.
[0027] For example, in one implementation, the reserved field "reserved 6" in the BTH of the traditional ACK can be used to carry the congestion information of the embodiment of the present application, that is, the "reserved 6" is used as the first CETH to carry related data, thereby achieving the purpose of congestion control and transmission confirmation aggregation.
[0028] For another example, in one implementation, the reserved field "reserved 7" in the BTH of the traditional ACK can be used to carry the congestion information of the embodiment of the present application, that is, the "reserved 7" is used as the second CETH to carry related data, thereby achieving the purpose of congestion control and transmission confirmation aggregation.
[0029] This embodiment can integrate the indication information and congestion information into the field of the confirmation information. In this case, the confirmation message can be regarded as an improvement of the traditional ACK message, which makes full use of the existing field space to carry the congestion information, thereby achieving full use of the field space in the confirmation information and avoiding changes to the existing message format.
[0030] Based on the first aspect, in a possible embodiment, when the indication information indicates that the network path is not congested, the first network device performs congestion control based on the confirmation message, including: the first network device maintains the sending rate of the first network device unchanged, for example, maintains a higher sending rate, thereby improving message transmission efficiency.
[0031] Based on the first aspect, in a possible embodiment, when the indication information indicates that the network path is not congested, the first network device performs congestion control based on the confirmation message, including: the first network device sets the sending rate of the first network device to a preset rate. For example, adjusting from a low sending rate (the low sending rate is designed when the network is congested, for example) to a higher sending rate (the high sending rate is designed when the network is not congested, for example), thereby improving the recovery efficiency of the sending rate and reducing the delay of message sending.
[0032] Based on the first aspect, in a possible embodiment, the indication information may be an indication bit, an indication field, an indication flag, etc.
[0033] For example, when the indication information is an indication bit, when the value of the indication bit is 0, it means that the RNIC of the source device is informed that there is no network congestion in the current network path, and the confirmation message does not carry congestion information; when the value of the indication bit is 1, it means that the destination device indicates to the RNIC of the source device that there is network congestion in the current network path, and the confirmation message carries congestion information.
[0034] For another example, the indication information may use existing fields to redefine functions. For example, the indication information may be the BTH.BECN field of a confirmation message. When the BTH.BECN field is 0, it means that the RNIC of the source device is informed that there is no network congestion in the current network path, and the confirmation message does not carry congestion information. When the BTH.BECN field is 1, it means that the RNIC of the source device is informed that there is network congestion in the current network path, and the confirmation message carries congestion information.
[0035] In a second aspect, the present application provides a method for RoCE network congestion control, the method comprising:
[0036] The second network device receives the RoCE protocol message from the first network device; the second network device checks whether the RoCE protocol message carries an explicit congestion notification; the second network device generates a confirmation message according to the check result, the confirmation message including confirmation information and indication information for the RoCE protocol message, the indication information being used to indicate whether congestion occurs in the network path between the first network device and the second network device; the second network device sends the confirmation message to the first network device, and the confirmation message is used for the first network device to perform congestion control.
[0037] It can be seen that, by implementing the embodiment of the present application, the destination device can analyze whether the RoCE protocol message carries an explicit congestion notification through the second network device, and then reply with an ACK confirmation message aggregated with indication information, and the indication information is used to inform the first network device whether there is network congestion on the current network path. In this way, the first network device can obtain the state of the current network, that is, whether the current network is congested or not, and the first network device can perform corresponding congestion control operations based on the state of the current network. For example, when the network is not congested, the first network device can maintain or restore the high-speed transmission rate in time. Therefore, on the one hand, through the aggregation of indication information and ACK, the disadvantage of sending independent CNPs in traditional solutions is avoided, the overhead of notification is reduced, and the delay of congestion notification in large traffic scenarios is reduced, which is beneficial to reduce the reaction speed of the destination device. On the other hand, through the indication information, the source device can obtain the network congestion situation at the first time, trigger congestion control earlier to achieve the regulation of the transmission rate, and improve the response speed of the source device. When the network congestion is relieved, the source device can also be informed through the indication information to restore the transmission rate in time, thereby improving the utilization rate of the network bandwidth.
[0038] Based on the second aspect, in a possible embodiment, when the indication information indicates that congestion occurs in the network path, the confirmation message also includes congestion information, and the congestion information specifically includes at least one of the congestion level, congestion location, message queue length, and network delay of the network path; the congestion information is used by the first network device to perform congestion control, which is beneficial for the first network device to perform diversified, differentiated, and specific congestion control according to the detailed congestion information, such as adjusting the sending rate at different levels, and realizing diversified adjustments such as the number of messages and sending time, thereby greatly improving the effect of congestion control.
[0039] Based on the second aspect, in a possible embodiment, the congestion degree belongs to one of a plurality of different levels of congestion degrees, and different levels of congestion degrees correspond to different sending rates of the first network device.
[0040] Based on the second aspect, in a possible embodiment, before the second network device generates a confirmation message according to the inspection result, the method further includes: the second network device generates the congestion information.
[0041] For example, when the current network of the second network device is congested, network status information can be obtained through message detection or hardware detection, such as congestion degree, congestion point, queue depth, network delay and other information of different dimensions.
[0042] Based on the second aspect, in a possible embodiment, when the congestion information includes the congestion degree, the second network device may obtain the congestion degree by:
[0043] (1) The second network device determines the congestion level of the network path according to the proportion of RoCE protocol messages carrying explicit congestion notifications in the historical message reception records. For example, the second network device performs periodic sliding window statistics on the proportion of received messages carrying ECN marks, thereby calculating the specific congestion level of the current network path.
[0044] (2) Obtaining the congestion level by using in-band network telemetry (INT) or in-situ operation administration and maintenance (IOAM). Taking INT as an example, the scope of INT support can be extended to server network cards, and the network cards can receive the measurement information inserted into the data packets by the switch. The current network status can be calculated through this information, for example, the network delay can be calculated through the timestamp, the congestion level can be calculated through the queue length and queue occupancy, and so on.
[0045] Based on the second aspect, in a possible embodiment, the confirmation message specifically includes a basic transport header (BTH) field and an extension field, the confirmation information and the indication information are carried in the BTH field, and the congestion information is carried in the extension field. The extension field is, for example, a congestion control extended transport header (CETH) described herein, that is, the congestion information can be carried by the extended CETH header.
[0046] By designing the extended field CETH, it is possible to ensure the carrying of congestion information without occupying the space of existing fields. It can also be used for compatibility docking in hybrid networking scenarios and support each manufacturer to customize congestion control notification information, which is conducive to meeting the needs of different manufacturers.
[0047] Based on the second aspect, in a possible embodiment, the confirmation message specifically includes a BTH field, and the confirmation information, the indication information and the congestion information are all carried in the BTH field.
[0048] By integrating the indication information and the congestion information into the field of the confirmation information, the existing field space is fully utilized to carry the congestion information, thereby achieving full utilization of the field space in the confirmation information and avoiding changes to the existing message format.
[0049] In a third aspect, an embodiment of the present application provides an apparatus, which is applied to a first network device, and includes: a message sending module, used to send a RoCE protocol message to a second network device; a message receiving module, used to receive a confirmation message from the second network device, the confirmation message including confirmation information and indication information for the RoCE protocol message, the indication information being used to indicate whether congestion occurs in the network path between the first network device and the second network device; and a congestion control module, used to perform congestion control based on the confirmation message.
[0050] Among them, each functional module of the device is specifically used to implement the method steps described in the first aspect, which will not be repeated here.
[0051] Based on the third aspect, in a possible embodiment, when the indication information indicates that congestion occurs in the network path, the confirmation message also includes congestion information, and the congestion information specifically includes at least one of the congestion degree, congestion location, message queue length, and network delay of the network path; the congestion control module is specifically used to perform congestion control according to the congestion information.
[0052] Based on the third aspect, in a possible embodiment, the congestion control module is specifically used to: adjust the sending rate of the first network device according to the congestion level, wherein there is a corresponding relationship between the congestion level and the sending rate; or, determine the number of messages to be sent in the next time window according to at least one of the congestion position and the message queue depth; or, adjust the sending rate of the first network device or determine the number of messages to be sent in the next time window according to the network delay.
[0053] Based on the third aspect, in a possible embodiment, the congestion degree belongs to one of a plurality of different levels of congestion degrees, and different levels of congestion degrees correspond to different sending rates.
[0054] Based on the third aspect, in a possible embodiment, the confirmation message specifically includes a basic transport header (BTH) field and an extension field, the confirmation information and the indication information are carried in the BTH field, and the congestion information is carried in the extension field.
[0055] Based on the third aspect, in a possible embodiment, the confirmation message specifically includes a BTH field, and the confirmation information, the indication information and the congestion information are all carried in the BTH field.
[0056] Based on the third aspect, in a possible embodiment, the congestion control module is specifically configured to maintain a sending rate of the first network device unchanged when the indication information indicates that the network path is not congested.
[0057] Based on the third aspect, in a possible embodiment, the congestion control module is specifically configured to set the sending rate of the first network device to a preset rate when the indication information indicates that the network path is not congested.
[0058] In a fourth aspect, the present application provides an apparatus, which is applied to a second network device, including: a message receiving module, used to receive a RoCE protocol message from a first network device; a congestion information determination module, used to check whether the RoCE protocol message carries an explicit congestion notification; a notification aggregation sending module, used to generate a confirmation message according to the inspection result, the confirmation message including confirmation information and indication information for the RoCE protocol message, the indication information being used to indicate whether congestion occurs in the network path between the first network device and the second network device; and also used to send the confirmation message to the first network device, the confirmation message being used for the first network device to perform congestion control.
[0059] Among them, each functional module of the device is specifically used to implement the method steps described in the second aspect, which will not be repeated here.
[0060] Based on the fourth aspect, in a possible embodiment, when the indication information indicates that congestion occurs in the network path, the confirmation message also includes congestion information, and the congestion information specifically includes at least one of the congestion level, congestion location, message queue length, and network delay of the network path; the congestion information is used for congestion control by the first network device.
[0061] Based on the fourth aspect, in a possible embodiment, the congestion degree belongs to one of a plurality of different levels of congestion degrees, and different levels of congestion degrees correspond to different sending rates of the first network device.
[0062] Based on the fourth aspect, in a possible embodiment, the congestion information determination module is further used to generate the congestion information.
[0063] Based on the fourth aspect, in a possible embodiment, when the congestion information includes the congestion level, the congestion information determination module is specifically used to: determine the congestion level according to the proportion of RoCE protocol messages carrying explicit congestion notifications in historical message reception records; or, obtain the congestion level through in-band network telemetry (INT); or, obtain the congestion level through on-site operation management and maintenance (IOAM).
[0064] Based on the fourth aspect, in a possible embodiment, the confirmation message specifically includes a basic transport header (BTH) field and an extension field, the confirmation information and the indication information are carried in the BTH field, and the congestion information is carried in the extension field.
[0065] Based on the fourth aspect, in a possible embodiment, the confirmation message specifically includes a BTH field, and the confirmation information, the indication information and the congestion information are all carried in the BTH field.
[0066] In a fifth aspect, the present application provides a device, comprising a host system and a first network device; the host system is used to interact with the first network device to realize data transmission, and the first network device is used to execute the method described in any embodiment of the first aspect.
[0067] In a sixth aspect, the present application provides a device, comprising a host system and a second network device; the host system is used to interact with the second network device to realize data transmission, and the second network device is used to execute the method described in any embodiment of the second aspect.
[0068] In a seventh aspect, the present application provides a first network device, which may include a controller, a register, a communication interface, and a logic operation component, and these components may be electrically connected through one or more internal buses. The first network device implements the method described in any embodiment of the first aspect through the cooperation of each component.
[0069] In an eighth aspect, the present application provides a second network device, which may include a controller, a register, a communication interface, and a logic operation component, and these components may be electrically connected through one or more internal buses. The first network device implements the method described in any embodiment of the second aspect through the cooperation of each component.
[0070] In the ninth aspect, an embodiment of the present application provides a chip, comprising a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to execute a method as described in any embodiment of the first aspect or the second aspect.
[0071] In a tenth aspect, an embodiment of the present invention provides a non-volatile computer-readable storage medium; the computer-readable storage medium is used to store an implementation code of the method described in any embodiment of the first aspect or the second aspect. When the program code is executed by a device, the method described in any embodiment of the first aspect or the second aspect can be implemented.
[0072] In a tenth aspect, an embodiment of the present invention provides a computer program product; the computer program product includes program instructions, and when the computer program product is executed by a device, the method described in any embodiment of the first aspect or the second aspect is executed. The computer program product can be a software installation package, and the computer program product can be downloaded and executed on a controller to implement the method described in any embodiment of the first aspect or the second aspect.
[0073] It can be seen that when implementing the embodiment of the present application, the second network device of the destination device can reply with ACK confirmation information that aggregates CETH and indication information when the RoCE protocol message carries the ECN mark, and the indication information is used to notify the source device that there is network congestion on the current network path, and CETH is used to provide detailed congestion information to the source device. The first network device of the source device extracts congestion information from CETH to perform quantitative and diversified congestion control actions. When the RoCE protocol message does not carry the ECN mark, the destination device can reply with ACK confirmation information carrying indication information to notify the source device that there is no network congestion on the current network path, so that the source device can maintain or restore the high-speed transmission rate in time.
[0074] In this way, on the one hand, by aggregated carrying of indication information, congestion information and ACK, the sending of independent CNPs is avoided, the notification overhead is reduced, which is beneficial to reducing the delay of congestion notification in high-traffic scenarios and improving the response speed of the destination device.
[0075] On the other hand, through the indication information, the source device can obtain the network congestion situation in the first place, trigger the congestion control earlier to adjust the sending rate, and improve the response speed of the source device. When the network congestion is relieved, the source device can also be informed through the indication information to restore the sending rate in time, thus improving the network bandwidth utilization.
[0076] Finally, the existing RDMA network congestion control has little notification information and slow network congestion convergence speed. The present application uses CETH to carry detailed congestion information of the network, such as congestion degree, congestion point, queue depth, network delay and other information of different dimensions, which is conducive to the source device to adjust the sending rate to the target rate in one step according to the detailed congestion information, achieve rapid convergence, and realize diversified adjustments such as the number of messages and sending time, which greatly improves the effect of congestion control. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] Figure 1 is a schematic diagram of the system architecture involved in the embodiments of the present application;
[0078] Figure 2 It is a device communication process scenario diagram of an existing RoCE protocol;
[0079] Figure 3 It is a device communication process scenario diagram under a high-traffic network scenario;
[0080] Figure 4 is a schematic diagram of a system architecture including functional modules provided in an embodiment of the present application;
[0081] Figure 5 It is a schematic diagram of the hardware structure of a network device provided in an embodiment of the present application;
[0082] Figure 6 It is a schematic diagram of the contents of some possible confirmation messages provided in the embodiments of the present application;
[0083] Figure 7 is a schematic diagram of the contents of some possible confirmation messages provided in the embodiments of the present application;
[0084] Figure 8 This is an example diagram of a data structure of congestion information provided by an embodiment of the present application;
[0085] Fig. 9 It is a data structure diagram of a confirmation message provided in an embodiment of the present application;
[0086] Fig.10 It is a schematic diagram of the data structure of another confirmation message provided in an embodiment of the present application;
[0087] Fig.11 It is a schematic diagram of the complete form of some RoCE protocol messages provided in the embodiments of the present application;
[0088] Fig.12 It is a schematic diagram of the complete form of some confirmation messages provided in the embodiments of the present application;
[0089] Fig.13 It is a flow chart of a method for RoCE network congestion control provided by an embodiment of the present application;
[0090] Fig.14 It is a flowchart of another RoCE network congestion control method provided by an embodiment of the present application;
[0091] Fig.15 It is a device communication process scenario diagram provided by an embodiment of the present application;
[0092] Fig.16 It is a device communication process scenario diagram in a network high-traffic scenario provided by an embodiment of the present application. DETAILED DESCRIPTION
[0093] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items. It should be noted that when used in this specification and the appended claims, the term "including" and any of their variations are intended to cover non-exclusive inclusions. For example, a system, product or device comprising a series of units / devices is not limited to the listed units / devices, but optionally also includes units / devices that are not listed, or optionally also includes other units / devices inherent to these products or devices.
[0094] It should also be noted that the terms "first", "second", "third", etc. in this specification and claims are used to distinguish different objects, and are not used to describe a specific order or a specific meaning.
[0095] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.
[0096] First, the system architecture used in the embodiments of the present application is introduced.
[0097] See also Figure 1 , Figure 1 Schematic diagram of the system architecture involved in the embodiment of the present application. Figure 1 As shown, the system architecture includes a source device 10 and a destination device 20, the source device 10 and the destination device 20 are connected to each other through a network 30, and both the source device 10 and the destination device 20 support network communication of the RoCE protocol. The source device 10 and the destination device 20 can be computing devices such as computers, desktop computers, laptops, servers, terminals, etc.
[0098] The network 30 may include a plurality of switching devices 31, which may be used for message transfer, transmission, network traffic detection, etc. between the source device 10 and the destination device 20 to achieve communication interaction between the source device 10 and the destination device 20. The switching device 31 may be, for example, a switch, a router, a relay device, or a gateway device, etc.
[0099] The source device 10 and the destination device 20 may include a network device and a host system, wherein the host system includes a host CPU (central processing unit) and a memory. Figure 1 The source device 10 includes a CPU 12, a memory 13 and a network device 11, and these components can be connected via a bus. The destination device 20 includes a CPU 22, a memory 23 and a network device 21, and these components can also be connected via a bus.
[0100] In the embodiment of the present application, the network device is a piece of hardware designed to allow a computing device to communicate on a network. The network device may be a network interface controller (NIC) used to implement communication between a device and a network, which may also be called a network interface controller, a network adapter, a network interface card, or a LAN adapter. In the embodiment of the present application, the network device supports the RDMA protocol, so the NIC may also be called an RNIC (RDMA NIC). This article will mainly describe the solution using RNIC as an example.
[0101] like Figure 1As shown, the network device 11 and the network device 21 are connected through the network 30 to realize the communication between the source device 10 and the destination device 20. Both the network device 11 and the network device 21 support the RoCE protocol. When the source device 10 initiates an RDMA read / write request to the destination device 20 through the network, the data to be written is directly written from the memory 13 to the memory 23 through the network device 11 and the network device 21, or the data to be read is directly written from the memory 23 to the memory 13.
[0102] For the host systems of the source device 10 and the destination device 20, the number of CPUs in the host system can be one or more, and the types of each CPU can be different or the same. The CPU may include one or more processor cores, or multiple CPUs may be integrated into a multi-core processor. The host system may run various software components through the CPU, such as an operating system, and applications running on the operating system, etc. The user may initiate business communications through the operating system or application, and then realize communication interaction between the source device 10 and the destination device 20 through the network device.
[0103] The memory in the host system can be used to store computer instructions and data. The memory can also store data read and written through RDMA, messages, etc. The memory can be any one or any combination of the following storage media: Storage Class Memory (SCM), Read-Only Memory (ROM), Random Access Memory (RAM) or Cache.
[0104] In the above system architecture, remote access between two computing devices is implemented by the RNIC of the computing devices. A network consisting of RNICs and switching devices supporting the RoCE protocol together constitutes a RoCE network. The present application scheme is applied to the RoCE network, and the method described in the embodiment of the present application can be deployed on the RNIC network card to implement RoCE network congestion control.
[0105] Although the above two computing devices are divided into source devices and destination devices, it should be understood that "source devices" and "destination devices" are two relative concepts:
[0106] The source device refers to the computing device that initiates the RDMA request, that is, the computing device that requests access to another computing device.
[0107] The destination device refers to the computing device that receives the RDMA request, that is, the computing device accessed by another computing device.
[0108] For example, the source device's access to the destination device may be that the source device writes data to the destination device. Specifically, the source device transmits the data in the source device to the RNIC of the destination device through the RNIC of the source device, and the destination device receives the data through the RNIC of the destination device, thereby transmitting the data in the source device to the destination device. The source device's access to the destination device may also be that the source device reads data from the destination device. Specifically, the source device may read the data in the memory of the destination device through the RNIC of the source device, and the destination device sends the data to be read by the source device to the RNIC of the source device through the RNIC of the destination device, and the RNIC of the source device receives the data, thereby completing the reading of the data in the destination device.
[0109] In the embodiment of the present application, the data communicated between them is mainly carried in the form of messages. In this article, the messages supporting the RoCE protocol can be referred to as RoCE protocol messages or RoCE data messages. Generally speaking, when the message sent by the source device 10 to the destination device 20 is successfully received by the destination device, the destination device needs to reply with an ACK confirmation message to inform the source device that the message is successfully received. The technical solution of the embodiment of the present application is mainly to optimize this process.
[0110] It should be noted that Figure 1 The above components and their connection relationships are only shown for describing the technical solution provided by the example of this application. Figure 1 The source device 10 and the destination device 20 shown may also include other components besides the above components, for example, they may also include hardware resources such as a hard disk, which will not be described in detail here.
[0111] Figure 2The figure shows a device communication process of an existing RoCE protocol. The RoCE protocol is a type of RDMA protocol. The RDMA protocol is a transport layer protocol, and the RoCE protocol is a protocol that additionally includes the network layer and the link layer. The RoCE protocol also supports reliable connection services. The source device carries the packet sequence number (PSN) when sending the protocol message. After receiving the protocol message, the destination device replies with an ACK confirmation message to the source device, notifying the RNIC of the source device that the message it sent has been successfully transmitted. When the traffic in the network is congested due to excessive traffic, the CP device in the network marks the message with RED ECN. When the destination device receives the message carrying the ECN mark, in addition to replying the ACK confirmation message to the source device, it also sends an independent CNP message to the source device to notify the network congestion according to the protocol. The CNP message is only defined as a signal and does not carry any status information. In addition, after the destination device replies with the ACK confirmation message and the CNP message respectively, the RNIC of the source device also needs to process these two messages separately.
[0112] Figure 3 The following is an example of a device communication process in a network with high traffic volume. Figure 3 As shown, network congestion occurred in data packet 1, but due to the limited sending capacity of the destination device, the sending of its first CNP message A was delayed, which caused the source device to start slowing down until protocol message 5. Before that, the network congestion level continued to increase due to the failure to reduce the sending rate in time. Network congestion began to be eliminated from protocol message 6, but due to the notification delay of CNP messages, the source device continued to slow down until protocol message 7. Excessive speed reduction will affect network bandwidth utilization. From CNP message B, the destination device did not notify the network congestion again, but the source device could not obtain the information of congestion elimination in time, and could only increase the speed slowly by timeout until protocol message 10 to restore to the target sending rate. During this period, the network bandwidth utilization was low.
[0113] From the above, we can see that in the existing scheme, since the message sending rate of the network card has an upper limit, after network congestion occurs, the network card of the destination device needs to continuously send ACK confirmation messages and CNP messages. When the network traffic pressure increases, it will cause the reply delay of the confirmation message and CNP message, thereby causing the response delay of the speed control. In addition, CNP can only notify the occurrence of congestion, but cannot notify that the congestion has been eliminated. It can only be detected by the sending end at regular intervals. The increase in ACK delay will lead to an excessive number of flying packet statistics, and the delay of CNP messages will cause the source device to react slowly to congestion control and the rate control to be untimely.
[0114] In addition, the RoCE protocol uses CNP messages to notify network congestion, which can only notify that congestion has occurred on the network, but not the specific status of the network congestion. This results in the sender being unable to achieve efficient congestion control, but can only slowly approach the target rate step by step, resulting in slow network convergence and low bandwidth utilization.
[0115] The embodiments of the present application improve the RNIC of the source device and the destination device, thereby solving some or all of the defects mentioned in the above existing solutions. Figure 4 , Figure 4 A specific system architecture of an embodiment of the present application is shown. In the system architecture, the RNICs of the source device 10 and the destination device 20 are respectively configured with relevant functional modules to support the implementation of the solution of the present application. Figure 4 As shown, the network device of the source device 10 is RNIC11, and the network device of the destination device 20 is RNIC21. RNIC11 is configured with a congestion control module 111, a message sending module 112, and a message receiving module 113. RNIC21 is configured with a congestion information determination module 211, a notification aggregation sending module 212, and a message receiving module 213, which are specifically described as follows:
[0116] The message sending module 112 is used to send the RoCE protocol message to the destination device 20.
[0117] The message receiving module 113 is used to receive a confirmation message from the destination device 20, and the confirmation message is an aggregate message designed in the embodiment of the present application. The confirmation message can aggregate the confirmation information (ACK) and indication information of the RoCE protocol message, and the indication information is used to indicate whether the network path between the source device 10 and the destination device 20 is congested. In the case of congestion in the network path, the confirmation message also aggregates the congestion information, and the congestion information specifically includes at least one of the congestion degree, congestion location, message queue length, and network delay of the network path. In other words, the confirmation message carries both the indication information of the network congestion and the specific status information of the network congestion. The specific implementation of the confirmation message will be described in detail later.
[0118] The congestion control module 111 is used to perform quantifiable congestion control based on the confirmation message. In the embodiment of the present application, congestion control is a function used to adjust the number of packets sent in a single transmission control protocol (RoCE protocol) connection (single sending amount). It can quantitatively increase or decrease the single message sending amount and sending frequency to approach the most appropriate carrying capacity of the current network.
[0119] The message receiving module 213 is used to receive the RoCE protocol message from the source device 10.
[0120] The congestion information determination module 211 is used to check whether the RoCE protocol message carries an explicit congestion notification. If the RoCE protocol message carries an explicit congestion notification, the congestion information determination module 211 can be used to generate the congestion information, and the congestion information specifically includes at least one of the congestion degree, congestion location, message queue length, and network delay of the network path. The congestion information is used to support the source device to perform quantitative congestion control.
[0121] The notification aggregation sending module 212 can be used to generate a confirmation message according to the inspection result of the message, which is the aggregation message designed in the embodiment of the present application, and is also used to send the confirmation message to the source device 10 so that the source device 10 can achieve quantitative congestion control.
[0122] The above functional modules of the RNIC in the source device and the destination device can be implemented through the cooperation of the software and hardware structures of their respective RNICs. Figure 5 , Figure 5 An exemplary RNIC hardware structure 30 is shown, which can be the structure of the RNIC in the source device or the structure of the RNIC in the destination device. In a specific implementation, the RNIC hardware structure 30 can be an independent standard network card (such as a network card with a PCIe interface) or an integrated network card integrated into a SoC chip, which is obtained by upgrading the existing RNIC network card (such as an ASIC chip, FW firmware, etc.) hardware to support the solution mentioned in the embodiment of the present application.
[0123] like Figure 5 As shown, the RNIC hardware structure 30 may include a controller 31, a register 32, a communication interface 33 and a logic operation unit 34, and these components may be electrically connected via one or more internal buses 35. Among them:
[0124] Register 32 is a memory with a small storage space. Register 32 can be used to save various instructions. Register 32 can also be used to store register operands and intermediate or final operation results temporarily stored during instruction execution. Registers can also be used to store data used by the logic operation unit 34 to complete the tasks requested by the controller 31.
[0125] The controller 31 is used to decode the instructions stored in the register and send control signals for each operation to be performed to complete each instruction. The controller 31 is a processor core that can run programs, and can be implemented by circuit devices such as system-on-a-Chip (SOC), field programmable gate array (FPGA), application-specific integrated circuit (ASIC) or FPGA+ASIC. The controller 31 can also be composed of various AND or OR gate arrays. The control method of the controller 31 can be, for example, a microprogram control method with microstorage as the core, and the microprogram can be stored in the register 32, or it can be a hardware control method based on a logical hard wiring structure, which is not limited in this application.
[0126] The logic operation unit 34 can be used to execute operation commands, such as addition commands, subtraction commands, multiplication commands, division commands, etc. The logic operation unit 34 can also be used to obtain logic commands, such as OR logic commands, AND logic commands, NOT logic commands, etc. The logic operation unit 34 can also be used to obtain a control signal from the controller 31, obtain data corresponding to the control signal from the register 32 according to the obtained control signal, and perform corresponding operations.
[0127] The communication interface 33 is used to send or receive data. There may be multiple communication interfaces 33, which may be used to receive data sent by the processor or send data to the CPU of the host system, or to receive data sent by an external computing device or send data to an external computing device (for example, sending or receiving RoCE protocol messages or aggregated confirmation messages).
[0128] Optionally, the RNIC may also include a crystal oscillator, a media access controller, a physical interface transceiver, etc., which is not limited in the embodiment of the present application.
[0129] In a specific embodiment, the controller 31 reads the instructions stored in the register and issues control signals for completing various operations to be performed for each instruction, so as to implement the RoCE network congestion control method described in any embodiment of this document.
[0130] In order to better understand the implementation scheme of the present application, the confirmation message capable of implementing congestion notification aggregation provided by the embodiment of the present application is described in detail below.
[0131] The embodiment of the present application improves the congestion notification mechanism of the RoCE protocol by extending the existing RoCE ACK confirmation message, and obtains the confirmation message of the present application, so that it can carry indication information and congestion information of network congestion, so as to achieve congestion notification aggregation and accurate network congestion notification.
[0132] See also Figure 6 and Figure 7 , Figure 6 and Figure 7 The contents of some possible confirmation messages in the embodiments of the present application are shown. The confirmation message can be generated by the destination device and used to reply to the source device.
[0133] Figure 6 A schematic diagram of two types of confirmation messages in a scenario without network congestion (for example, the RoCE protocol message does not carry an explicit congestion notification ECN) is shown. The confirmation message includes confirmation information and indication information for the RoCE protocol message. The confirmation information can realize the function of the existing ACK confirmation, that is, notifying the RNIC of the source device whether the message it sent has been successfully transmitted by the destination device. The indication information is used to indicate that there is no congestion in the network path between the source device and the destination device.
[0134] like Figure 6 As shown, in one embodiment, the confirmation message is implemented as confirmation message A, and the confirmation information and indication information in the confirmation message A can be set at different positions in the message, for example, they can be distributed in different message headers, thereby avoiding modification of the fields in the confirmation information. In another embodiment, the confirmation message is implemented as confirmation message B, and the indication information can be integrated and set into the fields of the confirmation information, thereby achieving full utilization of the field space in the confirmation information.
[0135] Figure 7 A schematic diagram of two types of confirmation messages in a network congestion scenario (for example, a RoCE protocol message carries an explicit congestion notification (ECN)) is shown. The confirmation message includes confirmation information, indication information and congestion information for the RoCE protocol message. The confirmation information can realize the function of the existing ACK confirmation, that is, notifying the RNIC of the source device whether the message it sent has been successfully transmitted by the destination device. The indication information is used to indicate that the network path between the source device and the destination device is congested. The congestion information is used to indicate the specific status of the network, which may specifically include at least one of the congestion degree, congestion location, message queue length, network delay, etc. of the network path.
[0136] like Figure 7 As shown, in one embodiment, the confirmation message is implemented as a confirmation message C, and the confirmation information, indication information and congestion information in the confirmation message C can be set at different positions in the message, for example, they can be distributed in different message headers, thereby avoiding modification of the fields in the confirmation information. In another embodiment, the confirmation message is implemented as a confirmation message D, and the indication information and / or congestion information can be integrated and set into the fields of the confirmation information, thereby achieving full utilization of the field space in the confirmation information.
[0137] In the embodiment of the present application, the indication information may be an indication bit, an indication field, an indication flag, etc.
[0138] For example, when the indication information is an indication bit, when the value of the indication bit is 0, it means that the RNIC of the source device is informed that there is no network congestion in the current network path, and the confirmation message does not carry congestion information; when the value of the indication bit is 1, it means that the destination device indicates to the RNIC of the source device that there is network congestion in the current network path, and the confirmation message carries congestion information.
[0139] For another example, the indication information may use existing fields to redefine functions. For example, the indication information may be the BTH.BECN field of a confirmation message. When the BTH.BECN field is 0, it means that the RNIC of the source device is informed that there is no network congestion in the current network path, and the confirmation message does not carry congestion information. When the BTH.BECN field is 1, it means that the RNIC of the source device is informed that there is network congestion in the current network path, and the confirmation message carries congestion information.
[0140] In an embodiment of the present application, congestion information can be a new message header (such as CETH described below) defined for content carrying, which supports carrying detailed network congestion status content, so that the destination device can accurately notify the source device of network congestion. Congestion information can also be carried by using existing field space, such as using an existing reserve field.
[0141] The following describes a design method for congestion information provided by an embodiment of the present application. Figure 8 , Figure 8 : is an example diagram of a data structure of congestion information provided by an embodiment of the present application. In this article, the data structure of congestion information may be referred to as a congestion control extended transport header (Congestion Extended Transport Header, CETH). Figure 8 As shown, the CETH (also called CETH header) includes two parts: standard definition and vendor defined information. The standard definition part can be used for compatibility docking in hybrid networking scenarios. The standard definition part may include the following fields: version number (Ver), CETH header length (Length). Among them:
[0142] The manufacturer-defined information field is used to support each manufacturer's customized congestion control notification information, and the total length is, for example, (Length*4-1) bytes. For example, the manufacturer can design at least one of the following information: the congestion level of the bearer network path, the congestion location, the message queue length, and the network delay. For example, the congestion level of the network path can be represented by a 2-bit ratio field, which is used to identify the congestion level. In the application scenario, the ratio field can indicate the congestion level by weight, such as: no congestion, light congestion, moderate congestion, heavy congestion, and so on.
[0143] In addition, in a specific implementation, more other contents can be designed for the manufacturer-defined information, such as a 1-bit field designed to indicate whether the current congestion notification is a normal CNP type or an enhanced CNP type. For another example, a 4-bit field can be designed to identify a service scenario, such as an RC / XRC write / send scenario, an RC / XRC read response scenario, or an UD send scenario.
[0144] The Ver field is used to indicate the CETH version number, which can occupy 4 bits, for example, to support the upgrade of the congestion control algorithm and compatibility docking. Among them, version number 0 represents a standard CNP notification without carrying other information, and version numbers 1 to 15 are customized by the manufacturer.
[0145] The Length field is used to indicate the length of the CETH header, for example, it occupies 4 bits and supports variable length of the CETH header to reduce fixed overhead. The Length field can take values of 1 to 4, for example, to indicate how many 4-bytes the CETH header length is.
[0146] Based on the above indication information and congestion information, some possible data structures of confirmation messages are described below.
[0147] See also Fig. 9 , Fig. 9 This is a data structure diagram of a confirmation message provided by this application. Fig. 9 As shown, the confirmation message specifically includes ACK confirmation information and CETH, where:
[0148] The ACK confirmation information specifically includes a base transport header (BaseTransport Header, referred to as BTH, or BTH header, or BTH field) and an acknowledgement extended transport header (Acknowledge Extended Transport Header, AETH), that is, the ACK confirmation information can be carried by BTH and AETH, and the ACK confirmation information is used to implement the function of ACK, that is, to inform the RNIC of the source device whether the message it sent has been successfully transmitted by the destination device. In this embodiment, the indication information used to indicate whether the network path between the source device and the destination device is congested can also be carried in the BTH field.
[0149] For the relevant subfields of BTH and AETH (such as OpCode, SE, target QP, Pad, TVer, etc.), please refer to the relevant description of the prior art solution, which will not be described in detail here.
[0150] CETH is an extended field designed by the present application. In this embodiment, CETH can be aggregated as an option behind AETH of ACK, thereby realizing the aggregation of ACK confirmation information, indication information and CETH. CETH is used to indicate the specific network status of the current network path. When network congestion occurs, ACK carries indication information and aggregates CETH, so that the destination device promptly notifies the source device of the occurrence of network congestion. When the network is not congested, ACK carries indication information but does not carry CETH, so that the destination device promptly notifies the source device of the elimination of congestion.
[0151] Since CETH carries network status information such as the degree of congestion, location of congestion, length of message queue, network delay, etc., it helps to solve the problem that the existing RoCE network is not friendly to efficient congestion control due to the small amount of information in congestion control notification. Due to the aggregation of messages and the carrying of indication information, it is also helpful to solve the problem of slow response speed of RoCE network congestion control.
[0152] In a specific application scenario, when the RoCE protocol message received by the RNIC of the destination device does not carry the ECN mark, the RNIC of the destination device replies to the source device with an ACK that does not carry the CETH header. The ACK only includes confirmation information and indication information to inform the RNIC of the source device that its protocol message has been received and there is no congestion on the network.
[0153] When the RoCE protocol message received by the RNIC of the destination device carries the ECN mark, the RNIC of the destination device replies to the source device with an ACK carrying the CETH header. The ACK includes confirmation information, indication information and congestion information to inform the RNIC of the source device that its protocol message has been received and congestion has occurred on the network, and enables the RNIC of the source device to obtain detailed network status information, such as congestion degree, congestion point, queue depth, network delay and other information in different dimensions, so as to achieve quantitative congestion control based on this information.
[0154] For example, the degree of network congestion indicated in the congestion information can be divided into multiple levels such as "no congestion, light congestion, moderate congestion, and heavy congestion". The RNIC of the source device can decide how to implement the speed reduction processing according to the difference in congestion degree, and realize the adjustment of different levels of message sending rates, thereby achieving faster rate convergence.
[0155] For example, for congestion point and queue depth information, the RNIC of the source device can determine how many data packets can be carried on the network path without causing packet loss, etc., and thus determine the number of packets that can continue to be sent. This is more friendly to network applications with high bandwidth requirements.
[0156] See also Fig.10 , Fig.10 FIG. 1 is a schematic diagram of another data structure of a confirmation message provided by the present application. In this embodiment, CETH (congestion information) can be carried using the existing field space. Fig.10 As shown, CETH is integrated into the BTH field of the ACK confirmation information. In this embodiment, the indication information for indicating whether the network path between the source device and the destination device is congested can also be carried in the BTH field. In this case, the confirmation message can be regarded as an improvement of the traditional ACK message, making full use of the existing field space to realize the carrying of congestion information. The confirmation message can not only realize the function of ACK, that is, notifying the RNIC of the source device whether the message it sent has been successfully transmitted by the destination device; but also realize the congestion indication function, that is, indicating whether the network path between the source device and the destination device is congested. It can also enable the RNIC of the source device to obtain detailed network status information, such as information of different dimensions such as congestion degree, congestion point, queue depth, network delay, etc., so that quantitative congestion control can be realized based on this information.
[0157] For example, in one implementation, the reserved field "reserved 6" in the BTH of the traditional ACK can be used to carry the congestion information of the embodiment of the present application, that is, the "reserved 6" is used as the first CETH to carry related data, thereby achieving the purpose of congestion control and transmission confirmation aggregation.
[0158] For another example, in one implementation, the reserved field "reserved 7" in the BTH of the traditional ACK can be used to carry the congestion information of the embodiment of the present application, that is, the "reserved 7" is used as the second CETH to carry related data, thereby achieving the purpose of congestion control and transmission confirmation aggregation.
[0159] The specific implementation form of the first CETH and the second CETH may include only information such as congestion degree, congestion point, queue depth, network delay, etc., and may also include information such as Figure 8 Other information described in the embodiment, such as version number, etc.
[0160] refer to Fig.11 and Fig.12 , Fig.11 and Fig.12 The complete forms of some RoCE protocol messages and the complete forms of corresponding confirmation messages in the embodiments of the present application are described as examples.
[0161] like Fig.11 As shown, a RoCE protocol message from a source device may include a MAC message header, an IP message header, a UDP message header, a BTH header, a data payload, an invariant cyclic redundancy check value (Invariant CyclicRedundancy Check, ICRC) and a variable cyclic redundancy check value (Variant CyclicRedundancy Check, VCRC). The MAC message header, the IP message header, and the UDP message header are the message headers corresponding to the MAC layer, the IP layer, and the UDP layer, respectively. The data payload is the data that needs to be transmitted for device communication interaction. ICRC and VCRC can be used to verify data integrity. Another RoCE protocol message may include a MAC message header, an IP message header, a UDP message header, a BTH header, an ImmDt field, a data payload, an ICRC, and a VCRC.
[0162] It should be noted that, in actual applications, the RoCE protocol message may also include more or less content, and this application does not impose any restrictions on this.
[0163] like Fig.12 As shown, a confirmation message from the destination device may include a MAC message header, an IP message header, a UDP message header, a BTH header, an AETH header, a CETH header, an ICRC and a VCRC. The MAC message header, the IP message header and the UDP message header are the message headers corresponding to the MAC layer, the IP layer and the UDP layer respectively. ICRC and VCRC can be used to verify data integrity. The specific content and implementation form of the BTH header, AETH header and CETH header here can be similarly referred to. Fig. 9The relevant description of the embodiment is not repeated here. Another confirmation message may include a MAC message header, an IP message header, a UDP message header, a BTH header, an AETH header, an ICRC and a VCRC. The specific content and implementation form of the BTH header and the AETH header here can be similarly referred to. Fig.10 The relevant description of the embodiment will not be repeated here.
[0164] It should be noted that, in actual applications, the confirmation message may also include more or less content, and this application does not limit this. For example, in a possible implementation, there may be a SACK (code b010) header after CETH, and so on.
[0165] Based on the system architecture and message data structure described above, the congestion control method provided by the embodiment of the present application is described below. For the various method embodiments described below, for convenience, they are all expressed as a combination of a series of action steps, but technicians in this field should be aware that the specific implementation of the technical solution of the present application is not limited by the order of the described series of action steps.
[0166] See also Fig.13 , Fig.13 This is a flow chart of a method for RoCE network congestion control provided by an embodiment of the present application. The method describes the scheme from the perspective of the interaction between the first network device and the second network device, wherein the first network device and the second network device can be RNIC, network interface controller, network adapter, network card, local area network receiver, etc. For example, the first network device can be the RNIC of the source device, and the second network device can be the RNIC of the destination device. The first network device and the second network device can be connected via a network. The method includes but is not limited to the following steps:
[0167] S301. The first network device sends a RoCE protocol message to the second network device. The RoCE protocol message may be generated based on the service requirements of the user, and the RoCE protocol message may be a periodic message. The specific content of the RoCE protocol message has been described above and will not be repeated here.
[0168] S302. The second network device checks whether the RoCE protocol message carries an explicit congestion notification.
[0169] When the traffic in the network is too large and congestion occurs, the CP device in the network can mark the message with RED ECN. When the second network device receives the message carrying the ECN mark, it confirms that the network is currently congested. Conversely, when the second network device receives the message carrying the ECN mark, it confirms that the network is currently not congested.
[0170] S303. The second network device generates an ACK confirmation message according to the inspection result, wherein the ACK confirmation message includes at least aggregated confirmation information and indication information, wherein the indication information is used to indicate whether the network path between the first network device and the second network device is congested.
[0171] In the embodiment of the present application, the indication information may be an indication bit, an indication field, an indication flag, etc. For example, when the indication information is an indication bit, when the value of the indication bit is 0, it means that the RNIC of the source device is informed that there is no network congestion in the current network path, and the confirmation message does not carry congestion information; when the value of the indication bit is 1, it means that the destination device indicates to the RNIC of the source device that there is network congestion in the current network path, and the confirmation message carries congestion information.
[0172] In one embodiment, the confirmation information and the indication information can be set at different positions in the message, for example, they can be distributed in different message headers, so as to avoid modifying the fields in the confirmation information. In another embodiment, the indication information can be integrated into the fields of the confirmation information, so as to fully utilize the field space in the confirmation information.
[0173] S304. The second network device replies the confirmation message to the first network device, and correspondingly, the first network device receives the confirmation message from the second network device.
[0174] S305. The first network device performs congestion control based on the confirmation message.
[0175] In an embodiment of the present application, congestion control is a function used to adjust the number of packets (single transmission volume) sent in a single transmission control protocol (RoCE protocol) connection. It can quantitatively increase or decrease the single message transmission volume and transmission frequency to approach the most appropriate carrying capacity of the current network. For example, when the indication information indicates that the current network is congested, the first network device can reduce the transmission rate of the RoCE protocol message in the next time window. When the indication information indicates that the current network is not congested, the first network device can maintain the transmission rate of the RoCE protocol message in the next time window unchanged, or set the transmission rate of the RoCE protocol message in the next time window to a preset rate.
[0176] It can be seen that when implementing the embodiment of the present application, the destination device can reply with ACK confirmation information aggregated with indication information through the second network device when the RoCE protocol message carries the ECN mark, and the indication information is used to notify the source device that there is network congestion on the current network path. The first network device of the source device can reduce the sending rate of the RoCE protocol message in the next time window. The destination device can reply with ACK confirmation information carrying indication information when the RoCE protocol message does not carry the ECN mark, and notify the source device that there is no network congestion on the current network path, so that the source device can maintain or restore the high-speed sending rate in time.
[0177] In this way, on the one hand, by aggregated carrying of indication information and ACK, the sending of independent CNPs is avoided, the notification overhead is reduced, which is beneficial to reducing the delay of congestion notification in high-traffic scenarios and improving the response speed of the destination device.
[0178] On the other hand, through the indication information, the source device can obtain the network congestion situation in the first place, trigger the congestion control earlier to adjust the sending rate, and improve the response speed of the source device. When the network congestion is relieved, the source device can also be informed through the indication information to restore the sending rate in time, thus improving the network bandwidth utilization.
[0179] See also Fig.14 , Fig.14 This is a flow chart of another RoCE network congestion control method provided by an embodiment of the present application. The method describes the scheme from the perspective of the interaction between the first network device and the second network device, wherein the first network device and the second network device can be RNIC, network interface controller, network adapter, network card, local area network receiver, etc. For example, the first network device can be the RNIC of the source device, and the second network device can be the RNIC of the destination device, and the first network device and the second network device can be connected via the network. The method includes but is not limited to the following steps:
[0180] S401. The first network device sends a RoCE protocol message to the second network device. The specific content of the RoCE protocol message has been described above and will not be repeated here.
[0181] S402. The second network device checks whether the RoCE protocol message carries an explicit congestion notification (ECN). When it is determined that the RoCE protocol message carries an explicit congestion notification, the second network device subsequently performs steps S403-S405; when it is determined that the RoCE protocol message does not carry an explicit congestion notification, the second network device subsequently performs steps S406-S407.
[0182] S403. The second network device obtains congestion information, where the congestion information is used to indicate a specific state of the network.
[0183] Specifically, when the current network is congested, the second network device can obtain network status information through message detection or hardware detection, such as information of different dimensions such as congestion degree, congestion point, queue depth, network delay, etc.
[0184] For example, when the congestion information includes the congestion level, the second network device may obtain the congestion level by:
[0185] (1) The second network device determines the congestion level of the network path according to the proportion of RoCE protocol messages carrying explicit congestion notifications in the historical message reception records. For example, the second network device performs periodic sliding window statistics on the proportion of received messages carrying ECN marks, thereby calculating the specific congestion level of the current network path.
[0186] (2) Obtaining the congestion level by using in-band network telemetry (INT) or in-situ operation administration and maintenance (IOAM). Taking INT as an example, the scope of INT support can be extended to server network cards, and the network cards can receive the measurement information inserted into the data packets by the switch. The current network status can be calculated through this information, for example, the network delay can be calculated through the timestamp, the congestion level can be calculated through the queue length and queue occupancy, and so on.
[0187] S404. The second network device generates a confirmation message, wherein the confirmation message includes aggregated confirmation information, indication information and congestion information. The congestion information may be implemented, for example, through the CETH described herein, which may be an extended field or may be implemented using an existing reserved field.
[0188] The confirmation information is used to implement the ACK function, and the indication information is used to indicate that the network path between the source device and the destination device is congested.
[0189] In one embodiment, the confirmation information, indication information and congestion information in the confirmation message may be set at different positions in the message, for example, they may be distributed in different message headers, thereby avoiding modification of fields in the confirmation information.
[0190] In yet another embodiment, the indication information and / or congestion information may be integrated into a field of the confirmation information, thereby achieving full utilization of the field space in the confirmation information.
[0191] For the aggregation of confirmation information, indication information and congestion information, please refer to Fig. 9 or Fig.10 The description of the embodiments will not be repeated here.
[0192] S405. The second network device sends the confirmation message of S404 to the first network device.
[0193] S406. The second network device generates a confirmation message, where the confirmation message includes aggregated confirmation information and indication information.
[0194] The confirmation information is used to implement the ACK function, and the indication information is used to indicate that there is no congestion in the network path between the source device and the destination device.
[0195] S407. The second network device sends the confirmation message of S406 to the first network device.
[0196] S408. The first network device performs quantitative congestion control based on the confirmation message.
[0197] Specifically, after the first network device receives the confirmation message of S405, it determines according to the indication information that the current network is congested, and the first network device may perform congestion control according to the congestion information in the confirmation message in at least one of the following ways:
[0198] (1) The first network device can quantitatively adjust the message sending rate of the first network device in the next time window according to the congestion level, wherein there is a corresponding relationship between the congestion level and the sending rate. For example, for multiple levels such as "no congestion, light congestion, moderate congestion, and heavy congestion", the first network device can decide how to implement the speed reduction process according to the difference in congestion levels. Different levels can correspond to different message sending rates, and different grades of message sending rate can be adjusted, thereby achieving faster rate convergence.
[0199] (2) The first network device may determine the number of packets to be sent in the next time window according to at least one of the congestion position and the packet queue depth. The RNIC of the source device may determine how many data packets can be carried on the network path without causing packet loss and the like according to the congestion position and / or the packet queue depth, thereby determining the number of packets that can continue to be sent, which is more friendly to network applications with high bandwidth requirements.
[0200] (3) The first network device may adjust the sending rate of the first network device or the number of messages to be sent in the next time window according to the network delay.
[0201] When the first network device receives the confirmation message of S407, it determines that the current network is not congested according to the indication information, and the first network device can maintain the sending rate of the RoCE protocol message in the next time window unchanged, or restore / set the sending rate of the RoCE protocol message in the next time window to the preset rate.
[0202] Based on the above embodiments, the solution of the present application can realize the aggregation of RoCE congestion control notification and ACK confirmation information, and notify the generation and elimination of network congestion through indication information. When there is network congestion, the defined CETH carries detailed network congestion status information. Specifically, Fig.15 As shown:
[0203] When the network is congested, the message received by the destination device carries an ECN mark (recorded as data w / ECN). The embodiment of the present application redefines the ACK message used for reply as ACK aggregate CETH (recorded as ACK w / CETH), and extends the CETH header after the AETH header to carry quantified congestion information. The CNPs of RDMA write and RDMAsend connected by the original RC, XRC, RD and other application scenarios can be aggregated to ACK, so in the embodiment of the present application, the destination device will no longer need to reply CNP to the source device separately.
[0204] When the network congestion is eliminated, the message received by the destination device does not carry the ECN mark (recorded as data w / oECN). The embodiment of the present application redefines the ACK message used for reply as ACK without aggregated CETH (recorded as ACK w / o CETH) so that the remote device can know the current network status and quickly restore the sending rate.
[0205] Below Fig.16 Take this as an example to further understand the technical effect of the present application solution. Fig.16 The following is an example of a device communication process in a network with high traffic volume. Fig.16 As shown in the figure, network congestion occurs in data packet 1. After the destination device aggregates the congestion notification CETH of RoCE into the ACK confirmation message, the source device obtains the congestion notification faster when congestion occurs, so that congestion control actions can be performed more quickly. For example, Fig.16 The speed is reduced from protocol message 4. Figure 3 For the independent CNP notification method, the message sending rate can be reduced more quickly; the aggregation method can also support the destination device to use the indication information in the ACK to notify the source device of the elimination of the network congestion state. After the source device receives the congestion elimination notification, it can quickly increase the message sending rate, such as Fig.16The sending rate can be restored starting from data message 8. Compared with the existing periodic detection method that relies on the source device, the message sending rate can be increased faster.
[0206] In addition, due to the expansion of accurate and detailed congestion notification information, the sending rate can be accurately controlled to be reduced to the target rate in the first cycle of speed reduction, achieving faster convergence, such as Fig.16 In the example, after receiving the congestion notification information A, the first data packet 4 sent can quickly converge to the target rate.
[0207] It can be seen that when implementing the embodiment of the present application, the second network device of the destination device can reply with ACK confirmation information that aggregates CETH and indication information when the RoCE protocol message carries the ECN mark, and the indication information is used to notify the source device that there is network congestion on the current network path, and CETH is used to provide detailed congestion information to the source device. The first network device of the source device extracts congestion information from CETH to perform quantitative and diversified congestion control actions. When the RoCE protocol message does not carry the ECN mark, the destination device can reply with ACK confirmation information carrying indication information to notify the source device that there is no network congestion on the current network path, so that the source device can maintain or restore the high-speed transmission rate in time.
[0208] In this way, on the one hand, by aggregated carrying of indication information, congestion information and ACK, the sending of independent CNPs is avoided, the notification overhead is reduced, which is beneficial to reducing the delay of congestion notification in high-traffic scenarios and improving the response speed of the destination device.
[0209] On the other hand, through the indication information, the source device can obtain the network congestion situation in the first place, trigger the congestion control earlier to adjust the sending rate, and improve the response speed of the source device. When the network congestion is relieved, the source device can also be informed through the indication information to restore the sending rate in time, thus improving the network bandwidth utilization.
[0210] Finally, the existing RDMA network congestion control has little notification information and slow network congestion convergence speed. The present application uses CETH to carry detailed congestion information of the network, such as congestion degree, congestion point, queue depth, network delay and other information of different dimensions, which is conducive to the source device to adjust the sending rate to the target rate in one step according to the detailed congestion information, achieve rapid convergence, and realize diversified adjustments such as the number of messages and sending time, which greatly improves the effect of congestion control.
[0211] It should be understood that in the various method embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0212] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0213] In the above embodiments, the description of each embodiment has different emphases. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0214] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the systems, devices and units described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0215] In addition, each functional module in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0216] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above embodiments, a person skilled in the art should also understand that any modification, deformation, or equivalent replacement of some of the technical features of the technical solutions described in the above embodiments should fall within the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for RoCE network congestion control, characterized in that: The method comprises: The first network device sends a RoCE protocol message to the second network device; The first network device receives a confirmation message from the second network device, the confirmation message including confirmation information and indication information of the RoCE protocol message, the indication information being used to indicate whether congestion occurs in a network path between the first network device and the second network device, and in the case where the indication information indicates that congestion occurs in the network path, the confirmation message further includes congestion information, and the congestion information specifically includes at least one of a congestion degree, a congestion location, a message queue length, and a network delay of the network path; The first network device performs congestion control according to the congestion information in at least one of the following ways: The first network device adjusts the sending rate of the first network device according to the congestion level, wherein there is a corresponding relationship between the congestion level and the sending rate, the congestion level belongs to one of a plurality of different levels of congestion levels, and different levels of congestion levels correspond to different sending rates; or The first network device determines the number of packets to be sent in a next time window according to at least one of the congestion position and the packet queue length; or, The first network device adjusts the sending rate of the first network device or determines the number of messages to be sent in the next time window according to the network delay.
2. The method according to claim 1, characterized in that The confirmation message specifically includes a basic transmission header (BTH) field and an extension field. The confirmation information and the indication information are carried in the BTH field, and the congestion information is carried in the extension field.
3. The method according to claim 1, characterized in that The confirmation message specifically includes a BTH field, and the confirmation information, the indication information and the congestion information are all carried in the BTH field.
4. The method according to any one of claims 1 to 3, characterized in that: When the indication information indicates that the network path is not congested, the first network device performing congestion control based on the confirmation message includes: The first network device maintains a sending rate of the first network device unchanged.
5. The method according to any one of claims 1 to 3, characterized in that: When the indication information indicates that the network path is not congested, the first network device performing congestion control based on the confirmation message includes: The first network device sets a sending rate of the first network device to a preset rate.
6. A method for RoCE network congestion control, characterized in that: The method comprises: The second network device receives the RoCE protocol message from the first network device; The second network device checks whether the RoCE protocol message carries an explicit congestion notification; The second network device generates a confirmation message according to the inspection result, the confirmation message including confirmation information and indication information of the RoCE protocol message, the indication information being used to indicate whether congestion occurs in the network path between the first network device and the second network device, and in the case where the indication information indicates that congestion occurs in the network path, the confirmation message also includes congestion information, and the congestion information specifically includes at least one of the congestion degree, congestion location, message queue length, and network delay of the network path; the congestion information is used by the first network device to perform congestion control; The second network device sends the confirmation message to the first network device, where the confirmation message is used by the first network device to perform congestion control; The first network device performs congestion control according to the congestion information in at least one of the following ways: The first network device adjusts the sending rate of the first network device according to the congestion level, wherein there is a corresponding relationship between the congestion level and the sending rate, the congestion level belongs to one of a plurality of different levels of congestion levels, and different levels of congestion levels correspond to different sending rates; or The first network device determines the number of packets to be sent in a next time window according to at least one of the congestion position and the packet queue length; or, The first network device adjusts the sending rate of the first network device or determines the number of messages to be sent in the next time window according to the network delay.
7. The method according to claim 6, characterized in that Before the second network device generates a confirmation message according to the checking result, the method further includes: The second network device generates the congestion information.
8. The method according to claim 7, characterized in that When the congestion information includes the congestion degree, the second network device obtains the congestion degree in at least one of the following ways: The second network device determines the congestion degree according to the proportion of RoCE protocol messages carrying explicit congestion notifications in historical message reception records; or, Obtain the congestion level by in-band network telemetry INT; or, The congestion level is obtained by on-site operation management and maintenance IOAM.
9. The method according to any one of claims 6 to 8, characterized in that: The confirmation message specifically includes a basic transmission header BTH field and an extension field. The confirmation information and the indication information are carried in the BTH field, and the congestion information is carried in the extension field.
10. The method according to any one of claims 6 to 8, characterized in that: The confirmation message specifically includes a BTH field, and the confirmation information, the indication information and the congestion information are all carried in the BTH field.
11. A device, characterized in that: The device is applied to a first network device, and includes: A message sending module, used for sending a RoCE protocol message to the second network device; a message receiving module, configured to receive a confirmation message from a second network device, the confirmation message including confirmation information and indication information of the RoCE protocol message, the indication information being used to indicate whether congestion occurs in the network path between the first network device and the second network device, and in the case where the indication information indicates that congestion occurs in the network path, the confirmation message further includes congestion information, and the congestion information specifically includes at least one of the congestion degree, congestion location, message queue length, and network delay of the network path; The congestion control module is configured to perform congestion control according to the congestion information, including at least one of the following methods: adjusting the sending rate of the first network device according to the congestion degree, wherein there is a corresponding relationship between the congestion degree and the sending rate, the congestion degree belongs to one of a plurality of different levels of congestion degrees, and different levels of congestion degrees correspond to different sending rates; or, Determine the number of packets to be sent in a next time window according to at least one of the congestion position and the packet queue length; or, The sending rate of the first network device is adjusted according to the network delay or the number of messages to be sent in the next time window is determined.
12. The device according to claim 11, characterized in that The confirmation message specifically includes a basic transmission header BTH field and an extension field. The confirmation information and the indication information are carried in the BTH field, and the congestion information is carried in the extension field.
13. The device according to claim 12, characterized in that The confirmation message specifically includes a BTH field, and the confirmation information, the indication information and the congestion information are all carried in the BTH field.
14. The device according to any one of claims 11 to 13, characterized in that: The congestion control module is specifically configured to maintain the sending rate of the first network device unchanged when the indication information indicates that the network path is not congested.
15. The device according to any one of claims 11 to 13, characterized in that: The congestion control module is specifically configured to set the sending rate of the first network device to a preset rate when the indication information indicates that the network path is not congested.
16. A device, characterized in that: The device is applied to a second network device, comprising: A message receiving module, used to receive a RoCE protocol message from a first network device; A congestion information determination module, configured to check whether the RoCE protocol message carries an explicit congestion notification; A notification aggregation sending module, used for generating a confirmation message according to the inspection result, the confirmation message including confirmation information and indication information of the RoCE protocol message, the indication information being used to indicate whether the network path between the first network device and the second network device is congested, and in the case where the indication information indicates that the network path is congested, the confirmation message also includes congestion information, and the congestion information specifically includes at least one of the congestion degree, congestion location, message queue length, and network delay of the network path; the congestion information is used for the first network device to perform congestion control; and is also used to send the confirmation message to the first network device, and the confirmation message is used for the first network device to perform congestion control; The first network device performs congestion control according to the congestion information in at least one of the following ways: The first network device adjusts the sending rate of the first network device according to the congestion level, wherein there is a corresponding relationship between the congestion level and the sending rate, the congestion level belongs to one of a plurality of different levels of congestion levels, and different levels of congestion levels correspond to different sending rates; or The first network device determines the number of packets to be sent in a next time window according to at least one of the congestion position and the packet queue length; or, The first network device adjusts the sending rate of the first network device or determines the number of messages to be sent in the next time window according to the network delay.
17. The device according to claim 16, characterized in that The congestion information determination module is further configured to generate the congestion information.
18. The device according to claim 17, characterized in that When the congestion information includes the congestion degree, the congestion information determination module is specifically configured to: Determining the congestion degree according to the proportion of RoCE protocol messages carrying explicit congestion notifications in historical message reception records; or, Obtain the congestion level by in-band network telemetry INT; or, The congestion level is obtained by on-site operation management and maintenance IOAM.
19. The device according to any one of claims 16 to 18, characterized in that: The confirmation message specifically includes a basic transmission header BTH field and an extension field. The confirmation information and the indication information are carried in the BTH field, and the congestion information is carried in the extension field.
20. The device according to any one of claims 16 to 18, characterized in that The confirmation message specifically includes a BTH field, and the confirmation information, the indication information and the congestion information are all carried in the BTH field.
21. A device, characterized in that The device includes a host system and a first network device; the host system is used to interact with the first network device to realize data transmission, and the first network device is used to execute the method according to any one of claims 1-5.
22. A device, characterized in that The device includes a host system and a second network device; the host system is used to interact with the second network device to realize data transmission, and the second network device is used to execute the method described in any one of claims 6-10.
Citation Information
Patent Citations
Congestion control method and device
CN107135164A
Congestion control method, device, equipment and storage medium
CN110445722A
Technologies for congestion control for IP-routable RDMA over converged ethernet
US20190044861A1
Network Congestion Control Method, Device, and System
US20190116126A1
Managing congestion in a network adapter based on host bus performance
US20200145349A1
Cited By
Roce network congestion control method and related apparatus
WO2022048647A1
