Congestion detection method and device

Abstract

A congestion detection method and apparatus, the method being applied to a source device, a remote direct memory access (RDMA) communication connection being established between the source device and a destination device, the method comprising: the source device sending a first detection message to the destination device, the first detection message being generated by the source device in a user mode, the network congestion detection module is used for detecting a network congestion condition between source end equipment and destination end equipment; the source end device receives a second detection message sent by the destination end device for the first detection message; and the source end equipment determines the network congestion condition between the source end equipment and the destination end equipment according to the second detection message. The congestion detection method can be realized in the user state of the source end equipment, a switch and a network card are not required to support the congestion detection function, and the requirements on the switch and the network card are lower.

Description

[0001] This application claims priority to the Chinese patent application filed with the Intellectual Property Office of the People's Republic of China on June 16, 2023, with application number 202310730083.9 and invention name "Congestion Detection Method", all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of communication technology, and in particular to a congestion detection method and device. Background Art

[0003] Remote direct memory access (RDMA) communication relies on a lossless network and requires special network configuration support from the network switch, such as priority-based flow control (PFC) and explicit congestion notification (ECN) configuration. However, there are many switch and network card manufacturers in the data center (the models, versions and specifications of switches and network cards are not uniform). When supporting the same ECN and flow control algorithms such as the DCQCN algorithm in the same RDMA network, configuration management is complex and difficult to implement in the data center. Summary of the invention

[0004] A congestion detection method and device provided in an embodiment of the present application are used to provide a congestion detection method for lossless network communication without the need for switch or network card configuration.

[0005] In the first aspect, the present application provides a congestion detection method, which can be applied to a source device, which can be a computing device such as a server, and a remote direct memory access (RDMA) connection is established between the source device and the destination device. The method includes: the source device (or a component in the source device such as a processor) generates a first detection message in user mode, and the first detection message is used to detect the network congestion between the source device and the destination device. The source device can send the first detection message to the destination device through the RDMA communication connection. Correspondingly, after receiving the first detection message, the destination device can generate a second detection message based on the first detection message and send the second detection message to the source device. The source device determines the network congestion between the source device and the destination device based on the second detection message received from the destination device through the RDMA communication connection.

[0006] Through the above method, the source device generates a first detection message for detecting the degree of network congestion between the source device and the destination device in the user state, and sends the first detection message to the destination device through the RDMA connection between the source device and the destination device. The source device determines the degree of network congestion between the source device and the destination device according to the second detection message returned by the destination device in response to the first detection message. The congestion detection method can be implemented in the user state of the source device, and does not require the switch and the network card to support the congestion detection function, and has lower requirements for the switch and the network card. It can save the labor cost and equipment cost of the lossless network construction and maintenance of RDMA communication, and is easy to implement and has strong applicability.

[0007] In a possible design, the first detection message is generated by a first intermediate library running in user mode in the source device, and the second detection message is generated by a second intermediate library running in user mode in the destination device based on the first detection message. The first intermediate library may be a layer located between the network communication driver layer (such as including a network card driver) and the application layer (including one or more user mode applications) of the source device, and similarly, the second intermediate library may be a layer located between the hardware driver layer of the destination device and the application running in user mode.

[0008] Through the above method, the source device generates a first detection message through the first intermediate library. Since the first intermediate library is located between the application layer and the network communication driver layer, the data generated by the application needs to communicate with the external device (such as the destination device) through the first intermediate library and the network communication driver layer. Therefore, based on the first intermediate library, network congestion detection can be initiated for the data of various applications respectively, unifying the congestion detection method for the business layer. Moreover, when the congestion detection message is generated by the first intermediate library, the processor (such as CPU) of the source device does not need to switch to the kernel state, which can greatly reduce the CPU overhead and latency. The second detection message generated by the second intermediate library at the destination device has the same effect, which can reduce the CPU overhead and latency of the entire system and improve the congestion detection efficiency in the RDMA network. In addition, since the network cards of the source device and the destination device are not required to generate the detection message, the method of the present application further releases the requirements for the network card.

[0009] In one possible design, the RDMA connection established between the source device and the destination device includes an RDMA connection established between the first intermediate library and the second intermediate library; a method under this architecture includes, the first intermediate library in the source device sends a first detection message to the second intermediate library through the RDMA connection between the first intermediate library and the second intermediate library, and receives the second detection message sent by the second intermediate library through the RDMA connection; thereafter, the first intermediate library determines the network congestion situation between the source device and the destination device according to the second detection message.

[0010] Through the above method, the first intermediate library communicates through the RDMA connection with the second intermediate library. The first intermediate library detects the network congestion between the source device and the destination device based on the received second detection message. The processor of the source device does not need to switch to the kernel state to determine the network congestion situation, which can greatly reduce the CPU overhead and latency and improve the congestion detection efficiency in the RDMA network. In addition, since the network cards of the source device and the destination device are not required to determine the network congestion situation, the method of the present application further releases the requirements for the network card.

[0011] In one possible design, the second detection message includes service congestion information of the first service. Specifically, the destination device obtains the second detection message after adding fill information in the first detection message, and the fill information includes service congestion information of the first service. The service congestion information is used to indicate the degree of congestion of data of the first service running in the destination device in the destination device; thereafter, when the source device determines the network congestion situation between the source device and the destination device according to the second detection message, it can determine the network congestion situation between the source device and the destination device according to the service congestion information of the first service.

[0012] Through the above method, the destination device can determine the congestion level of the first service, and carry the service congestion information indicating the congestion level through the second detection message. In this way, the source device can obtain the congestion level of the service in the destination device, and thus determine the network congestion level between the source device and the destination device according to the congestion level of the service in the destination device, and realize congestion detection at the service granularity level. In the prior art, the realization of service granular congestion detection requires the configuration of a specific switch, which has high requirements on the switch. The present application can be realized without relying on the configuration of a specific switch.

[0013] In one possible design, the first detection message includes first time information, where the first time information is used to indicate a sending time of the first detection message, that is, a time when the source device sends the first detection message, and the second detection message includes the first time information;

[0014] When the source device determines the network congestion between the source device and the destination device according to the second detection message, the network congestion can be determined according to the RTT of the first detection message, wherein the RTT of the first detection message is determined according to the sending time of the first detection message and the receiving time of the second detection message (i.e., the time when the source device receives the second detection message). Alternatively, the source device can determine the network congestion between the source device and the destination device according to the round trip time (RTT) of the first detection message and the service congestion information of the first service.

[0015] Through the above method, determining the network congestion level between the source device and the destination device based on the RTT of the first detection message is a relatively accurate and intuitive measurement method. This application can implement this method in the user state of the computing device to obtain accurate congestion detection results. On this basis, the network congestion level between the source device and the destination device can also be measured in combination with the service congestion level to achieve service granularity congestion detection, providing a multi-dimensional, fine-grained congestion detection method. Both methods do not rely on specific switch and network card configurations, have lower requirements for switches and network cards, and are more widely applicable to networks.

[0016] In a possible design, when the source device determines the network congestion situation between the source device and the destination device according to the second detection message, the network congestion measurement value may be determined according to the second detection message, and the network congestion measurement value is used to indicate the network congestion situation between the source device and the destination device;

[0017] After the source device determines the network congestion situation between the source device and the destination device according to the second detection message, it can also convert the network congestion measurement value into a rate parameter according to the measurement algorithm. The source device uses the rate parameter to control the packet rate at which the source device sends data to the destination device.

[0018] Through the above method, the source device determines the rate parameter according to the congestion detection result, that is, the network congestion degree between the source device and the destination device, and controls the packet sending rate of the source device to send data to the destination device according to the rate parameter to avoid packet loss. Congestion detection and flow rate control are implemented in the user state of the computing device. RDMA lossless communication can be achieved without configuring the switch and the network card, and the type and function requirements of the switch and the network card are lower. At the same time, the labor and equipment costs of lossless network construction and scheduling of RDMA communication can be reduced.

[0019] In a possible design, before the source device sends the data of the first service to the destination device, it can also adjust the communication mode of the first service to the RDMA communication mode, such as adjusting from the TCP communication mode to the RDMA communication mode. Afterwards, when the source device sends the data of the first service to the destination device, it uses the RDMA communication mode to send the data packet of the first service to the destination device.

[0020] Through the above method, the source device adjusts the communication mode of the service to the RDMA communication mode, and uses the RDMA communication mode to access service data. Since the computing device performs congestion detection and flow rate control in the user state, RDMA lossless communication can be achieved. Therefore, this method is applicable to a wider range of networks and has better mobility. RDMA communication can be achieved without changing the software and hardware configuration of the existing network. Compared with TCP communication, it can achieve low latency, high throughput and low CPU overhead.

[0021] In the second aspect, the present application also provides a congestion detection device, which has the function of implementing the method described in the first aspect above. The beneficial effects can be found in the description of the first aspect, which will not be repeated here. The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the structure of the device includes a sending module, a receiving module, a determination module, and optionally, a communication mode conversion module. These modules can implement the functions of the method example of the first aspect above. Please refer to the detailed description in the method example for details, which will not be repeated here.

[0022] In a third aspect, the present application also provides a computing device. The computing device includes a processor and a memory, and the memory is used to store instructions and / or data. The memory is coupled to the processor, and when the processor executes the program instructions stored in the memory, the method described in the first aspect can be implemented. The computing device also includes a communication interface in the structure, which is used to communicate with other devices, such as sending a first detection message, receiving a second detection message, etc.

[0023] In a fourth aspect, the present application also provides a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer executes the method in the above-mentioned first aspect and various possible designs of the first aspect.

[0024] In a fifth aspect, the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect and various possible designs of the first aspect.

[0025] In a sixth aspect, the present application also provides a computing chip, which is connected to a memory, and the chip is used to read and execute a software program stored in the memory, and to execute the methods in the above-mentioned first aspect and various possible implementation methods of the first aspect.

[0026] For the description of the beneficial effects of any of the second to sixth aspects mentioned above, please refer to the description of the technical effects in any possible implementation method of the first aspect mentioned above, and this application will not repeat them here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of an application scenario to which the embodiment of the present application may be applicable;

[0028] Figure 2 A schematic diagram comparing the working mechanisms of TCP and RDMA;

[0029] Figure 3This is a schematic diagram of x86 CPU permission division;

[0030] Figure 4 A schematic diagram of a system architecture that may be applicable to the embodiments of the present application;

[0031] Figure 5 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0032] Figure 6 A flowchart corresponding to a congestion detection method provided in an embodiment of the present application;

[0033] Figure 7 A schematic diagram of a congestion detection scenario provided in an embodiment of the present application;

[0034] Figure 8 A schematic diagram of the structure of a congestion detection device provided in an embodiment of the present application;

[0035] Fig. 9 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0036] Fig.10 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The congestion detection algorithm provided in this application can be applied to a variety of application scenarios, such as data interaction scenarios. In this scenario, Figure 1 As shown, hosts can communicate with each other through the network to exchange data. In one possible scenario, when a source host (such as host #1) sends data to a destination host (such as host #n) through the network, when the amount of data sent exceeds the transmission capacity of the network or the processing capacity of the destination host, network congestion may occur, and network congestion may cause data loss (i.e., packet loss). The congestion detection algorithm provided by the present application can realize an end-to-end network congestion detection that does not rely on network configuration.

[0038] In order to better understand the solutions of the embodiments of the present application, some relevant terms and concepts that may be involved in the embodiments of the present application are first introduced below.

[0039] 1. Network

[0040] A network is a network that connects multiple computer systems or hosts with independent functions through network devices (such as switches, routers, etc.) and lines (such as network cables, optical fibers, etc.) (or wireless) to form a data link, thereby achieving the purpose of resource sharing and communication. In addition to various network devices and data transmission lines, the hardware facilities that constitute the network also require fully functional network software, such as network protocols. At present, the mainstream network protocols include TCP / IP protocol, RDMA protocol, etc.

[0041] 2. Transmission Control Protocol (TCP) / Internet Protocol (IP)

[0042] TCP and IP are two different protocols that work together to enable data transmission on the Internet. The IP protocol was created for data transmission on the Internet and is responsible for the network layer that sends data from one computer to another. It is responsible for determining the addresses of the source and destination computers and is able to route through multiple intermediate points on the Internet. The TCP protocol was created for reliable data transmission on the Internet and is responsible for the transport layer that transfers data from one application to another. It ensures the integrity and reliability of data transmission by communicating with the other computer.

[0043] They mainly complete the following functions during data transmission:

[0044] (1) The TCP protocol first divides the data into several packets and writes a sequence number to each packet so that the receiving end can restore the data to its original format.

[0045] (2) The IP protocol writes the addresses of the sending host and the receiving host to each data packet. Once the source address and destination address are written, the data packet can be transmitted on the physical network. The IP protocol also has the function of using routing algorithms to select routes.

[0046] (3) These data packets can be transmitted through different transmission paths (routes). Due to different paths and other reasons, the order may be reversed, data may be lost, data may be distorted, or even repeated. These problems are handled by the TCP protocol, which has the function of checking and handling errors, and can request the sender to resend when necessary. In short, the IP protocol is responsible for data transmission, while the TCP protocol is responsible for reliable data transmission, that is, ensuring the integrity of the data packet.

[0047] Traditional data centers usually use Ethernet technology to build a network architecture and use TCP / IP network protocol for transmission. However, TCP / IP network communication is gradually not suitable for high-performance computing business requirements. Figure 2(a) The main limitations are as follows:

[0048] Limitation 1: TCP / IP protocol stack processing causes a delay of tens of microseconds.

[0049] When the TCP protocol stack receives / sends messages, the kernel needs to perform multiple context switches. Each switch consumes a delay of about 5 to 10 us. In addition, at least three data copies and reliance on the CPU for protocol encapsulation are required. This results in a fixed delay of tens of microseconds for protocol stack processing alone, making protocol stack delay the most obvious bottleneck in microsecond-level systems such as AI data computing and SSD distributed storage.

[0050] Limitation 2: TCP protocol stack processing causes the server CPU load to remain high.

[0051] In addition to the long fixed latency problem, TCP / IP networks require the host CPU to participate in protocol stack memory copies multiple times. The larger the network scale and the higher the network bandwidth, the greater the scheduling burden on the CPU when sending and receiving data, resulting in a continuous high CPU load. According to industry estimates: 1Hz of CPU is required to transmit 1bit of data, so when the network bandwidth reaches more than 25G (full load), for most servers, at least half of the CPU capacity will have to be used to transmit data.

[0052] In order to reduce network latency and CPU occupancy, the RDMA protocol was created.

[0053] 3. Remote direct memory access (RDMA)

[0054] See also Figure 2 (b) RDMA is a technology that bypasses the kernel of the remote host operating system to access data in its memory. In other words, it can transfer data directly from the memory of one computer to another, and quickly move data from one system to the remote system memory without the intervention of the operating systems of both parties. Since it does not go through the operating system and does not require time-consuming processing by the CPU, it not only saves a lot of CPU resources, but also improves system throughput and reduces the system's network communication latency. It is particularly suitable for wide application in large-scale parallel computer clusters.

[0055] RDMA has several major features: (1) Data is transmitted between remote machines over the network; (2) There is no involvement of the operating system kernel, and all content related to sending and transmitting is offloaded to the smart network card; (3) Data is directly transmitted between the user space virtual memory and the smart network card without involving the operating system kernel, and there is no additional data movement and copying.

[0056] 4. Kernel state / user state

[0057] It is well known that the CPU can execute software-defined instructions, but the range of instructions that software programs can execute is also changing with the development of the times.

[0058] In the early CPUs, any program could use the CPU to execute any instruction, including accessing external devices and setting internal mode control registers (such as the configuration of the interrupt controller). This mode of CPU operation allows software programs to use the computer flexibly and efficiently. However, since any program can execute instructions that have a greater impact on the CPU state, it may be easy for a software program to crash the entire computer (it may be a malicious program or an unconscious bug in the program, such as a program mistakenly writing data to the memory area where the operating system code is located and overwriting the operating system code). Especially after the computer has developed into a multi-user and multi-process model, it is more likely to be affected by other users.

[0059] In order to reduce the interference of software programs on computer systems, hierarchical protection is generally implemented in modern computer systems. That is, certain operations must be performed by certain roles with corresponding permissions according to the severity of the impact on the computer system. For example, the operations mentioned above such as directly accessing hardware and modifying hardware working modes require the highest permissions to execute.

[0060] This kind of computer protection requires the cooperation of the CPU and the operating system. Modern CPUs generally provide multiple levels of operating permissions. Currently, common X86 CPUs and ARM CPUs support multi-level operating permissions, as follows Figure 3A schematic diagram of x86 cpu authority division is shown. The authority division of ARM cpu is similar, but more refined. The operating system is generally divided into multiple running states to cooperate with the cpu. The common states of the operating system are user state and kernel state. Among them, the kernel state generally has the highest authority, and all instructions and operations are allowed to be executed by the cpu. On the x86 cpu, it is generally in ring 0 authority; while the user state is generally a lower authority, for example, on the x86 cpu, it is generally in ring 3 authority. In this state, the software program can only execute limited instructions and operations, and high-risk operations are not allowed by the cpu hardware, such as configuring the cpu internal control register, accessing the memory address of the kernel part, etc. The software executes high-risk operation commands in the user state, which will trigger the cpu hardware to generate an abnormal signal. When the cpu finds the abnormal signal, it will automatically switch to the kernel state authority and execute the processing logic predetermined by the operating system. The kernel state processing logic may help execute this user instruction request, or it may stop execution and report that the user program has a fatal error, so the entire user program is stopped. When the operating system needs to execute software under different permissions, it usually switches the cpu's authority state to the corresponding state first, and then executes the corresponding software program. When a user-mode program needs to perform some high-risk operations, it generally needs to tell the kernel through a system call, and the relevant program in the kernel mode will complete the operation requested by the user-mode program.

[0061] In summary, the user state and kernel state of software are actually different permission states of the CPU when we set the software to execute. The instructions and operation ranges allowed to be executed by the software are different in different permission states. Programs in user state are generally executed in a low-privilege CPU state. If high-privilege instructions need to be executed, they need to apply to the operating system, which will execute them in the kernel state.

[0062] According to the different permission states in which the software is expected to run, we generally divide them into user-state programs and kernel-state programs. Generally, user-state programs refer to programs that run under the lowest CPU permissions, while kernel-state programs can refer to programs that need to run under the highest CPU permissions in a narrow sense, or they can refer to programs that run under permissions higher than user-state permissions, such as the device driver in the figure above.

[0063] 5. Priority-based flow control (PFC)

[0064] PFC is a priority-based flow control method that suspends the peer traffic before the switch port buffer overflows, preventing packet loss.

[0065] The general rule is that when the receiver is unable to process the received message, in order to prevent the message from being discarded, the receiver needs to notify the sender of the message to temporarily stop sending. The notification method is to send a pause message to the sender, and the pause message carries the pause time. The sender will wait within this timeout period until it receives a control frame with a timeout of 0 and then continue to send.

[0066] The disadvantage is that once the link is suspended, the switch can no longer send any data packets. If the suspension is caused by some low-priority data packets, it is not worth it to cause the high-priority data packets to be suspended as well.

[0067] In view of this, PFC allows the creation of 8 virtual channels on an Ethernet link, and specifies corresponding priorities for each virtual channel, allowing any virtual channel to be suspended and restarted individually, while allowing traffic on other virtual channels to pass uninterrupted.

[0068] PFC mainly achieves the purpose of preventing packet loss by controlling the way downstream network devices control upstream network devices. However, the most effective flow control is to control the sending speed of the source host that generates data, so that the speed at which the host injects data into the network slows down. This is the fundamental way to solve the problem.

[0069] This led to the development of the ECN congestion algorithm, which is used to achieve end-to-end flow control.

[0070] 6. Explicit congestion notification (ECN)

[0071] ECN is an end-to-end congestion management method, which means from the source host to the destination host that generates the data flow. The main method is: when the data packets marked by ECN reach their original destination, the congestion notification will be fed back to the source host node, and the source host node will respond to the congestion notification by limiting the speed of network data packets for the problematic queue pairs (QP).

[0072] ECN interaction process:

[0073] (1) The message sent by the sender supports ECN.

[0074] (2) When the switch receives the message when the queue is congested, it changes the ECN field to 11 and sends it out. Other switches in the network will transparently transmit the message.

[0075] (3) The receiving end receives a message with ECN 11 and finds congestion, so it processes the message normally.

[0076] (4) The receiving end generates a congestion notification and sends a CNP message every ms with the ECN field set to 01, requiring that the message cannot be discarded by the network. The receiving end can send a single CNP for multiple packets marked with the same QP by ECN.

[0077] (5) After receiving the CNP message, the switch forwards it normally.

[0078] (6) After receiving the CNP message with ECN marked as 01, the sender parses it and applies the rate limiting algorithm to the corresponding flow (corresponding to the QP of ECN).

[0079] 7. Static library / dynamic library

[0080] A library is a binary executable file used to store functional functions, including static libraries and dynamic libraries. Among them, dynamic libraries can also be called dynamic link libraries (DLL). Among them, a dynamic library is a library containing code and data that can be used by multiple applications at the same time. Dynamic linking provides a way for a process to call functions that do not belong to its executable code. The executable code of a function is located in a dynamic library, which protects one or more functions that have been compiled, linked, and stored separately from the processes that use them. Static libraries refer to some common codes that need to be used repeatedly in applications. These codes are compiled into "library" files. In the linking step, the connector will obtain the required code from the library file and copy it to the generated executable file. In general, when an executable program actually uses a dynamic library, it does not load the compiled content in the dynamic library into the executable program, but directly searches the dynamic library. If a static library is used, the compiled content in the static library will be loaded into the executable program.

[0081] The following is an explanation based on the above terminology:

[0082] As can be seen from the above, compared with TCP, RDMA can achieve the effects of high bandwidth, low latency and low resource utilization, but RDMA communication relies on a lossless network environment, and RDMA is very sensitive to packet loss. TCP packet loss retransmission is an accurate retransmission. When retransmission occurs, the received messages at the receiving end will be removed to reduce unnecessary retransmissions, so that the lost messages will be retransmitted. However, in the RDMA protocol, every time a packet is lost, all messages included in the entire message will be retransmitted, which will greatly reduce the transmission efficiency of RDMA. RDMA can transmit at full rate in a lossless state, but once packet loss and retransmission occur, the performance will drop sharply. In order to ensure that RDMA throughput is not affected, the packet loss rate is preferably zero. Therefore, in order to maximize the performance of RDMA, it is necessary to build a lossless network environment with no packet loss for RDMA, and the key to achieving no packet loss is to solve network congestion.

[0083] Currently, there are three options for RDMA network layer protocols: Infiniband, (RDMA over ConvergedEthernet, RoCE), and (internet Wide Area RDMA Protocol, iWARP).

[0084] InfiniBand is a network protocol designed specifically for RDMA. It was proposed by IBTA (InfiniBand Trade Association). It ensures lossless network at the hardware level, with extremely high throughput and extremely low latency. However, InfiniBand switches are special products provided by specific manufacturers and use private protocols. Most existing networks use IP Ethernet networks, and InfiniBand cannot meet interoperability requirements. At the same time, closed architectures also have the problem of vendor lock-in. For business systems that require large-scale elastic expansion in the future, if they are locked in by a single manufacturer, the risk is uncontrollable.

[0085] iWarp, a network protocol that allows RDMA to be performed on TCP, requires a special network card that supports iWarp and supports the use of RDMA on standard Ethernet switches. However, due to the limitations of the TCP protocol, its performance loses most of the advantages of the RDMA protocol.

[0086] RoCE is a network protocol that allows applications to access remote memory through Ethernet. It was also proposed by IBTA and is a protocol that applies RDMA technology to Ethernet. It also supports the use of RDMA on standard Ethernet switches, which requires a special network card that supports RoCE. Currently, RoCE has two protocol versions, RoCEv1 and RoCEv2: RoCEv1 is a link layer protocol that allows any two hosts in the same broadcast domain to access directly; RoCEv2 is a network layer protocol that can implement routing functions, allowing hosts in different broadcast domains to access each other through Layer 3, and is based on UDP protocol encapsulation. However, due to the sensitivity of RDMA to packet loss, and the fact that traditional Ethernet is a best-effort network with packet loss problems, in order to ensure high throughput and low latency of the RDMA network, the switch needs to support lossless Ethernet.

[0087] Currently, lossless protection is mainly achieved by deploying PFC and ECN functions in the network through specific switches. Among them, PFC technology realizes flow control of RDMA exclusive queues on the connection path, and performs back pressure on upstream device traffic when congestion occurs at the switch inlet. ECN technology can be used to achieve end-to-end congestion control. When the switch outlet is congested, the data packet is marked with ECN, and the source host is asked to reduce the sending rate. However, due to the large number of switch and network card manufacturers (network card models, versions and specifications are not unified), when supporting the same ECN congestion and flow control algorithms in the same RDMA network, the configuration management is complicated and difficult to implement in the data center.

[0088] In view of this, this embodiment provides a congestion detection method, which provides a process for implementing congestion detection in the user state of the source host, and can achieve lossless communication of RDMA without relying on specific switch and network card configuration.

[0089] Next, the technical solution of this application is described in detail with reference to the accompanying drawings.

[0090] Figure 4 A schematic diagram of a system architecture that may be applicable to the embodiments of the present application, the system may include multiple computing devices (such as Figure 4 The computing devices 70-75 in the figure are connected to each other through the network 30.

[0091] The system can be a data center, a computing device cluster, a cloud computing system, a storage system, a supercomputing system, a communication system consisting of user devices and business servers, etc. The computing device can be a physical machine or a virtual machine. Physical machines include but are not limited to servers, desktop computers, laptops, and mobile devices.

[0092] In this embodiment, data can be transmitted between two computing devices through the network 30, wherein the sender of the data can be referred to as the source device or the sender device, and the destination of the data can be referred to as the destination device. The source device can be specifically used to send data, and the destination device is used to receive data. For example, in a business scenario, the system is a communication system composed of a user device and a business server, wherein the source device can be a business server for providing media services such as web pages, audio and video, and the destination device can be a user device. The user device can request the source device to access a web page, and the source device can carry the content of the web page to be accessed in a series of data packets and send it to the destination device. In another business scenario, the system is a cloud computing system for providing cloud services, such as virtual machine services. The multiple virtual machines included in the cloud computing system are used to run the tenant's business. At this time, the source device and the destination device can be two virtual machines of the tenant. When the tenant's business requires back-end data sharing, the source device (a virtual machine) may send the data of the business to the destination device (another virtual machine). Regardless of the system, during the provision of services or operation, the multiple computing devices included in the system can interact with each other through the network 30. In this process, data may be transmitted in the form of data messages, data packets, data frames, etc., which is not limited in this embodiment.

[0093] The network 30 includes a plurality of network devices (such as Figure 4 The network devices 50-52 in the figure are used for forwarding data, including but not limited to switches, routers, hubs, optical modems and other devices with data forwarding functions. According to the communication distance, the network 30 can be divided into a wide area network (also known as the Internet) and a local area network. In one example, a switch can be used to connect some computing devices to form a local area network. The aggregation switch (or router) can be used to connect different network segments to realize data interaction between computing devices in different local area networks.

[0094] exist Figure 4 In the embodiment, any two computing devices are connected to each other through the network 30. The connection between two computing devices includes an indirect connection between the two computing devices. For example, A and B are connected to each other through one or more other devices (such as network devices). For example, A and B are connected to each other through wired or wireless communication. A and C are directly connected, C and B are directly connected, and A and B are connected to each other through C. For example, Figure 4In the example, computing device 70 and computing device 71 are indirectly connected via network device 50. At this time, the role of network device 50 is to connect computing device 70 and computing device 71 together via a network cable so that they can be connected on the physical line. Computing device 70 and computing device 75 are indirectly connected via network devices 50-52. In some applications, connection may also have other meanings or other names, such as connection may also be called network channel or link, etc. In short, the communication connection between A and B enables data exchange between A and B. Specifically, based on the different network protocols adopted by the computing devices, the connection between two computing devices may include TCP connection, RDMA connection, etc.

[0095] Figure 5 A schematic diagram of a structure that may be applicable to a computing device provided in this embodiment.

[0096] At the hardware level, the computing device is equipped with a processor, memory and network card. At the software level, the computing device can run an operating system, application programs and network acceleration intermediate libraries. Among them, the application programs and network acceleration intermediate libraries run in user mode. The application program can be an application program of the business party, or an application program used to provide services to the business party, such as big data, databases, distributed storage and other applications. The network acceleration intermediate library is a library located between the application program and the network communication driver. It can be a static library or a dynamic library. It can be used to achieve network acceleration. Therefore, it is called a network acceleration intermediate library.

[0097] Specifically, the network acceleration intermediate library can be used to implement network congestion monitoring, flow control and other functions. Specifically, the network acceleration intermediate library can be used to trigger the congestion detection process. In each congestion detection process, the network acceleration intermediate library is also used to generate a detection message and measure the network congestion between the source device and the destination device based on the detection message, thereby realizing network congestion monitoring. The flow control function can be used to control the packet sending rate of the source device to the destination device according to the network congestion situation.

[0098] In one example, Figure 5 The two computing devices shown are the source device and the destination device. An RDMA connection is established between the network acceleration intermediate library of the source device (recorded as the first intermediate library) and the network acceleration intermediate library of the destination device (recorded as the second intermediate library), such as a RoCE connection, which can be used to transmit business data, detection messages, etc. Optionally, a TCP connection can also be established between the source device and the destination device, and the TCP connection can also be used to transmit business data. The source device controls the packet sending rate of the business data based on the network congestion between the source device and the destination device to avoid packet loss.

[0099] The following applies to Figure 5Taking the system shown in the figure as an example, the congestion detection method of this embodiment is introduced in detail.

[0100] Figure 6 A schematic diagram of a flow chart corresponding to the congestion detection method provided in this embodiment. The method can be performed by Figure 5 The two computing devices in the source device and the destination device are executed, or the components (such as processors or network acceleration intermediate libraries) included in the two computing devices are executed. For ease of description, the two computing devices are referred to as the source device and the destination device below. Figure 6 As shown, the method includes the following process:

[0101] Step 601: The source device generates a congestion detection message in user mode. The congestion detection message is used to detect the degree of network congestion between the source device and the destination device.

[0102] In one example, a congestion detection message is generated by a first intermediate library running in user mode of a source device.

[0103] When the first intermediate library detects that the congestion detection condition is met, it generates a congestion detection message for this congestion detection.

[0104] The congestion detection condition includes: detecting a connection between the source device and the destination device for transmitting business data, such as a TCP connection or an RDMA connection. Taking TCP connection as an example, when the first intermediate library detects the first TCP connection between the source device and the destination device, a congestion detection is initiated. Figure 7 It is understood that the process may include: when detecting that a new TCP connection is established between the source device and the destination device, the first intermediate library generates a congestion detection message for this congestion detection. Exemplarily, the congestion detection message is used to detect the network congestion level of the TCP connection.

[0105] The congestion detection condition may also include: reaching a set detection time. For an existing TCP connection, congestion detection may be initiated periodically. For example, after initiating congestion detection for a TCP connection for the first time, congestion detection may be initiated periodically for the TCP connection. In this way, whenever the detection time of the next cycle is reached, the first intermediate library generates a congestion detection message for this congestion detection.

[0106] The congestion detection condition may also include: receiving a congestion detection instruction, the congestion detection instruction is used to indicate the start of congestion detection, the congestion detection instruction may be triggered by an application running in the source device or other program (such as a kernel-mode program), and is not specifically limited. Optionally, the congestion detection instruction may also specify the start of congestion detection for a TCP connection between the source device and the destination device. When the first intermediate library receives the congestion detection instruction, it generates a congestion detection message for this congestion detection.

[0107] In another example, an application running in user mode of the source device may generate a congestion detection message when detecting that the congestion detection condition is met. Similarly, the application generates a congestion detection message, see the above introduction, which will not be repeated here.

[0108] The congestion detection message in this embodiment may have a customized format, and the format of the congestion detection message will be introduced below.

[0109] Step 602: The source device sends the congestion detection message to the destination device.

[0110] In one example, an RDMA connection is established between the first intermediate library of the source device and the second intermediate library of the destination device. When the source device sends the congestion detection message to the destination device, the first intermediate library may send the congestion detection message to the second intermediate library through the RDMA connection. The RDMA connection may be established when starting this congestion detection, such as establishing the RDMA connection when it is detected that the congestion detection conditions are met. Alternatively, the RDMA connection may have been established before starting this congestion detection.

[0111] As mentioned above, there are three types of RDMA networks, Infiniband, RoCE, and iWARP. Therefore, the RDMA connection here can specifically be an Infiniband connection, a RoCE connection, and an iWARP connection. According to the two standards of connection and reliability, RDMA includes three transmission modes, namely: reliable connection (Reliable connected, RC), unreliable connection (Unreliable connected, UC), and unreliable datagram (Unreliable datagram, UD). This embodiment can use any transmission mode under any RDMA connection to transmit congestion detection messages. In one embodiment, the first intermediate library sends a congestion detection message through the UD mode of the RoCE connection, or the first intermediate library sends a congestion detection message through the RC mode of the RoCE connection, and so on. Since the reliability of the UD mode is relatively low, packet loss may occur due to network congestion, and the RC mode does not allow packet loss due to high reliability requirements, the UD mode is used to send and receive congestion detection messages, and whether packet loss occurs on the RoCE connection can be detected, thereby more accurately determining the degree of network congestion between the source device and the destination device. The RoCE connection has a binding relationship with a TCP connection between the source device and the destination device, and the detection message transmitted through the RoCE connection is used to detect the network congestion level of the corresponding TCP connection (referring to the TCP connection with the RoCE connection). Optionally, this embodiment can also use the RC mode of the RoCE connection to transmit the service data that should be transmitted by the corresponding TCP connection. For details, please refer to the description of step 608, which will not be emphasized here.

[0112] Step 603: After receiving the congestion detection message (referred to as the first detection message) from the source device, the destination device generates a new congestion detection message (referred to as the second detection message) according to the first congestion detection message.

[0113] In one example, after the second intermediate library receives the first detection message, the second intermediate library generates the second detection message according to the first detection message. In another example, an application in user mode of the destination device generates the second detection message according to the first detection message.

[0114] Taking the second intermediate library generating the second detection message as an example, specifically, in one way, the second detection message can be generated by the second intermediate library after adding padding information in the first detection message. In another way, the second detection message and the first detection message are two independent messages. The first detection message and the second detection message have the same format. Alternatively, the first detection message and the second detection message have different formats.

[0115] The format of the congestion detection message in this embodiment is introduced below in conjunction with Table 1. Table 1 is an example of the format of a first detection message provided by this embodiment, wherein B represents byte.

[0116] Table 1

[0117]

[0118] The following is an introduction to each parameter in Table 1. Items (1), (3), and (4) of the following parameters can be filled in by the first intermediate library, and items (2), (5) to (9) can be filled in by the second intermediate library.

[0119] (1) Special characters can be used to indicate that the message type is a congestion detection message.

[0120] (2) Packet status, which is used to indicate whether the message status is abnormal. If the destination device detects an abnormal status, such as when a detection message is lost, the destination device changes the packet status of the second detection message in the reply to the abnormal status.

[0121] (3) The packet sequence number is the sequence number of the first detection message itself. For example, on the source device side, the sequence number of the congestion detection message periodically generated for the same TCP connection can be a series of consecutive numbers. For example, the sequence number of the first congestion detection message is a set starting sequence number (such as 0 or 1), and the sequence number of the congestion detection message generated for the same TCP connection is equal to the sequence number of the previous congestion detection message plus 1. For example, the sequence numbers of multiple congestion detection messages sent sequentially by the first intermediate library for the same TCP connection are 0, 1, 2, 3, ..., or 1, 2, 3, 4 ..., respectively. The starting sequence number can also be other values, which are not specifically limited.

[0122] (4) The packet sending timestamp is used to indicate the sending time of the first detection message. Exemplarily, the first intermediate repository sends the first detection message at the sending time indicated by the packet sending timestamp.

[0123] (5) The reply packet sequence number is the sequence number of the first detection message, which is used to indicate which first detection message the destination device uses to generate the second detection message. For example, if the second intermediate library generates the second detection message based on a first detection message, and the packet sequence number of the first detection message is 10, then the reply packet sequence number in the generated second detection message is also 10.

[0124] (6) Packet receiving time difference, used to indicate the time difference between the time when the destination device receives the first detection message and the time when the source device sends the first detection message. The time when the source device sends the first detection message can be determined according to the packet sending timestamp.

[0125] (7) Packet loss number, used to indicate the number of packet losses counted by the destination device for the congestion detection message sent by the source device. The destination device can determine the number of packet losses based on the sequence numbers of two adjacent congestion detection messages received from the source device. For example, if the sequence numbers of the two adjacent congestion detection messages are 1 and 10, the destination device determines that the number of packet losses is 10-1=9, that is, 9 congestion detection messages are lost. It should be understood that when the UD mode is used to send and receive congestion detection messages, congestion detection messages may be lost.

[0126] (8) Packet sending rate, used to indicate the rate at which the second intermediate repository sends the second detection message.

[0127] (9) Service congestion information, used to indicate the congestion level of the current service at the destination device. The current service may be the service to which the data currently transmitted on the specified TCP connection belongs, and the specified TCP connection may be the TCP connection that triggers this congestion detection. The congestion level of a service at the destination device may be determined based on the frequency of the service calling the second intermediate library to send data, or based on the amount of data of the service sent within a period of time, or based on the congestion level of the data of the service at the network card of the destination device, such as the average queuing time and number of queuing times of the data of the service in the sending queue of the network card within a period of time.

[0128] In this embodiment, the service data is sent by the application calling the network intermediate acceleration library. Therefore, the network intermediate acceleration library can sense the degree of service congestion. However, in the prior art, the network card can only passively send service data and cannot sense the degree of service congestion. Therefore, in the prior art, the source device cannot know the degree of service congestion in the destination device.

[0129] Taking Table 1 as an example, in one example, the first detection message and the second detection message have the same format, such as the first detection message and the second detection message both include the above parameters (1)-(9), and the difference is that the first detection message (1), (3), and (4) are filled in by the first intermediate library, and the second detection message (2), (5)-(9) are filled in by the second intermediate library. In another example, the first detection message and the second detection message have different formats, such as the first detection message only includes the above items (1), (3), and (4) or one or more of the above items (1), (3), and (4), and the second detection message only includes the above items (2), (5)-(9) or one or more of the above items (2), (5)-(9).

[0130] It should be noted that the format shown in Table 1 is only an example, and this embodiment does not limit the parameters included in the congestion detection message, the order of arrangement of the parameters, the byte length of each parameter, etc. In addition, the format of the congestion detection message in actual application may have more or fewer parameters relative to those shown in Table 1. For example, the congestion detection message in this embodiment may not have one or more information such as packet status, packet receiving time difference, packet sending rate, etc., or the congestion detection message in this embodiment may also include parameter synchronization information, and parameter synchronization is used to indicate the status of the first intermediate library, and can be used to synchronize the status of the first intermediate library to the second intermediate library, etc. This embodiment does not limit this either.

[0131] Step 604: The destination device sends the second detection message to the source device.

[0132] Specifically, in one example, the second intermediate library of the destination device sends the second detection message to the second intermediate library through the UD mode under the RDMA connection with the first intermediate library of the source device. Correspondingly, the second intermediate library of the destination device receives the second detection message based on the UD mode under the RDMA connection.

[0133] Step 605: The source device determines a congestion detection result according to the second detection message, where the congestion detection result is used to indicate a degree of network congestion between the source device and the destination device.

[0134] Among them, the network congestion level between the source device and the destination device refers to the congestion level of a certain network channel between the source device and the destination device. For example, the network channel is the TCP connection that triggers this congestion detection (or a TCP connection that has a binding relationship with the RDMA connection used to transmit congestion detection messages) or the RDMA connection between the source device and the destination device for transmitting business data, etc.

[0135] In one embodiment, the first intermediate library may determine one or more parameters such as the round trip time (RTT) of the first probe message, the degree of business congestion, the packet loss rate, whether the RDMA communication has timed out, etc. based on the second probe message, and the first intermediate library may use the value of the one or more parameters (which may be called network congestion measurement value) as the congestion detection result, that is, the congestion detection result may include the value of one or more parameters such as the round trip time (RTT) of the first probe message, the degree of business congestion, the packet loss rate, whether the RDMA communication has timed out, etc.

[0136] Among them, the RTT of the first detection message can be determined according to the reception time of the second detection message and the sending time of the first detection message, such as the RTT of the first detection message is equal to the difference between the time when the first intermediate library receives the second detection message and the time when the first intermediate library sends the first detection message. The degree of business congestion can be determined according to the business congestion information carried in the second detection message. Similarly, the packet loss rate can be determined according to the number of packet losses carried in the second detection message. Whether the RDMA communication has timed out can be determined according to whether the second detection message corresponding to the first detection message is received within the set time interval after the first intermediate library sends the first detection message. If it is received, it has not timed out, otherwise it is determined that the RDMA communication has timed out. The second detection message corresponding to the first detection message refers to the packet sequence number of the reply carried is the same as the packet sequence number of the first detection message. For example, the packet sequence number of the first detection message is 10, and the packet sequence number of the reply carried in the second detection message corresponding to the first detection message is also 10.

[0137] In another embodiment, the source device determines the congestion detection result according to one or more parameters such as the round trip time (RTT) of the first detection message, the degree of service congestion, the packet loss rate, whether the RDMA communication has timed out, etc. For example, the first intermediate library determines the congestion detection result according to a preset corresponding relationship and the above one or more parameters, and the preset corresponding relationship includes the corresponding relationship between different intervals corresponding to the values ​​of one or more parameters and the network congestion measurement result.

[0138] The following are some examples to describe how to determine the congestion detection result between the source device and the destination device:

[0139] In an example, the first intermediate library determines the congestion detection result according to the RTT of the first detection message and the preset correspondence shown in Table 2.

[0140] Table 2

[0141] RTT of the first detection message Congestion detection results RTT(i)>high(RTT) Overload high(RTT)>RTT(i)>low(RTT) normal RTT(i)<low(RTT) Low load

[0142] Wherein, RTT(i) represents the RTT value of the first detection message. High(RTT) and low(RTT) are preset values.

[0143] In yet another example, the first intermediate library determines the congestion detection result according to the service congestion degree and the preset corresponding relationship shown in Table 3.

[0144] Table 3

[0145] Traffic congestion level Congestion detection results d(r)(i)>high(d(r)) Overload high(d(r))>d(r)(i)>low(d(r)) normal d(r)(i)<low(d(r)) Low load

[0146] Wherein, d(r)(i) represents the value of the service congestion degree. high(d(r)) and low(d(r)) are preset values.

[0147] In another example, the first intermediate library determines the congestion detection result according to the RTT of the first detection message, the service congestion level, and the preset corresponding relationship shown in Table 4.

[0148] Table 4

[0149] RTT / service congestion level of the first detection message Congestion detection results RTT(i)>high(RTT)||d(r)(i)>high(d(r)) Overload high(RTT)>RTT(i)>low(RTT)||high(d(r))>d(r)(i)>low(d(r)) normal RTT(i)<low(RTT)||d(r)(i)<low(d(r)) Low load

[0150] Among them, || means or.

[0151] In one example, the first intermediate library determines the network congestion metric value according to RTT, service congestion level, packet loss rate, and whether RoCE communication times out.

[0152] Taking the RoCE network as an example, if the RoCE communication times out, the RTT value of the first detection message may be assigned a value exceeding high(RTT). When the RoCE communication does not time out, the RTT of the first detection message is a value determined based on the reception time of the second detection message and the transmission time of the first detection message. Similarly, when the packet loss rate exceeds a first preset value (the first preset value is a value greater than 0), the RTT value of the first detection message is assigned a value exceeding high(RTT). When the packet loss rate does not exceed the first preset value, the RTT of the first detection message is a value determined based on the reception time of the second detection message and the transmission time of the first detection message.

[0153] Thereafter, the congestion detection result is determined according to the RTT of the first detection message and the service congestion degree. Please refer to the introduction of the above-mentioned related examples, which will not be described in detail here.

[0154] Step 606: The source device determines a packet sending rate from the source device to the destination device according to the determined congestion detection result.

[0155] Taking the first intermediate library as an example, the first intermediate library converts the network congestion measurement value into a rate parameter according to the flow control rate measurement algorithm, and uses the rate parameter to control the packet sending rate of the source device to send the specified business data to the destination device.

[0156] Taking the flow control rate measurement algorithm including RTT and service congestion degree as an example, different flow control rate measurement algorithms can be used in different network congestion situations, as shown below:

[0157] Overload: RTT(i)>high(RTT)||d(r)(i)>high(d(r));

[0158] R(i)=w(RTT(i),d(r)(i))*β;

[0159] Low load: RTT(i) <low(RTT)||d(r)(i)<low(d(r));

[0160] R(i)=NIC max rate;

[0161] Normal: high(RTT)>RTT(i)>low(RTT)||high(d(r))>d(r)(i)>low(d(r));

[0162] R(i)=w(RTT(i),d(r)(i))*α;

[0163] Among them, || represents or, R(i) represents the rate parameter, NIC max rate represents the set rate value, and w() represents the set function. α and β are coefficients and can be constants.

[0164] The rate parameter R(i) may be used as the packet sending rate, or the packet sending rate may be further determined based on the rate parameter R(i).

[0165] Step 607: The source device adjusts the communication mode of the designated service to the RDMA communication mode.

[0166] In this embodiment, the network acceleration intermediate library may also include a communication conversion function, which can be used to adjust the communication model of the service, such as converting the TCP communication mode to the RDMA communication mode. Here, the first intermediate library can adjust the communication mode of the specified service to the RDMA communication mode, and the specified service refers to the service to which the data transmitted on the TCP connection that triggered this congestion detection belongs. In one embodiment, this embodiment can transmit the service data of the specified service through an RDMA connection (such as a RoCE connection) used to transmit congestion detection messages, that is, encapsulate the service data into an RC stream for transmission. This method implements congestion detection and flow rate control in the user state of the computing device, can ensure a lossless network environment for RDMA communication, and converts the communication mode of the service data from traditional TCP communication to RDMA communication based on the communication conversion function to save the CPU resources of the computing device, and also improves throughput and reduces network communication delays.

[0167] It should be noted that step 607 is an optional step and not a must-be-performed step.

[0168] Step 608: The source device sends data to the destination device using the determined packet sending rate.

[0169] When step 607 is not executed, the source device sends data of the designated service to the destination device at a determined packet sending rate.

[0170] After executing step 607, the source device uses the determined packet rate and RDMA communication mode to send the data of the designated service to the destination device. For example, in this embodiment, the RC stream corresponding to the service data of the designated service can be sent to the destination device at the determined packet rate through the RC mode under the RoCE connection between the source device and the destination device (such as the aforementioned RoCE connection for transmitting congestion detection messages). Please note that periodic congestion detection can be initiated for the RoCE connection later. At this time, congestion detection messages may be added to the RC stream transmitted on the RoCE connection, but the transmission modes used by the two may be different, such as the service data (RC stream) is transmitted in RC mode, and the congestion detection message can be transmitted in UD mode. Of course, the same mode can also be used for transmission, such as both using RC mode to transmit service data and congestion detection messages, etc., which is not specifically limited. Correspondingly, at this time, the service congestion information in the second detection message returned by the destination device is used to indicate the degree of congestion of the service data currently transmitted on the RoCE connection at the destination device.

[0171] The above design implements congestion detection and flow rate control in the user state of the computing device, and can realize RDMA lossless communication without configuring the switch and the network card, providing a more lightweight lossless network communication solution, which has low requirements on the type and function of the switch and the network card. The switch and the network card do not need to support lossless network configuration, and are applicable to a wide range of networks. The solution provided in this embodiment can be applied without changing the software and hardware configuration of the existing network, thereby realizing RDMA communication in a variety of networks and reducing the labor and equipment costs of lossless network construction and scheduling of RDMA communication.

[0172] The method provided by the embodiment of the present application is introduced above in combination with the accompanying drawings, and the device provided by the embodiment of the present application is introduced below in combination with the accompanying drawings.

[0173] Based on the same inventive concept as the method embodiment, the present application embodiment also provides a congestion detection device, which is used to perform the above Figure 5 The method implemented by the source device in the method embodiment. Figure 8 As shown, the congestion detection device 800 includes a sending module 801, a receiving module 802, a determining module 803, and optionally, a communication mode conversion module 804. Specifically, in the congestion detection device 800, each module is connected via a communication path.

[0174] In one example, an RDMA connection is established between the congestion detection apparatus 800 and the destination device.

[0175] The sending module 801 is used to send a first detection message to the destination device, where the first detection message is generated by the source device in the user mode and is used to detect the network congestion between the source device and the destination device; for details, see Figure 3 The description of step 602 in the method embodiment will not be repeated here.

[0176] The receiving module 802 is used to receive a second detection message sent by the destination device, where the second detection message is generated based on the first detection message; Figure 6 The description of step 603 to step 604 in the method embodiment will not be repeated here.

[0177] The determination module 803 is used to determine the network congestion between the apparatus and the destination device according to the second detection message. Figure 6 The description of step 605 in the method embodiment will not be repeated here.

[0178] In one possible implementation, the second detection message includes service congestion information of the first service, and the service congestion information is used to indicate the degree of congestion of data of the first service running in the destination device in the destination device; when the determination module 803 determines the network congestion situation between the device 800 and the destination device based on the second detection message, it is specifically used to: determine the network congestion situation based on the service congestion information of the first service.

[0179] In a possible implementation, the first detection message also includes first time information, and the first time information is used to indicate the sending time of the first detection message, and the second detection message includes the first time information; when the determination module 803 determines the network congestion situation between the device 800 and the destination device according to the second detection message, it is specifically used to: determine the network congestion situation according to the RTT of the first detection message, wherein the RTT of the first detection message is determined according to the sending time of the first detection message and the receiving time of the second detection message; or, determine the network congestion situation according to the RTT of the first detection message and the service congestion information of the first service.

[0180] In a possible implementation, when the determination module 803 determines the network congestion between the apparatus 800 and the destination device according to the second detection message, it is specifically used to: determine a network congestion metric value according to the second detection message, where the network congestion metric value is used to indicate the network congestion between the apparatus 800 and the destination device;

[0181] After determining the network congestion measurement value, the determination module 803 is further used to: convert the network congestion measurement value into a rate parameter according to a measurement algorithm; Figure 6The description of step 606 in the method embodiment is not repeated here. The rate parameter is used to control the rate at which the sending module 801 sends the data message of the first service to the destination device. For details, see Figure 6 The description of step 608 in the method embodiment will not be repeated here.

[0182] In a possible implementation, the communication mode conversion module 804 is used to adjust the communication mode of the first service from the TCP communication mode to the RDMA communication mode; Figure 6 The description of step 607 in the method embodiment will not be repeated here.

[0183] When the sending module 801 sends the data of the first service to the destination device, it is specifically used to: send the data of the first service to the destination device using the RDMA communication mode. Figure 6 The description of step 608 in the method embodiment will not be repeated here.

[0184] It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The functional modules in the embodiments of the present application may be integrated into one module, or each module may exist physically separately, or two or more modules may be integrated into one module, for example, the sending module 801 and the receiving module 802 are integrated into one module, or the determination module 803 and the communication module conversion module 804 are the same module. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0185] The present application also provides a computing device 900. Fig. 9 As shown, the computing device 900 includes: a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate through the bus 902. The computing device 900 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 900.

[0186] The bus 902 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9The bus 902 may include a path for transmitting information between various components of the computing device 900 (eg, the memory 906, the processor 904, and the communication interface 908).

[0187] The processor 904 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0188] The memory 906 may include a volatile memory, such as a random access memory (RAM). The processor 904 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0189] In one embodiment, the computing device 900 is used to execute the functions of the congestion detection device 800, wherein the memory 906 stores executable program codes, and the processor 904 executes the executable program codes to respectively implement the functions of the aforementioned sending module 801, receiving module 802, determining module 803, and communication mode conversion module 804, thereby implementing the congestion detection method. That is, the memory 906 stores instructions for the computing device 900 to execute the congestion detection method provided by the present application.

[0190] In another embodiment, the computing device 900 is used to execute the functions of the computing device 800, wherein the memory 906 stores an executable program code, and the processor 904 executes the executable program code to respectively implement the functions of the aforementioned determination module 801 and the sending module 802 ( Fig. 9 (not shown) to implement the congestion detection method. That is, the memory 906 stores instructions for the computing device 900 to execute the congestion detection method provided by the present application.

[0191] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 900 and other devices or communication networks.

[0192] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0193] Take the computing device 900 executing the function of the congestion detection device 800 as an example. Fig.10 As shown, the computing device cluster includes at least one computing device 900. The memory 906 in one or more computing devices 900 in the computing device cluster may store the same instructions for executing the congestion detection method.

[0194] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the congestion detection method. In other words, the combination of one or more computing devices 900 may jointly execute instructions for executing the congestion detection method.

[0195] It should be noted that the memory 906 in different computing devices 900 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the computing device. That is, the instructions stored in the memory 906 in different computing devices 900 can implement the functions of one or more modules among the sending module 801, the receiving module 802, the determining module 803, and the communication mode conversion module 804.

[0196] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the congestion detection method. In other words, the combination of one or more computing devices 900 may jointly execute instructions for executing the congestion detection method.

[0197] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the congestion detection method.

[0198] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the congestion detection method.

[0199] Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.

[0200] It should be understood that the processor mentioned in the embodiments of the present application can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor implemented by reading software code stored in a memory.

[0201] Exemplarily, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0202] It should be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DirectRambus RAM, DR RAM).

[0203] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0204] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0205] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0206] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0207] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0208] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0209] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A congestion detection method, characterized in that: Applied to a source device, a remote direct memory access (RDMA) connection is established between the source device and the destination device, and the method includes: The source device sends a first detection message to the destination device, where the first detection message is generated by the source device in a user state and is used to detect network congestion between the source device and the destination device; The source device receives a second detection message sent by the destination device in response to the first detection message; The source device determines the network congestion status between the source device and the destination device according to the second detection message.

2. The method according to claim 1, characterized in that The first detection message is generated by a first intermediate library running in user mode in the source device, and the second detection message is generated by a second intermediate library running in user mode in the destination device based on the first detection message.

3. The method according to claim 2, characterized in that The RDMA connection includes an RDMA connection established between the first intermediate library and the second intermediate library; The source device sends a first detection message to the destination device, including: The first intermediate library sends the first detection message to the second intermediate library through the RDMA connection; The source device receives a second detection message sent by the destination device in response to the first detection message; The first intermediate library receives the second detection message through the RDMA connection; The source device determines, according to the second detection message, a network congestion situation between the source device and the destination device, including: The first intermediate library determines the network congestion situation according to the second detection message.

4. The method according to any one of claims 1 to 3, characterized in that: The second detection message includes service congestion information of the first service, where the service congestion information is used to indicate the congestion degree of data of the first service running in the destination device in the destination device; The source device determines, according to the second detection message, a network congestion situation between the source device and the destination device, including: The source device determines the network congestion situation according to the service congestion information of the first service.

5. The method according to claim 4, characterized in that The first detection message further includes first time information, where the first time information is used to indicate a sending time of the first detection message; the second detection message includes the first time information; The source device determines, according to the second detection message, a network congestion situation between the source device and the destination device, including: The source device determines the network congestion situation according to the round-trip delay RTT of the first detection message and the service congestion information of the first service; The RTT is determined according to the sending time of the first detection message and the receiving time of the second detection message; or, The source device determines the network congestion situation according to the round-trip time RTT of the first detection message.

6. The method according to any one of claims 1 to 5, characterized in that: The source device determines, according to the second detection message, a network congestion situation between the source device and the destination device, including: The source device determines a network congestion metric value according to the second detection message, where the network congestion metric value is used to indicate a network congestion condition between the source device and the destination device; After the source device determines the network congestion between the source device and the destination device according to the second detection message, the method further includes: The source device converts the network congestion measurement value into a rate parameter according to a measurement algorithm; The source device uses the rate parameter to control a packet sending rate of data of the first service sent by the source device to the destination device.

7. The method according to claim 6, characterized in that Before the source device sends the data of the first service to the destination device, the method further includes: The source device adjusts the communication mode of the first service from the TCP communication mode to the RDMA communication mode; The source device sends data of a first service to the destination device, including: The source device sends the data message of the first service to the target device using the RDMA communication mode.

8. A congestion detection device, characterized in that: A remote direct memory access (RDMA) communication connection is established between the device and the destination device, and the device includes: A sending module, used for sending a first detection message to the destination device, where the first detection message is generated by the apparatus in user mode and is used for detecting a network congestion situation between the apparatus and the destination device; A receiving module, configured to receive a second detection message sent by the destination device in response to the first detection message; A determination module is used to determine the network congestion situation between the apparatus and the destination device according to the second detection message.

9. The device according to claim 8, characterized in that The first detection message is generated by a first intermediate library running in user mode in the source device, and the second detection message is generated by a second intermediate library running in user mode in the destination device based on the first detection message.

10. The device according to claim 9, characterized in that The RDMA connection includes an RDMA connection established between the first intermediate library and the second intermediate library; The sending module sends the first detection message to the destination device, specifically used to: send the first detection message to the second intermediate library through the RDMA connection; When receiving the second detection message sent by the destination device in response to the first detection message, the receiving module is specifically configured to receive the second detection message through the RDMA connection.

11. The device according to any one of claims 8 to 10, characterized in that: The second detection message includes service congestion information of the first service, where the service congestion information is used to indicate the congestion degree of data of the first service running in the destination device in the destination device; When the determination module determines the network congestion situation between the apparatus and the destination device according to the second detection message, it is specifically used to: determine the network congestion situation according to the service congestion information of the first service.

12. The device according to claim 11, characterized in that The first detection message further includes first time information, where the first time information is used to indicate a sending time of the first detection message; the second detection message includes the first time information; When the determination module determines the network congestion situation between the device and the destination device according to the second detection message, it is specifically used to: determine the network congestion situation according to the round-trip time RTT of the first detection message, wherein the RTT of the first detection message is determined according to the sending time of the first detection message and the receiving time of the second detection message; or, determine the network congestion situation according to the RTT of the first detection message and the service congestion information of the first service.

13. The device according to any one of claims 8 to 12, characterized in that: When the determination module determines the network congestion between the device and the destination device according to the second detection message, the determination module is specifically used to: determine a network congestion metric value according to the second detection message, wherein the network congestion metric value is used to indicate the network congestion between the device and the destination device; After determining the network congestion metric value, the determination module is further used to: convert the network congestion metric value into a rate parameter according to a metric algorithm; and use the rate parameter to control the packet sending rate of the sending module to send the data of the first service to the destination device.

14. The device according to claim 13, characterized in that The device also includes a communication mode conversion module; The communication mode conversion module is used to adjust the communication mode of the first service from the TCP communication mode to the RDMA communication mode; When the sending module sends the data of the first service to the destination device, it is specifically used to: send the data of the first service to the destination device using the RDMA communication mode.

15. A computing device, characterized in that: The device comprises a processor and a memory, wherein the memory stores instructions, and when the processor executes the instructions, the device executes the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or an instruction. When the computer program or the instruction is executed by the communication device, the method according to any one of claims 1 to 7 is implemented.

Images