A RoCE network load balancing method and device based on end-network cooperation, electronic equipment and computer readable medium

Abstract

The application discloses a RoCE network load balancing method and device based on end-network cooperation, electronic equipment and a computer readable medium, wherein the method comprises the following steps: when sending a RoCE message at a sending end server, according to the type of the message, writing a corresponding load balancing mode mark in a predetermined reserved field of an InfiniBand base address transmission header; after a network switching device receives the RoCE message, analyzing the load balancing mode mark, and directly calling a corresponding load balancing algorithm for message forwarding according to the analysis result, the load balancing algorithm comprising a flow-by-flow hash algorithm and a packet-by-packet hash algorithm. The application effectively solves the problem of uneven load when elephant flows and mouse flows coexist in a RoCE network, has the advantages of high bandwidth utilization and low forwarding delay, and greatly reduces the processing burden and operation and maintenance complexity of the network device.

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, and in particular to a RoCE network load balancing method and apparatus, electronic device, and computer-readable medium based on end-to-end network collaboration. Background Technology

[0002] With the advancement of data center network technology, RoCE (RDMA over Converged Ethernet) technology has become a key component of modern data center network design. RoCE technology significantly improves network transmission efficiency and reduces CPU load by implementing efficient RDMA communication over Ethernet, thereby enhancing the data center's ability to process large-scale data.

[0003] Equal-cost multipath routing (ECMP) is a fundamental technology for achieving load balancing and high availability in data center networks. It allows traffic destined for the same destination to be distributed across multiple equally cost-effective paths. However, traditional ECMP employs a static per-flow hash algorithm based on 5-tuples.

[0004] In practical applications, RoCE networks face the problem of uneven load sharing and low bandwidth utilization when elephant streams (data packets carried in ROCE) and mouse streams (ROCE protocol messages) coexist. Traditional ECMP's flow-by-flow hashing algorithm causes traffic with similar characteristics to always choose a fixed path. This not only leads to congestion on that path, causing high latency and packet loss, but also results in idle bandwidth on other equally costly paths, leading to low overall network utilization.

[0005] like Figure 1 As shown, existing solutions rely on network-side devices to passively identify and classify traffic using complex rules such as Deep Packet Inspection (DPI) and Access Control Lists (ACLs). This involves setting complex ACL policies on the network side to identify RoCE protocol packets and RoCE data packets, performing packet-by-packet load balancing for protocol packets and flow-by-flow load balancing for data packets. This approach has the following drawbacks: 1) It consumes expensive and limited hardware table resources such as TCAM in network equipment, which restricts the scale of the network; 2) Configuring and maintaining a large number of ACL rules is extremely complex, prone to errors, and has high operation and maintenance costs. Summary of the Invention

[0006] According to a first aspect of the present invention, a RoCE network load balancing method based on end-to-end network collaboration is provided, comprising the following steps: When sending a RoCE message at the sending server, the corresponding load balancing mode flag is written into the reserved field of the InfiniBand base address transport header according to the message type. After receiving a RoCE message, the network switching device parses the load balancing mode flag and directly calls the corresponding load balancing algorithm to forward the message based on the parsing result. The load balancing algorithms include flow-by-flow hashing algorithm and packet-by-packet hashing algorithm.

[0007] Furthermore, the message type is determined based on the opcode attribute of the RDMA work request; if it is a connection management or acknowledgment message, it is determined to be a "protocol message" that is sensitive to order; if it is a large-scale data read / write message, it is determined to be a "data message" that is sensitive to throughput.

[0008] Furthermore, the corresponding load balancing mode flag is written into the reserved field of the InfiniBand base address transmission header. Specifically, the value "1" is written for protocol messages to indicate flow-by-flow, and the value "2" is written for data messages to indicate packet-by-packet.

[0009] Furthermore, the network switching equipment has a dedicated TCAM matching entry pre-installed to match specific header features in RoCE messages.

[0010] Furthermore, when the network switching device matches a specific header feature in the RoCE message, it triggers hardware parsing logic to extract the load balancing mode flag from the InfiniBand base address transport header.

[0011] Furthermore, the load balancing mode flag is parsed, and the corresponding load balancing algorithm is directly called for packet forwarding based on the parsing result. When the load balancing mode flag is set to "1", the per-stream hash algorithm is invoked for packet forwarding. When the load balancing mode flag is set to "2", the packet forwarding algorithm is called.

[0012] Furthermore, the load balancing mode flag is inserted into a pre-reserved field of the InfiniBand base address transport header by the RDMA protocol stack or smart NIC on the sending server when constructing the RoCE message.

[0013] According to a second aspect of the present invention, a RoCE network load balancing device based on end-to-end network collaboration is provided, comprising: The write module is used to write the corresponding load balancing mode flag in the pre-reserved field of the InfiniBand base address transport header according to the type of the RoCE message when the sending server sends the RoCE message. The parsing and forwarding module is used by network switching devices to parse the load balancing mode mark after receiving RoCE messages, and directly call the corresponding load balancing algorithm to forward the message based on the parsing result. The load balancing algorithms include flow-by-flow hashing algorithm and packet-by-packet hashing algorithm.

[0014] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory, a processor, and a computer program, wherein the computer program is stored in the memory, and the processor executes the computer program to perform a RoCE network load balancing method based on end-to-end network collaboration according to the first aspect.

[0015] According to a fourth aspect of the present invention, a computer-readable medium having processor-executable non-volatile program code is provided, the program code causing the processor to perform a RoCE network load balancing method based on end-to-end network collaboration according to the first aspect.

[0016] A RoCE network load balancing method based on end-to-end network collaboration according to an embodiment of the present invention has the following beneficial effects: Reduce network-side overhead: Existing network devices need to deploy complex ACL logic to distinguish between data packets and protocol packets in order to ensure that RoCE data packets are load-balanced per packet and RoCE protocol packets are load-balanced per flow. This invention transfers the policy decision and packet identification overhead from the network to the server, which greatly saves valuable hardware resources such as ACL and TCAM of network devices and reduces processing latency.

[0017] Improve bandwidth utilization: By employing packet-by-packet load balancing, dynamic load balancing algorithms, and other methods to effectively break up large data flows, avoid link congestion, and fully utilize multi-path bandwidth.

[0018] Ensuring the reliability of critical traffic: By maintaining flow-by-flow load balancing of protocol messages, ensuring message order and low latency, the stable operation of the RoCE network is guaranteed.

[0019] Enhanced flexibility and scalability: Decoupling policy configuration from network topology facilitates fine-grained traffic scheduling and adapts to the needs of future new applications.

[0020] Simplified operation and maintenance management: There is no need to configure and maintain a large number of complex ACLs or policy routing rules on network devices, which reduces operation and maintenance costs and the probability of errors.

[0021] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the working mechanism of the existing technical solution.

[0023] Figure 2 This is a flowchart of a RoCE network load balancing method based on end-to-end network collaboration according to an embodiment of the present invention.

[0024] Figure 3 This is an architecture diagram of an end-to-end network collaborative load balancing system in a RoCE network load balancing method based on end-to-end network collaboration according to an embodiment of the present invention.

[0025] Figure 4 This is a flowchart of server-side processing in a RoCE network load balancing method based on end-to-end network collaboration according to an embodiment of the present invention.

[0026] Figure 5 This is a flowchart of the network device-side processing in a RoCE network load balancing method based on end-to-end network collaboration according to an embodiment of the present invention.

[0027] Figure 6 This is a structural diagram of a RoCE network load balancing device based on end-to-end network collaboration according to an embodiment of the present invention.

[0028] Figure 7 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0029] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, further illustrating the present invention.

[0030] First, combine Figures 2-5 This invention describes a RoCE network load balancing method based on end-to-end network collaboration, which is used to solve the load imbalance problem when elephant flow and mouse flow coexist in RoCE network, and has a wide range of applications.

[0031] like Figures 2-5 As shown in the figure, a RoCE network load balancing method based on end-to-end network collaboration according to an embodiment of the present invention has the following steps: like Figure 2 As shown, in S1, when sending a RoCE message at the sending server, the corresponding load balancing mode flag is written into the reserved field of the InfiniBand base address header (BTH) according to the message type.

[0032] Furthermore, in this embodiment, the message type is determined based on the opcode attribute of the RDMA work request; if it is a connection management or acknowledgment message, it is determined to be a "protocol message" that is sensitive to order; if it is a large-scale data read / write message, it is determined to be a "data message" that is sensitive to throughput. It should be noted that: the application submits a work request through the RDMA Verbs interface (such as ibv_post_send) to prepare to send data, and the RDMA driver stack or smart network card firmware parses the work request and determines the message type based on its opcode and other attributes.

[0033] Furthermore, in this embodiment, writing the corresponding load balancing mode flag in the predetermined reserved field of the InfiniBand base address transmission header specifically means: writing the value "1" for protocol messages to indicate flow-by-flow, and writing the value "2" for data messages to indicate packet-by-packet.

[0034] Furthermore, in this embodiment, the load balancing mode flag is inserted into a predetermined reserved field of the InfiniBand base address transport header by the RDMA protocol stack or smart network card on the sending server when constructing the RoCE message.

[0035] like Figure 2 As shown in S2, after receiving the RoCE message, the network switching device parses the load balancing mode flag and directly calls the corresponding load balancing algorithm to forward the message based on the parsing result. The load balancing algorithms include flow-by-flow hashing algorithm and packet-by-packet hashing algorithm.

[0036] Furthermore, in this embodiment, the network switching device has a pre-configured dedicated TCAM matching entry for matching specific header features in RoCE packets. This dedicated TCAM matching entry enables rapid packet identification and classification. The design of this TCAM entry is key to distinguishing this solution from traditional ACL solutions; it is not used to match complex, configurable IP 5-tuple rules, but rather to precisely match specific header features in RoCE packets (such as UDP destination port 4791).

[0037] Furthermore, in this embodiment, when the network switching device matches a specific header feature in the RoCE message (i.e., a UDP message with a destination port number of 4791), it triggers hardware parsing logic to extract the load balancing mode flag in the InfiniBand base address transport header.

[0038] Furthermore, in this embodiment, parsing the load balancing mode flag and directly calling the corresponding load balancing algorithm for packet forwarding based on the parsing result specifically involves: When the load balancing mode flag is set to "1", the per-stream hash algorithm is invoked for packet forwarding. When the load balancing mode flag is set to "2", the packet forwarding algorithm is called.

[0039] As described above, according to an embodiment of the present invention, a RoCE network load balancing method based on end-to-end network collaboration moves the intelligent decision-making of load balancing strategies from the network side to the server side, which best understands the traffic characteristics, thus decoupling "decision-making and execution". The server side completes intelligent decision-making and marking when packets are generated, and the network-side devices only need to execute agilely based on the marking, thereby maximizing network resource utilization and simplifying operation and maintenance to the extreme.

[0040] The above combined with the appendix Figures 2-5 A RoCE network load balancing method based on end-to-end network collaboration according to an embodiment of the present invention is described. Furthermore, the present invention can also be applied to a RoCE network load balancing device based on end-to-end network collaboration.

[0041] like Figure 6 As shown, according to a second aspect of the present invention, a RoCE network load balancing device based on end-to-end network collaboration is provided, comprising: The writing module 100 is used to write the corresponding load balancing mode flag in the pre-reserved field of the InfiniBand base address transmission header according to the type of the RoCE message when sending the RoCE message at the sending server. The parsing and forwarding module 200 is used by the network switching device to parse the load balancing mode mark after receiving the RoCE message, and directly call the corresponding load balancing algorithm to forward the message based on the parsing result. The load balancing algorithms include flow-by-flow hash algorithm and packet-by-packet hash algorithm.

[0042] The above combined with the appendix Figure 6 A RoCE network load balancing device based on end-to-end network collaboration according to an embodiment of the present invention is described. Furthermore, the present invention can also be applied to an electronic device.

[0043] According to a third aspect of the present invention, such as Figure 7 As shown, an electronic device is provided, including: a memory 1, a processor 2, and a computer program 3. The computer program 3 is stored in the memory 1, and the processor 2 runs the computer program 3 to execute a RoCE network load balancing method based on end-to-end network collaboration, as described in the first aspect.

[0044] According to a fourth aspect of the present invention, a computer-readable medium having processor-executable non-volatile program code is provided, the program code causing the processor to perform a RoCE network load balancing method based on end-to-end network collaboration according to the first aspect.

[0045] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of a computer program from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the ASIC can reside within a device. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. The present invention also provides a program product comprising executable instructions stored in the readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the execution of the executable instructions by the at least one processor causes the device to implement the RoCE network load balancing method based on end-to-end network collaboration provided in the various embodiments described above. In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0046] Common terms RDMA stands for Remote Direct Memory Access.

[0047] RoCE—RDMA over Converged Ethernet—is a network bearer scheme for RDMA technology that allows RDMA to run on standard Ethernet infrastructure.

[0048] RoCEv1 -- RoCE version 1, based on the Ethernet link layer, is not routable.

[0049] RoCEv2—RoCE version 2, based on the UDP / IP protocol, is routable and is the core application scenario of this invention.

[0050] ECMP stands for Equal Cost Multiple Path.

[0051] BTH – Base Transport Header. The header of transport layer data packets in the InfiniBand protocol, containing crucial information such as the opcode and destination queue number. Its reserved fields are the key carriers of this invention.

[0052] Elephant streams are network data streams that are massive in volume, long in duration, and consume a lot of bandwidth, such as video streams and backup data streams.

[0053] Mouse flow – a network control flow with small data volume and short duration, but extremely sensitive to latency and packet loss, such as RoCE protocol messages.

[0054] Relevant standards and protocols RFC 4392 – IP over InfiniBand (IPoIB) Architecture. This RFC provides an architectural reference for carrying IP protocols over InfiniBand networks, and its encapsulation concepts are related to RoCE technology.

[0055] The InfiniBand Architecture Specification – published by the InfiniBand Trade Association – authoritatively defines the details of each layer of the InfiniBand protocol, including the format of the base address transport header.

[0056] It should be noted that, in this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0057] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. A RoCE network load balancing method based on end-to-end network collaboration, characterized in that, It includes the following steps: When sending a RoCE message at the sending server, the corresponding load balancing mode flag is written into the reserved field of the InfiniBand base address transport header according to the type of the message. After receiving the RoCE message, the network switching device parses the load balancing mode flag and directly calls the corresponding load balancing algorithm to forward the message based on the parsing result. The load balancing algorithm includes flow-by-flow hashing algorithm and packet-by-packet hashing algorithm. The network switching device has a dedicated TCAM matching entry pre-configured for matching specific header features in RoCE messages.

2. The RoCE network load balancing method based on end-to-end network collaboration as described in claim 1, characterized in that, The message type is determined based on the opcode attribute of the RDMA work request; if it is a connection management or acknowledgment message, it is determined to be a "protocol message" that is sensitive to order. If it is a large-scale data read / write message, it is judged as a "data message" that is sensitive to throughput.

3. The RoCE network load balancing method based on end-to-end network collaboration as described in claim 2, characterized in that, Specifically, writing the corresponding load balancing mode flag in the predetermined reserved field of the InfiniBand base address transmission header means writing the value "1" for the protocol message to indicate flow-by-flow and writing the value "2" for the data message to indicate packet-by-packet.

4. The RoCE network load balancing method based on end-to-end network collaboration as described in claim 1, characterized in that, When a network switching device matches a specific header feature in a RoCE message, it triggers hardware parsing logic to extract the load balancing mode flag from the InfiniBand base address transport header.

5. The RoCE network load balancing method based on end-to-end network collaboration as described in claim 3, characterized in that, Parsing the load balancing mode flag and directly calling the corresponding load balancing algorithm for packet forwarding based on the parsing result is as follows: When the load balancing mode flag is set to "1", the per-stream hash algorithm is invoked for packet forwarding. When the load balancing mode flag is set to "2", the packet forwarding algorithm is called.

6. The RoCE network load balancing method based on end-to-end network collaboration as described in claim 1, characterized in that, The load balancing mode flag is inserted into a pre-reserved field in the InfiniBand base address transport header by the RDMA protocol stack or smart NIC on the sending server when constructing the RoCE message.

7. A RoCE network load balancing device based on end-to-end network collaboration, characterized in that, Include: The writing module is used to write the corresponding load balancing mode flag in the predetermined reserved field of the InfiniBand base address transmission header according to the type of the RoCE message when the sending server sends the RoCE message. The parsing and forwarding module is used by the network switching device to parse the load balancing mode tag after receiving the RoCE message, and directly call the corresponding load balancing algorithm to forward the message based on the parsing result. The load balancing algorithm includes flow-by-flow hashing algorithm and packet-by-packet hashing algorithm. The network switching device has a dedicated TCAM matching table entry for matching specific header features in the RoCE message.

8. An electronic device, characterized in that, include: The device includes a memory, a processor, and a computer program, wherein the computer program is stored in the memory, and the processor executes the computer program to perform the RoCE network load balancing method based on end-to-end network collaboration as described in any one of claims 1 to 6.

9. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to run the RoCE network load balancing method based on end-to-end network collaboration as described in any one of claims 1-6.

Images