A Remote Direct Memory Access Method and Device

Abstract

The present invention provides a remote direct memory access method and apparatus, which relates to the field of computer technology. The method includes: receiving a first VXLAN message sent by a first virtual machine to a second virtual machine via a VXLAN tunnel; the first VXLAN message is generated by performing VXLAN encapsulation on a RoCE message; performing VXLAN decapsulation on the first VXLAN message using a hardware programmable network card based on a virtual switch to obtain a first RoCE message; and performing access control on the first RoCE message using the hardware programmable network card, wherein the hardware programmable network card based on the virtual switch is preconfigured based on the software of the virtual switch to implement the virtual switch software functions through the network card hardware.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a remote direct memory access method and device. Background Art

[0002] Remote Direct Memory Access (RDMA) is a memory access technology, and RoCEv2 (RDMA over Converged Ethernet version 2, a remote direct memory access protocol based on Ethernet) is the second version of RoCE. RoCEv2 allows data to be transferred directly from the memory of one computer to another, quickly moving data from one system to the remote system's memory without the intervention of either operating system or processor processing, ultimately achieving high bandwidth, low latency, and low resource utilization. On the other hand, Virtual Private Cloud (VPC) can provide network isolation and security, allowing users to create a private network environment in the public cloud and build multiple virtual machines (VMs) on the same physical infrastructure, thereby supporting multi-tenant architectures and allowing users to define and implement sophisticated security policies.

[0003] In a virtual private cloud environment, virtual machines transmit messages between each other through a virtual switch, which forwards messages using software. Software forwarding in a virtual switch can meet the latency requirements of most applications based on the Transmission Control Protocol (TCP). However, RDMA applications are very sensitive to latency, and the latency jitter introduced by software forwarding in a virtual switch can prevent this software forwarding from meeting the latency requirements of RoCE-based messages. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a remote direct memory access method and apparatus for reducing the delay of RoCE messages in a virtual private cloud environment.

[0005] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a remote direct memory access method, the method comprising:

[0007] Receive a first VXLAN message sent by a first virtual machine to a second virtual machine through a virtual extensible local area network (VXLAN) tunnel; the first VXLAN message is generated by performing VXLAN encapsulation on a Remote Direct Memory Access (RoCE) message based on a converged Ethernet;

[0008] Performing VXLAN decapsulation on the first VXLAN message to obtain a first RoCE message through a hardware programmable network card based on a virtual switch, and performing access control on the first RoCE message through the hardware programmable network card, wherein the hardware programmable network card based on the virtual switch is pre-configured based on software of the virtual switch to implement virtual switch software functions through the network card hardware.

[0009] As an optional implementation of the embodiment of the present application, the performing access control on the first RoCE message by the hardware programmable network card includes:

[0010] Obtaining the source network protocol address, destination network protocol address, and destination port of the first RoCE message;

[0011] Obtaining an access control identifier corresponding to the first RoCE message according to the source network protocol address, the destination network protocol address, and the destination port of the first RoCE message;

[0012] Determining the access control policy group corresponding to the first RoCE message according to the access control identifier corresponding to the first RoCE message and the access control identifiers of each access control policy group preconfigured in the hardware programmable network card;

[0013] Determine whether to allow forwarding of the first RoCE message according to the access control policy in the access control policy group corresponding to the first RoCE message.

[0014] As an optional implementation of the embodiment of the present application, the method further includes:

[0015] If it is determined that forwarding of the first RoCE message is allowed, forwarding the first RoCE message to the RoCE network card of the second virtual machine through the hardware programmable network card;

[0016] When it is determined that forwarding of the first RoCE message is prohibited, the first RoCE message is processed based on first preset software.

[0017] As an optional implementation of the embodiment of the present application, the method further includes:

[0018] Determining, by the hardware programmable network card, whether the outer network protocol header of the first RoCE message includes an explicit congestion notification (ECN) mark; and adding the ECN mark to the inner network protocol header of the first RoCE message if the outer network protocol header of the first RoCE message includes the explicit congestion notification (ECN) mark.

[0019] As an optional implementation of the embodiment of the present application, the method further includes:

[0020] The hardware programmable network card is used to measure the bandwidth of the RoCE message sent by the first virtual machine to obtain a first bandwidth, and it is determined whether the first bandwidth is greater than a bandwidth threshold. If so, an ECN mark is added to the inner network protocol header of the first RoCE message.

[0021] As an optional implementation of the embodiment of the present application, the method further includes:

[0022] Receive a second RoCE message output by the RoCE network card of the second virtual machine;

[0023] Access control is performed on the second RoCE message through the hardware programmable network card, and when it is determined that forwarding of the second RoCE message is allowed, VXLAN encapsulation is performed on the second RoCE message through the hardware programmable network card to obtain a second VXLAN message, and the second VXLAN message is sent through the VXLAN tunnel.

[0024] As an optional implementation of the embodiment of the present application, the access control on the second RoCE message by the hardware programmable network card includes:

[0025] Obtaining the source network protocol address, destination network protocol address, and destination port of the second RoCE message;

[0026] Obtaining an access control identifier corresponding to the second RoCE message according to the source network protocol address, the destination network protocol address, and the destination port of the second RoCE message;

[0027] Determining the access control policy group corresponding to the second RoCE message according to the access control identifier corresponding to the second RoCE message and the access control identifiers of each access control policy group preconfigured in the hardware programmable network card;

[0028] Determine whether to allow forwarding of the second RoCE message according to the access control policy in the access control policy group corresponding to the second RoCE message.

[0029] As an optional implementation of the embodiment of the present application, the method further includes:

[0030] When it is determined that forwarding of the second RoCE message is prohibited, the second RoCE message is processed based on second preset software.

[0031] As an optional implementation of the embodiment of the present application, the method further includes:

[0032] determining, by the hardware programmable network card, whether the second RoCE message is a congestion notification packet (CNP) message; and setting, if the second RoCE message is the CNP message, a value of a differentiated services code (DSCP) in an outer network protocol header of the second RoCE message to a first preset value; and setting, if the second RoCE message is not the CNP message, a value of the DSCP in the outer network protocol header of the second RoCE message to a second preset value.

[0033] As an optional implementation of the embodiment of the present application, sending the second VXLAN message through the VXLAN tunnel includes:

[0034] The second VXLAN message is written into a message queue, and the VXLAN messages in the message queue are sent sequentially at a rate within the second bandwidth.

[0035] In a second aspect, an embodiment of the present application provides a remote direct memory access device, comprising:

[0036] A receiving unit, configured to receive a first VXLAN message sent by a first virtual machine to a second virtual machine through a virtual extensible local area network (VXLAN) tunnel; the first VXLAN message is generated by performing VXLAN encapsulation on a Remote Direct Memory Access (RoCE) message based on Converged Ethernet;

[0037] a processing unit, configured to perform VXLAN decapsulation on the first VXLAN message to obtain a first RoCE message through a hardware programmable network card based on a virtual switch, and perform access control on the first RoCE message through the hardware programmable network card, wherein the hardware programmable network card based on the virtual switch is preconfigured based on software of the virtual switch to implement virtual switch software functions through the network card hardware.

[0038] As an optional implementation manner of the embodiment of the present application, the processing unit is specifically configured to obtain, through the hardware programmable network card, a source network protocol address, a destination network protocol address, and a destination port of the first RoCE message; obtain, based on the source network protocol address, the destination network protocol address, and the destination port of the first RoCE message, an access control identifier corresponding to the first RoCE message; determine, based on the access control identifier corresponding to the first RoCE message and the access control identifiers of each access control policy group preconfigured in the hardware programmable network card, an access control policy group corresponding to the first RoCE message; and determine, based on the access control policy in the access control policy group corresponding to the first RoCE message, whether to allow forwarding of the first RoCE message.

[0039] As an optional implementation manner of the embodiment of the present application, the processing unit is further configured to, when it is determined that forwarding of the first RoCE message is allowed, forward the first RoCE message to the RoCE network card of the second virtual machine through the hardware programmable network card; and when it is determined that forwarding of the first RoCE message is prohibited, process the first RoCE message based on first preset software.

[0040] As an optional implementation manner of the embodiment of the present application, the processing unit is further configured to determine, through the hardware programmable network card, whether the outer network protocol header of the first RoCE message includes an explicit congestion notification (ECN) mark; and if the outer network protocol header of the first RoCE message includes the explicit congestion notification (ECN) mark, add the ECN mark to the inner network protocol header of the first RoCE message.

[0041] As an optional implementation of the embodiment of the present application, the processing unit is further configured to perform bandwidth measurement on the traffic of the RoCE message sent by the first virtual machine through the hardware programmable network card to obtain a first bandwidth, determine whether the first bandwidth is greater than a bandwidth threshold, and if so, add an ECN mark to the inner network protocol header of the first RoCE message.

[0042] As an optional implementation manner of the embodiment of the present application, the receiving unit is further configured to receive a second RoCE message output by the RoCE network card of the second virtual machine;

[0043] The processing unit is further configured to perform access control on the second RoCE message through the hardware programmable network card, and when it is determined that forwarding of the second RoCE message is allowed, perform VXLAN encapsulation on the second RoCE message through the hardware programmable network card to obtain a second VXLAN message, and send the second VXLAN message through the VXLAN tunnel.

[0044] As an optional implementation manner of the embodiment of the present application, the processing unit is further configured to obtain, through the hardware programmable network card, a source network protocol address, a destination network protocol address, and a destination port of the second RoCE message; obtain, based on the source network protocol address, the destination network protocol address, and the destination port of the second RoCE message, an access control identifier corresponding to the second RoCE message; determine, based on the access control identifier corresponding to the second RoCE message and the access control identifiers of each access control policy group preconfigured in the hardware programmable network card, an access control policy group corresponding to the second RoCE message; and determine, based on the access control policy in the access control policy group corresponding to the second RoCE message, whether to allow forwarding of the second RoCE message.

[0045] As an optional implementation of the embodiment of the present application, the processing unit is further configured to, when it is determined that forwarding of the second RoCE message is prohibited, process the second RoCE message based on second preset software.

[0046] As an optional implementation manner of the embodiment of the present application, the processing unit is further configured to determine, through the hardware programmable network card, whether the second RoCE message is a congestion notification packet (CNP) message; if the second RoCE message is the CNP message, set a value of a Differentiated Services Code (DSCP) in an outer network protocol header of the second RoCE message to a first preset value; and if the second RoCE message is not the CNP message, set the value of the DSCP in the outer network protocol header of the second RoCE message to a second preset value.

[0047] As an optional implementation of the embodiment of the present application, the processing unit is specifically used to write the second VXLAN message into the message queue, and send the VXLAN messages in the message queue in sequence at a rate within the second bandwidth.

[0048] In a third aspect, an embodiment of the present application provides a hardware device comprising: a memory, a processor, and a hardware programmable network card, wherein the memory is used to store a computer program, and the processor is used to enable the hardware device to implement the remote direct memory access method described in any of the above embodiments when executing the computer program.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which, when the computer program is executed by a computing device, enables the computing device to implement the remote direct memory access method described in any of the above embodiments.

[0050] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to implement the remote direct memory access method described in any one of the above embodiments.

[0051] The remote direct memory access method provided in the embodiment of the present application, upon receiving a first VXLAN message sent by a first virtual machine through a VXLAN tunnel, performs VXLAN decapsulation on the first VXLAN message through a hardware programmable network card based on a virtual switch to obtain a first RoCE message, and performs access control on the first RoCE message through the hardware programmable network card. Since the hardware programmable network card based on the virtual switch is pre-configured based on the software of the virtual switch to implement the virtual switch software function through the network card hardware, and the remote direct memory access method provided in the embodiment of the present application can perform VXLAN decapsulation on the first VXLAN message through the hardware programmable network card to obtain a first RoCE message, and perform access control on the first RoCE message through the hardware programmable network card, compared to software forwarding based on a virtual switch in the related art, the remote direct memory access method provided in the embodiment of the present application can process the received RoCE message through the hardware programmable network card. Therefore, the embodiment of the present application can reduce the delay of RoCE messages in a virtual private cloud environment, thereby realizing the deployment of RoCEv2 in a virtual private cloud environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A schematic diagram of the hardware device provided in the embodiment of the present application;

[0055] Figure 2 One of the flowcharts of the remote direct memory access method provided in an embodiment of the present application;

[0056] Figure 3 A schematic diagram of the structure of a RoCE message in a non-virtualized scenario provided in an embodiment of the present application;

[0057] Figure 4 A schematic diagram of the structure of a RoCE message in a virtualized scenario provided in an embodiment of the present application;

[0058] Figure 5 A schematic diagram of the process flow of a hardware programmable network card for processing received RoCE messages provided in an embodiment of the present application;

[0059] Figure 6 Flowchart 2 of the remote direct memory access method provided in an embodiment of the present application;

[0060] Figure 7 A schematic diagram of the process flow of the hardware programmable network card provided in an embodiment of the present application for processing RoCE messages that need to be sent;

[0061] Figure 8 A schematic diagram of the structure of a remote direct memory access device provided in an embodiment of the present application;

[0062] Figure 9 A schematic diagram of the hardware structure of the hardware device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0064] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein. Obviously, the embodiments in the specification are only part of the embodiments of the present application, not all of the embodiments.

[0065] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete way. In addition, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more.

[0066] The following first describes the hardware device used to execute the remote direct memory access method provided in the embodiment of the present application.

[0067] Reference Figure 1As shown, a hardware device 100 for executing the remote direct memory access method provided in an embodiment of the present application includes: at least one virtual machine (virtual machine 1, virtual machine 2, ..., virtual machine n) 11 created based on a virtualization function and a network interface card (NIC) 12. The NIC 12 includes: RoCE NICs (RoCE NIC 1, RoCE NIC 2, ..., RoCE NIC n) 121 virtualized into respective virtual machines and a hardware programmable NIC 122 based on a virtual switch.

[0068] In some embodiments, the network card 12 can be virtualized into a RoCE network card for each virtual machine through single root I / O virtualization (SR-IOV) technology.

[0069] SR-IOV is a hardware-assisted virtualization technology that allows a single network interface card (NIC) to be virtualized into multiple independent virtual functions (VFs). Each VF can be independently assigned to a different virtual machine (VM) or container. Because SR-IOV allows each virtual instance to directly access hardware resources without the need for host intervention, it can significantly improve the performance of virtualized environments.

[0070] SR-IOV enables the device (VERBSDEV) used to perform RDMA operations in each virtual machine in the virtualized environment to have a globally unique identifier (GUID) bound to the VPC IP and independent RoCE processing capabilities.

[0071] In some embodiments, a hardware programmable network card 122 based on a virtual switch refers to a hardware programmable network card 122 configured based on the software of the virtual switch so that the hardware programmable network card 122 can implement the same or similar functions as the virtual switch. In other words, the hardware programmable network card 122 based on the virtual switch is pre-configured based on the software of the virtual switch to implement the virtual switch software functions through the network card hardware.

[0072] In a virtual private cloud (VPC) environment, VPC traffic is handled using VPC private IP addresses. RoCE devices are interconnected via Virtual Extensible Local Area Network (VXLAN) tunnels. Different virtual network identifiers (VNIs) isolate different tenants. VXLAN tunnels remain transparent to RoCE users, who are unaware of the transport layer implementation. VPC traffic is typically handled by a virtual switch, which primarily provides the following functions:

[0073] 1. VPC security policy, which executes security group configuration and determines whether the current host can communicate with the remote host via RoCE.

[0074] 2. VPC mapping: Map the VPC destination VM IP address to the underlying IP address of the host where the destination VM is located.

[0075] 3. VXLAN encapsulation / decapsulation: encapsulate the VPC message into a VXLAN message and send it. In reverse, decapsulate the VXLAN message to obtain the VPC message.

[0076] Therefore, configuring the hardware programmable network card 122 based on the software of the virtual switch can at least enable the real hardware programmable network card to implement the functions of executing VPC security policies, VPC mapping, and VXLAN encapsulation / decapsulation.

[0077] Based on the above content, the embodiment of the present application also provides a remote direct memory access method, which is applied to Figure 1 The hardware devices shown are Figure 2 As shown, the remote direct memory access method includes the following steps:

[0078] S21. Receive a first VXLAN message sent by a first virtual machine to a second virtual machine among the at least one virtual machine through a VXLAN tunnel.

[0079] The first VXLAN message is generated by performing VXLAN encapsulation on the RoCE message.

[0080] That is, the first VXLAN message sent by the first virtual machine is received through the VXLAN tunnel between the second virtual machine and the first virtual machine.

[0081] S22. Perform VXLAN decapsulation on the first VXLAN message through a hardware programmable network card based on a virtual switch to obtain a first RoCE message.

[0082] The hardware programmable network card based on the virtual switch is pre-configured based on the software of the virtual switch to implement the virtual switch software function through the network card hardware.

[0083] That is, after receiving the first VXLAN message, the VXLAN message is first decapsulated by the hardware programmable network card based on the virtual switch to obtain the original VPC message before VXLAN encapsulation. The first VXLAN message is a VXLAN message obtained by VXLAN encapsulating the RoCE message. Therefore, the first RoCE message can be obtained by decapsulating the first VXLAN message by the hardware programmable network card.

[0084] S23. Perform access control on the first RoCE message through the hardware programmable network card.

[0085] In some embodiments, performing access control on the first RoCE message by the hardware programmable network card includes the following steps a to d:

[0086] Step a: Obtain the source network protocol address (Source Network Protocol Address), destination network protocol address (Destination Network Protocol Address), and destination port (Destination Port) of the first RoCE message.

[0087] In this embodiment of the present application, the source network protocol address of the first RoCE message refers to the network protocol address used by the device sending the first RoCE message; that is, the IP address of the first virtual machine. The destination network protocol address of the first RoCE message refers to the network protocol address of the device receiving the first RoCE message, that is, the IP address of the virtual machine constructed on the hardware device for receiving the first RoCE message. The destination port of the first RoCE message refers to the address of the port used to receive the first RoCE message on the device receiving the first RoCE message.

[0088] Step b: Obtain an access control identifier corresponding to the first RoCE message according to the source network protocol address, destination network protocol address, and destination port of the first RoCE message.

[0089] That is, the access control identifier key1 corresponding to the first RoCE message is: key1={source network protocol address of the first RoCE message, destination network protocol address of the first RoCE message, destination port of the first RoCE message}.

[0090] It should be noted that since the source port (Source Port) is generally used as part of the equal-cost multipath routing (ECMP) hash entropy and a random value is obtained through a specific calculation method, the access control identifier corresponding to the first RoCE message is not obtained according to the source port of the first RoCE message in the embodiment of the present application.

[0091] Step c: Determine the access control policy group corresponding to the first RoCE message according to the access control identifier corresponding to the first RoCE message and the access control identifiers of each access control policy group pre-configured in the hardware programmable network card.

[0092] In some embodiments, the access control identifier corresponding to the first RoCE message can be matched with the access control identifier of the access control policy group preconfigured in the hardware programmable network card, and the access control policy group that matches the access control identifier and the access control identifier corresponding to the first RoCE message can be determined as the access control policy group corresponding to the first RoCE message.

[0093] Step d: Determine whether to allow forwarding of the first RoCE message according to the access control policy in the access control policy group corresponding to the first RoCE message.

[0094] Based on the above embodiments, users can configure different access control policy groups for communications between different virtual machines according to their own security requirements and business needs.

[0095] In some embodiments, after executing the above step d (determining whether to allow forwarding of the first RoCE message according to the access control policy in the access control policy group corresponding to the first RoCE message), the remote direct memory access method provided in the embodiment of the present application further includes: if it is determined that forwarding of the first RoCE message is allowed, forwarding the first RoCE message to the RoCE network card of the second virtual machine through the hardware programmable network card.

[0096] The second virtual machine is a virtual machine whose IP address is the destination IP address of the first RoCE message.

[0097] In some embodiments, after executing step d above (determining whether forwarding of the first RoCE message is allowed according to the access control policy in the access control policy group corresponding to the first RoCE message), the remote direct memory access method provided in the embodiment of the present application further includes: if it is determined that forwarding of the first RoCE message is prohibited, processing the first RoCE message based on first preset software.

[0098] In some embodiments, processing the first RoCE message based on the first preset software includes: generating a security log corresponding to the first RoCE message, and saving the security log corresponding to the first RoCE message to a designated storage location.

[0099] The remote direct memory access method provided in the embodiment of the present application, upon receiving a first VXLAN message sent by a first virtual machine through a VXLAN tunnel, performs VXLAN decapsulation on the first VXLAN message through a hardware programmable network card based on a virtual switch to obtain a first RoCE message, and performs access control on the first RoCE message through the hardware programmable network card. Since the hardware programmable network card based on the virtual switch is pre-configured based on the software of the virtual switch to implement the virtual switch software function through the network card hardware, and the remote direct memory access method provided in the embodiment of the present application can perform VXLAN decapsulation on the first VXLAN message through the hardware programmable network card to obtain a first RoCE message, and perform access control on the first RoCE message through the hardware programmable network card, compared to software forwarding based on a virtual switch in the related art, the remote direct memory access method provided in the embodiment of the present application can process the received RoCE message through the hardware programmable network card. Therefore, the embodiment of the present application can reduce the delay of RoCE messages in a virtual private cloud environment, thereby realizing the deployment of RoCEv2 in a virtual private cloud environment.

[0100] In some embodiments, the remote direct memory access method provided by the embodiments of the present application further includes: determining, by the hardware programmable network card, whether the outer network protocol header of the first RoCE message includes an ECN mark; and if the outer network protocol header of the first RoCE message includes an explicit congestion notification ECN mark, adding the ECN mark to the inner network protocol header of the first RoCE message.

[0101] Reference Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of the RoCE message in a non-virtual private cloud environment. Figure 3As shown, a RoCE message in a non-virtual private cloud environment includes an Ethernet header 31, an IP header 32, a UDP header 33, an InfiniBand Base Transport Header (BTH) 34, and a message payload 35. The Ethernet header 31 includes the source and destination MAC addresses; the IP header 32 includes an Explicit Congestion Notification (ECN) flag 321, a source IP address 322, and a destination IP address 323; and the UDP header 33 includes a source port 331 and a destination port 332. In some embodiments, the destination port of a RoCE message is a fixed value of 4791.

[0102] Reference Figure 4 As shown, Figure 4 Figure 1 is a schematic diagram of the structure of RoCE messages in a virtual private cloud environment. Figure 4 As shown, the RoCE message in the virtual private cloud environment includes: an outer Ethernet header 41, an outer network protocol header 42, an outer UDP protocol header 43, a virtual network identifier (VXLAN Network Identifier, VNI) 44, an inner Ethernet header 45, an inner network protocol header 46, an inner UDP protocol header 47, BTH 48 and a message payload 49. The outer network protocol header 42 includes: Differentiated Services Code Point (DSCP) and ECN field 421, source IP address 422 and destination IP address 423, where source IP address 422 is the physical address of the host machine of the source virtual machine, and destination IP address 423 is the physical address of the host machine of the destination virtual machine. The UDP protocol header 43 includes: source port 431 and destination port 432, where source port 431 is the physical port to which the source port belongs, and destination port 432 is the physical port to which the port belongs. The inner network protocol header 46 includes: Differentiated Services Code Point (DSCP) and ECN field, source IP address and destination IP address. The inner UDP protocol header 47 includes: source port and destination port.

[0103] When the physical switch in the physical network forwards and processes RoCE messages in the virtual private cloud environment, it forwards and processes them based on the outer Ethernet header 41, the outer network protocol header 42, and the outer UDP protocol header 43. Therefore, when the traffic of the RoCE message exceeds the preset threshold, the physical switch will set the value of the ECN field in the outer network protocol header 42 to a preset value (for example: 1). The hardware programmable network card forwards and processes messages based on the VNI 44, the inner Ethernet header 45, the inner network protocol header 46, and the inner UDP protocol header 47. The virtual machine will only determine whether congestion occurs and perform subsequent processing based on the value of the ECN field in the inner network protocol header 46. Therefore, when the hardware programmable network card determines that the outer network protocol header of the first RoCE message includes an ECN mark, it is necessary to add the ECN mark in the outer network protocol header to the inner network protocol header of the first RoCE message to prevent the virtual machine from being unable to properly implement the congestion detection function.

[0104] In some embodiments, adding the ECN mark to the inner network protocol header of the first RoCE message includes:

[0105] The value of the ECN field in the inner network protocol header of the first RoCE message is set to a preset value.

[0106] In some embodiments, the preset value is 1.

[0107] In some embodiments, the remote direct memory access method provided by the embodiments of the present application further includes:

[0108] The hardware programmable network card is used to measure the bandwidth of the RoCE message sent by the first virtual machine to obtain a first bandwidth, and it is determined whether the first bandwidth is greater than a bandwidth threshold. If so, an ECN mark is added to the inner network protocol header of the first RoCE message.

[0109] That is, if it is detected that the bandwidth occupied by the traffic of the RoCE message sent by the first virtual machine is greater than the preset bandwidth, Figure 4 An ECN mark is added to the inner network protocol header 46 in the message structure shown.

[0110] Similarly, adding the ECN mark in the inner network protocol header of the first RoCE message may include setting a value of an ECN field in the inner network protocol header of the first RoCE message to a preset value.

[0111] Reference Figure 5 As shown, in combination with the above embodiment, the processing of the first VXLAN message by the hardware programmable network card includes:

[0112] S51: VXLAN decapsulation. That is, perform VXLAN decapsulation on the first VXLAN message to obtain a first RoCE message.

[0113] S52, ECN mapping: That is, if the outer network protocol header includes an ECN tag, the ECN tag is copied to the inner network protocol header.

[0114] S53: Access control. That is, obtaining an access control identifier corresponding to the first RoCE message based on the source network protocol address, destination network protocol address, and destination port of the first RoCE message, determining an access control policy group corresponding to the first RoCE message based on the access control identifier corresponding to the first RoCE message, and determining whether to allow forwarding of the first RoCE message based on the access control policy in the access control policy group corresponding to the first RoCE message.

[0115] S54: Speed limiting. That is, obtaining the bandwidth occupied by the traffic of the RoCE message sent by the first virtual machine, and adding an ECN mark to the inner network protocol header of the first RoCE message when the first bandwidth is greater than a bandwidth threshold.

[0116] It should be noted that the order in which steps S51 to S54 are executed is not limited in the embodiments of the present application. The hardware programmable network card can execute steps S51 to S54 on the first RoCE message in any order. For example, step S53 can be executed first to perform access control on the first RoCE message, and then step S54 can be executed to limit the rate of the first RoCE message. Alternatively, step S55 can be executed first to limit the rate of the first RoCE message, and then step S53 can be executed to perform access control on the first RoCE message.

[0117] Reference Figure 6 As shown, the remote direct memory access method provided in the embodiment of the present application further includes the following steps:

[0118] S61. Receive a second RoCE message output by the RoCE network card of the second virtual machine.

[0119] S62: Perform access control on the second RoCE message through the hardware programmable network card.

[0120] In some embodiments, performing access control on the second RoCE message by the hardware programmable network card includes the following steps 1 to 4:

[0121] Step 1: Obtain the source network protocol address, destination network protocol address, and destination port of the second RoCE message.

[0122] The difference from the above step a is that the source network protocol address and the destination network protocol address of the first RoCE message obtained in step a are the first virtual network protocol address and the second virtual network protocol address, respectively, and the source network protocol address and the destination network protocol address of the second RoCE message obtained in the above step 1 are the second virtual network protocol address and the first virtual network protocol address, respectively.

[0123] Step 2: Obtain an access control identifier corresponding to the second RoCE message according to the source network protocol address, destination network protocol address, and destination port of the second RoCE message.

[0124] That is, the access control identifier key2 corresponding to the second RoCE message is: key2={source network protocol address of the second RoCE message, destination network protocol address of the second RoCE message, destination port of the second RoCE message}.

[0125] Step 3: Determine the access control policy group corresponding to the second RoCE message according to the access control identifier corresponding to the second RoCE message and the access control identifiers of each access control policy group pre-configured in the hardware programmable network card;

[0126] Step 4: Determine whether to allow forwarding of the second RoCE message according to the access control policy in the access control policy group corresponding to the second RoCE message.

[0127] In the above step S62, if it is determined that the second RoCE message is allowed to be forwarded, the following step S63 is executed:

[0128] S63. Perform VXLAN encapsulation on the second RoCE message through the hardware programmable network card to obtain a second VXLAN message, and send the second VXLAN message through the VXLAN tunnel.

[0129] In some embodiments, sending the second VXLAN message through the VXLAN tunnel includes:

[0130] The second VXLAN message is written into a message queue, and the VXLAN messages in the message queue are sent sequentially at a rate within the second bandwidth.

[0131] In a multi-tenant environment, the RDMA bandwidths of multiple tenants need to be isolated, so the above embodiment sends the VXLAN messages in the message queue sequentially at a rate within the second bandwidth, where the second bandwidth is the maximum bandwidth available to the current tenant.

[0132] In addition, the second VXLAN message is first written into the message queue, and then the VXLAN messages in the message queue are sent in sequence, which can ensure that the RoCE message will not be lost due to speed limit.

[0133] In some embodiments, if in step S62 , it is determined that forwarding of the second RoCE message is prohibited when performing access control on the second RoCE message through the hardware programmable network card, the second RoCE message is processed based on second preset software.

[0134] Exemplarily, processing the second RoCE message based on the second preset software includes: generating a security log corresponding to the second RoCE message, and writing the security log corresponding to the second RoCE message to a designated storage location.

[0135] In some embodiments, the remote direct memory access method provided by the embodiments of the present application further includes:

[0136] determining, by the hardware programmable network card, whether the second RoCE message is a congestion notification packet (CNP) message; and setting, if the second RoCE message is the CNP message, a DSCP value in an outer network protocol header of the second RoCE message to a first preset value; and setting, if the second RoCE message is not the CNP message, the DSCP value in the outer network protocol header of the second RoCE message to a second preset value.

[0137] Specifically, a CNP message is a control packet in RoCEv2 that notifies the sender of network congestion. When a receiver receives a packet marked with ECN, it sends a CNP message to the sender, instructing it to proactively reduce the sending rate based on the congestion algorithm to alleviate the current congestion. The use of the ECN / CNP control loop prevents RoCE message loss and the resulting bandwidth drop.

[0138] Since the hardware programmable network card sets the DSCP value in the outer network protocol header of the second RoCE message to the first preset value when the second RoCE message is the CNP message, and sets the DSCP value in the outer network protocol header of the second RoCE message to the second preset value when the second RoCE message is not the CNP message, the above embodiment can distinguish between ordinary RoCE messages and CNP messages by the DSCP value in the outer network protocol header, thereby ensuring that CNP messages are processed preferentially.

[0139] Reference Figure 7As shown, in combination with the above embodiment, the processing of the second RoCE message by the hardware programmable network card includes:

[0140] S71: Rate limiting. That is, writing the second VXLAN message into a message queue, and sequentially sending the VXLAN messages in the message queue at a rate within the second bandwidth.

[0141] S72: Access control. That is, obtaining an access control identifier corresponding to the second RoCE message based on the source network protocol address, destination network protocol address, and destination port of the second RoCE message, determining an access control policy group corresponding to the second RoCE message based on the access control identifier corresponding to the second RoCE message, and determining whether to allow forwarding of the second RoCE message based on the access control policy in the access control policy group corresponding to the second RoCE message.

[0142] S73, DSCP mapping. That is, if the second RoCE message is a common RoCE message, the DSCP value in the outer network protocol header is set to the first preset value; if the second RoCE message is a CNP message, the DSCP value in the outer network protocol header is set to the second preset value.

[0143] S74: VXLAN encapsulation. That is, perform VXLAN encapsulation on the second RoCE message to obtain a second VXLAN message.

[0144] Similarly, the execution order of the above steps S71 to S74 is not limited in the embodiment of the present application. The hardware programmable network card can execute the above steps S71 to S74 on the second RoCE message in any order.

[0145] Based on the same inventive concept, as an implementation of the above method, an embodiment of the present application further provides a remote direct memory access device, which corresponds to the above method embodiment. For ease of reading, this embodiment will no longer describe the details of the above method embodiment one by one, but it should be clear that the remote direct memory access device in this embodiment can implement all the contents of the above method embodiment.

[0146] The embodiment of the present application provides a remote direct memory access device, Figure 8 is a structural diagram of the remote direct memory access device, such as Figure 8 As shown, the remote direct memory access device 800 includes:

[0147] A receiving unit 81 is configured to receive a first VXLAN message sent by a first virtual machine to a second virtual machine through a virtual extensible local area network (VXLAN) tunnel; the first VXLAN message is generated by performing VXLAN encapsulation on a Remote Direct Memory Access (RoCE) message based on Converged Ethernet;

[0148] A processing unit 82 is configured to perform VXLAN decapsulation on the first VXLAN message to obtain a first RoCE message through a hardware programmable network card based on a virtual switch, and perform access control on the first RoCE message through the hardware programmable network card, wherein the hardware programmable network card based on the virtual switch is preconfigured based on the software of the virtual switch to implement the virtual switch software function through the network card hardware.

[0149] As an optional implementation manner of the embodiment of the present application, the processing unit 82 is specifically configured to obtain, through the hardware programmable network card, a source network protocol address, a destination network protocol address, and a destination port of the first RoCE message; obtain an access control identifier corresponding to the first RoCE message based on the source network protocol address, the destination network protocol address, and the destination port of the first RoCE message; determine, based on the access control identifier corresponding to the first RoCE message and the access control identifiers of each access control policy group preconfigured in the hardware programmable network card, an access control policy group corresponding to the first RoCE message; and determine, based on the access control policy in the access control policy group corresponding to the first RoCE message, whether to allow forwarding of the first RoCE message.

[0150] As an optional implementation of the embodiment of the present application, the processing unit 82 is further configured to, when it is determined that forwarding of the first RoCE message is allowed, forward the first RoCE message to the RoCE network card of the second virtual machine through the hardware programmable network card; and when it is determined that forwarding of the first RoCE message is prohibited, process the first RoCE message based on first preset software.

[0151] As an optional implementation of the embodiment of the present application, the processing unit 82 is further configured to determine, through the hardware programmable network card, whether the outer network protocol header of the first RoCE message includes an explicit congestion notification (ECN) tag; and if the outer network protocol header of the first RoCE message includes the explicit congestion notification (ECN) tag, add the ECN tag to the inner network protocol header of the first RoCE message.

[0152] As an optional implementation of the embodiment of the present application, the processing unit 82 is further configured to perform bandwidth measurement on the traffic of the RoCE message sent by the first virtual machine through the hardware programmable network card to obtain a first bandwidth, determine whether the first bandwidth is greater than a bandwidth threshold, and if so, add an ECN mark to the inner network protocol header of the first RoCE message.

[0153] As an optional implementation of the embodiment of the present application, the receiving unit 81 is further configured to receive a second RoCE message output by the RoCE network card of the second virtual machine;

[0154] The processing unit 82 is further configured to perform access control on the second RoCE message through the hardware programmable network card, and when it is determined that forwarding of the second RoCE message is allowed, perform VXLAN encapsulation on the second RoCE message through the hardware programmable network card to obtain a second VXLAN message, and send the second VXLAN message through the VXLAN tunnel.

[0155] As an optional implementation manner of the embodiment of the present application, the processing unit 82 is further configured to obtain, through the hardware programmable network card, a source network protocol address, a destination network protocol address, and a destination port of the second RoCE message; obtain an access control identifier corresponding to the second RoCE message based on the source network protocol address, the destination network protocol address, and the destination port of the second RoCE message; determine, based on the access control identifier corresponding to the second RoCE message and the access control identifiers of each access control policy group preconfigured in the hardware programmable network card, an access control policy group corresponding to the second RoCE message; and determine, based on the access control policy in the access control policy group corresponding to the second RoCE message, whether to allow forwarding of the second RoCE message.

[0156] As an optional implementation of the embodiment of the present application, the processing unit 82 is further configured to process the second RoCE message based on second preset software when it is determined that forwarding of the second RoCE message is prohibited.

[0157] As an optional implementation manner of the embodiment of the present application, the processing unit 82 is further configured to determine, through the hardware programmable network card, whether the second RoCE message is a congestion notification packet (CNP) message; if the second RoCE message is the CNP message, set a value of a Differentiated Services Code (DSCP) in an outer network protocol header of the second RoCE message to a first preset value; and if the second RoCE message is not the CNP message, set the value of the DSCP in the outer network protocol header of the second RoCE message to a second preset value.

[0158] As an optional implementation of the embodiment of the present application, the processing unit 82 is specifically configured to write the second VXLAN message into a message queue, and send the VXLAN messages in the message queue sequentially at a rate within the second bandwidth.

[0159] The hardware device provided in the embodiments of the present application can execute the remote direct memory access method provided in any of the above embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0160] Based on the same inventive concept, an embodiment of the present application also provides a hardware device. Figure 9 A schematic diagram of the structure of the hardware device provided in the embodiment of the present application is shown in FIG. Figure 9 As shown, the hardware device provided in this embodiment includes: a memory 901, a processor 902 and a hardware programmable network card 903, wherein the memory 901 is used to store computer programs, and the processor 902 and the hardware programmable network card 903 are used to execute the remote direct memory access method provided in the above embodiment when executing the computer program.

[0161] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the computing device implements the remote direct memory access method provided in the above embodiment.

[0162] Based on the same inventive concept, an embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computing device implements the remote direct memory access method provided in the above embodiment.

[0163] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0164] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), dedicated injection circuits,

[0165] (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0166] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0167] Computer-readable media includes both permanent and non-permanent, removable and non-removable storage media. Storage media can implement any method or technology for storing information, which can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A remote direct memory access method, characterized in that: The method comprises: Receive a first VXLAN message sent by a first virtual machine to a second virtual machine through a virtual extensible local area network (VXLAN) tunnel; the first VXLAN message is generated by performing VXLAN encapsulation on a Remote Direct Memory Access (RoCE) message based on a converged Ethernet; performing VXLAN decapsulation on the first VXLAN message to obtain a first RoCE message through a hardware programmable network card based on a virtual switch, and performing access control on the first RoCE message through the hardware programmable network card, wherein the hardware programmable network card based on the virtual switch is preconfigured based on software of the virtual switch to implement virtual switch software functions through the network card hardware; The method further comprises: The hardware programmable network card is used to measure the bandwidth of the RoCE message sent by the first virtual machine to obtain a first bandwidth, and it is determined whether the first bandwidth is greater than a bandwidth threshold. If so, an ECN mark is added to the inner network protocol header of the first RoCE message.

2. The method according to claim 1, characterized in that The performing access control on the first RoCE message by using the hardware programmable network card includes: Obtaining the source network protocol address, destination network protocol address, and destination port of the first RoCE message; Obtaining an access control identifier corresponding to the first RoCE message according to the source network protocol address, the destination network protocol address, and the destination port of the first RoCE message; Determining the access control policy group corresponding to the first RoCE message according to the access control identifier corresponding to the first RoCE message and the access control identifiers of each access control policy group preconfigured in the hardware programmable network card; Determine whether to allow forwarding of the first RoCE message according to the access control policy in the access control policy group corresponding to the first RoCE message.

3. The method according to claim 2, characterized in that The method further comprises: If it is determined that forwarding of the first RoCE message is allowed, forwarding the first RoCE message to the RoCE network card of the second virtual machine through the hardware programmable network card; When it is determined that forwarding of the first RoCE message is prohibited, the first RoCE message is processed based on first preset software.

4. The method according to claim 1, wherein The method further comprises: Determining, by the hardware programmable network card, whether the outer network protocol header of the first RoCE message includes an explicit congestion notification (ECN) mark; and adding the ECN mark to the inner network protocol header of the first RoCE message if the outer network protocol header of the first RoCE message includes the explicit congestion notification (ECN) mark.

5. The method according to claim 1, wherein The method further comprises: Receive a second RoCE message output by the RoCE network card of the second virtual machine; Access control is performed on the second RoCE message through the hardware programmable network card, and when it is determined that forwarding of the second RoCE message is allowed, VXLAN encapsulation is performed on the second RoCE message through the hardware programmable network card to obtain a second VXLAN message, and the second VXLAN message is sent through the VXLAN tunnel.

6. The method according to claim 5, characterized in that The performing access control on the second RoCE message by the hardware programmable network card includes: Obtaining the source network protocol address, destination network protocol address, and destination port of the second RoCE message; Obtaining an access control identifier corresponding to the second RoCE message according to the source network protocol address, the destination network protocol address, and the destination port of the second RoCE message; Determining the access control policy group corresponding to the second RoCE message according to the access control identifier corresponding to the second RoCE message and the access control identifiers of each access control policy group preconfigured in the hardware programmable network card; Determine whether to allow forwarding of the second RoCE message according to the access control policy in the access control policy group corresponding to the second RoCE message.

7. The method according to claim 6, characterized in that The method further comprises: When it is determined that forwarding of the second RoCE message is prohibited, the second RoCE message is processed based on second preset software.

8. The method according to claim 5, characterized in that The method further comprises: determining, by the hardware programmable network card, whether the second RoCE message is a congestion notification packet (CNP) message; and setting, if the second RoCE message is the CNP message, a value of a differentiated services code (DSCP) in an outer network protocol header of the second RoCE message to a first preset value; and setting, if the second RoCE message is not the CNP message, a value of the DSCP in the outer network protocol header of the second RoCE message to a second preset value.

9. The method according to claim 5, characterized in that The sending the second VXLAN message through the VXLAN tunnel includes: The second VXLAN message is written into a message queue, and the VXLAN messages in the message queue are sent sequentially at a rate within the second bandwidth.

10. A remote direct memory access device, characterized in that: include: A receiving unit, configured to receive a first VXLAN message sent by the first virtual machine to the second virtual machine through a virtual extensible local area network (VXLAN) tunnel; The first VXLAN message is generated by performing VXLAN encapsulation on a Remote Direct Memory Access (RoCE) message based on a converged Ethernet; a processing unit, configured to perform VXLAN decapsulation on the first VXLAN message to obtain a first RoCE message through a hardware programmable network card based on a virtual switch, and perform access control on the first RoCE message through the hardware programmable network card, wherein the hardware programmable network card based on the virtual switch is preconfigured based on software of the virtual switch to implement virtual switch software functions through the network card hardware; The processing unit is further configured to perform bandwidth measurement on the traffic of the RoCE message sent by the first virtual machine through the hardware programmable network card to obtain a first bandwidth, determine whether the first bandwidth is greater than a bandwidth threshold, and if so, add an ECN mark to an inner network protocol header of the first RoCE message.

11. A hardware device, characterized in that: include: A memory, a processor and a hardware programmable network card, wherein the memory is used to store a computer program, and the processor and the hardware programmable network card are used to enable the hardware device to implement the remote direct memory access method according to any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the remote direct memory access method according to any one of claims 1 to 9.

13. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to implement the remote direct memory access method according to any one of claims 1 to 9.

Images