InfiniBand network card virtualization networking system

Abstract

The present invention discloses an InfiniBand network card virtualization networking system, comprising: a driver software module located in a host InfiniBand network card driver, configured to forward configuration information of a virtual network card from an upper-layer interface to a network card hardware function module, and further configured to receive and parse InfiniBand virtualization networking messages and generate corresponding response messages; a network card hardware function module disposed in the InfiniBand network card, configured to correctly forward messages received and sent by each virtual network card according to the virtual network card configuration information issued by the driver software module; and a virtual QP mapping relationship module configured to implement a one-to-one mapping relationship between a logical QP and a virtual QP number, thereby ensuring the correctness and efficiency of the virtualization networking function. The InfiniBand network card virtualization networking system of the present invention can reduce the network card hardware resource overhead used for virtualization functions, shorten the debugging and development cycle, and thus reduce the cost of the network card.

Description

Technical Field

[0001] The present invention belongs to the field of network technology, and in particular relates to an InfiniBand network card virtualization networking system. Background Art

[0002] InfiniBand (IB) is a high-bandwidth, low-latency network technology widely used in high-performance computing, cloud computing, large language models, and other fields. From top to bottom, an InfiniBand network can be divided into the application layer, transport layer, network layer, link layer, and physical layer. InfiniBand and RoCE networks are both RDMA networks. Both use the IB application layer and transport layer, but the network layer, link layer, and physical layer differ. RoCE networks use the Ethernet network layer and link layer, uniquely identifying network nodes through IP and MAC addresses. InfiniBand networks, on the other hand, use the IB network layer and link layer, uniquely identifying network nodes through GIDs and LIDs. Therefore, the InfiniBand networking process differs from that of RoCE networks.

[0003] With the development of virtualization technology, network card virtualization has become widely used. This technology virtualizes a single physical network card into multiple virtual network cards. According to the IB network specification, the network card needs to exchange various virtual network card information with the subnet manager (SubnetManager) to complete the virtual network card networking process.

[0004] Currently, the virtualization networking logic of mainstream IB network cards is implemented within the card hardware, which is inflexible, difficult to debug and develop, and consumes a lot of hardware resources. Therefore, to optimize the flexibility of IB network card virtualization functions and reduce hardware resource overhead, further research is needed on virtualization networking methods for InfiniBand network cards. Summary of the Invention

[0005] The present invention provides an InfiniBand network card virtualization networking system to solve the above-mentioned technical problems, specifically adopting the following technical solutions:

[0006] An InfiniBand network card virtualization networking system includes a driver software module, a network card hardware function module and a virtual QP mapping relationship module;

[0007] The driver software module is located in the host InfiniBand network card driver, and is used to forward the configuration information of the upper layer interface to the virtual network card to the network card hardware function module, and is also used to receive and parse the InfiniBand virtualization networking message and generate a corresponding response message;

[0008] The network card hardware function module is set in the InfiniBand network card and is used to correctly forward the messages sent and received by each virtual network card according to the virtual network card configuration information issued by the driver software module;

[0009] The virtual QP mapping relationship module is used to implement a one-to-one mapping relationship between logical QP and virtual QP number, ensuring the correctness and efficiency of the virtualized networking function.

[0010] Furthermore, the mapping method of the virtual QP mapping relationship module is:

[0011] The InfiniBand physical network card is split into k virtual network cards. Among them, the virtual network card numbered 0 is assigned m virtual QPs, and its virtual QP number is equal to the logical QP number. The virtual QP number increases from 0. The virtual network cards numbered 1 to k-1 are each assigned n virtual QPs. Their virtual QP numbers are not equal to the logical QP number. The virtual QP number increases from 1. For the virtual network card numbered 0, QP number mapping is not required. For the virtual network cards numbered 1 to k-1, the mapping formula between the logical QP number and the virtual QP number is: logical QP number = virtual QP number + (virtual network card number - 1) * n + m, thus ensuring one-to-one mapping.

[0012] Furthermore, the network card hardware function module includes:

[0013] Unicast lookup table, used to store the one-to-one mapping relationship between LID and virtual network card number;

[0014] Multicast forwarding lookup table, used to store the one-to-many mapping relationship between MLID and virtual network card number;

[0015] The unicast forwarding unit obtains the DLID and DQPN information of a unicast message, queries the unicast lookup table to obtain the virtual network card number, and then calculates the corresponding logical QP number according to the virtual QP mapping formula and forwards the message to the logical QP number.

[0016] The multicast forwarding unit queries the multicast forwarding lookup table in the network card hardware for a message whose DLID is a multicast type, and forwards the message to all nodes in the multicast group except the source node.

[0017] Furthermore, the driver software module includes:

[0018] a forwarding management unit, configured to receive configuration operations of the upper layer interface on the LID and MLID, establish a one-to-one mapping relationship between the LID and the virtual network card number, and a one-to-many mapping relationship between the MLID and the virtual network card number, and fill in the unicast lookup table and the multicast forwarding lookup table in the network card hardware function module respectively;

[0019] The virtual network card management unit is used to receive the configuration operations such as enabling and disabling, resource quota, priority, etc. of the virtual network card from the upper layer interface, and forward them to the network card hardware function module to realize the management function of the virtual network card;

[0020] The virtualization networking unit is configured to receive virtualization networking messages from QP0 and obtain information such as the operation type, operation field, and operation data of the message; for query operations, request corresponding information from the virtual network card management unit; for configuration operations, first perform a validity check and then issue a corresponding configuration command to the virtual network card management unit; and after completing all operations on the message, return a response message;

[0021] The multicast management unit is configured to receive multicast type messages from the virtual QP1 of each virtual network card, obtain information such as MLID and MGID of the multicast record, save the multicast record locally, and send a configuration command to the forwarding management unit.

[0022] Furthermore, the receiving process of the multicast forwarding unit is:

[0023] S11: The multicast forwarding unit receives a multicast message, where the multicast message comes from a network card port or its own virtual network card;

[0024] S12: The multicast forwarding unit searches for the DLID of the message in the multicast forwarding lookup table to determine whether there is a corresponding record for the DLID. If there is a record, the process jumps to S13; if not, the process jumps to S14.

[0025] S13: Search the multicast forwarding lookup table to obtain the corresponding virtual network card list, exclude the source node of the message from the list, and obtain the target virtual network card list to be forwarded;

[0026] S14: There is no record for this DLID, so the message is discarded directly and the process goes to S18.

[0027] S15: For each target virtual network card, calculate the corresponding logical QP number according to its virtual network card number and virtual QP number;

[0028] S16: Copy the message and forward it to the logical QP calculated in S15;

[0029] S17: Determine whether the request has been forwarded to all target virtual network cards in the list. If all requests have been forwarded, jump to S18. If not, jump to S15.

[0030] S18: The multicast forwarding unit completes the processing flow of a single multicast message.

[0031] Furthermore, the receiving process of the unicast forwarding unit is:

[0032] S21: The unicast forwarding unit receives a unicast message, where the unicast message comes from a network card port or its own virtual network card;

[0033] S22: The unicast forwarding unit searches the unicast lookup table for the DLID of the message and determines whether there is a corresponding record for the DLID. If there is a record, the process jumps to S23; if not, the process jumps to S24.

[0034] S23: Search the unicast lookup table to obtain the corresponding target virtual network card number;

[0035] S24: There is no record for this DLID, so the message is discarded directly and the process goes to S27.

[0036] S25: For each target virtual network card, calculate the corresponding logical QP number according to the target virtual network card number and the virtual QP number;

[0037] S26: Forward the message to the logical QP calculated in S25;

[0038] S27: The unicast forwarding unit completes the processing flow of a single unicast message.

[0039] The benefit of the present invention is that the provided InfiniBand network card virtualization networking system moves most of the virtualization networking logic from the network card hardware to the driver software, which can reduce the network card hardware resource overhead used for virtualization functions, shorten the debugging and development cycle, and thus reduce the cost of the network card. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A schematic diagram of an InfiniBand network card virtualization networking system of the present application;

[0042] Figure 2 This is a schematic diagram of the virtual QP mapping relationship module of this application;

[0043] Figure 3 This is a schematic diagram of the internal structure of the driver software module and the network card hardware function module of this application;

[0044] Figure 4 This is a schematic diagram of the receiving process of the multicast forwarding unit of this application;

[0045] Figure 5 This is a schematic diagram of the receiving process of the unicast forwarding unit of this application. DETAILED DESCRIPTION

[0046] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0047] like Figure 1 The figure shows the InfiniBand virtualized networking method and system of the present application, which includes a driver software module 10 , a network card hardware function module 20 and a virtual QP (queue pair) mapping relationship module 30 .

[0048] The driver software module 10 is located in the host InfiniBand network card driver, and is used to forward the upper layer interface's configuration information for the virtual network card to the network card hardware function module 20, and is also used to receive and parse InfiniBand virtualization networking messages, and then generate appropriate response messages.

[0049] The network card hardware function module 20 is provided in the InfiniBand network card, and is used to correctly forward the messages received and sent by each virtual network card according to the virtual network card configuration information delivered by the driver software module 10 .

[0050] The virtual QP mapping module 30 is used to implement a one-to-one mapping relationship between a real QP number (hereinafter referred to as a logical QP) and a virtual QP number, thereby ensuring the correctness and efficiency of the virtualized networking function.

[0051] like Figure 2 The figure shows the mapping relationship diagram of the virtual QP mapping relationship module 30 of the present application. The InfiniBand physical network card is split into k virtual network cards. Among them, the virtual network card numbered 0 is allocated m virtual QPs, and its virtual QP number is equal to the logical QP number, and the virtual QP number increases from 0. The virtual network cards numbered 1 to k-1 are each allocated n virtual QPs, and their virtual QP numbers are not equal to the logical QP number, and the virtual QP number increases from 1. For the network card numbered 0, the QP number does not need to be mapped; for the virtual network cards numbered 1 to k-1, the mapping calculation formula between the logical QP number and the virtual QP number is as follows, which can ensure a one-to-one mapping:

[0052] Logical QP number = virtual QP number + (virtual network card number - 1) * n + m

[0053] like Figure 3 Shown is a schematic diagram of the internal structure of the driver software module and the network card hardware function module of this application.

[0054] In an embodiment of the present application, the driver software module 10 includes a forwarding management unit 11 , a virtual network card management unit 12 , a virtualized networking unit 13 and a multicast management unit 14 .

[0055] The forwarding management unit 11 is used to receive the configuration operation of the upper-layer interface on the LID (local identifier) ​​and MLID (multicast local identifier), establish a one-to-one mapping relationship between the LID and the virtual network card number, and a one-to-many mapping relationship between the MLID and the virtual network card number, and fill in the unicast lookup table 21 and the multicast forwarding lookup table 22 in the network card hardware function module 20 respectively.

[0056] The virtual network card management unit 12 is used to receive configuration operations such as enabling and disabling, resource quota, priority, etc. of the virtual network card from the upper layer interface, and forward them to the network card hardware function module 20 to implement the management function of the virtual network card.

[0057] The virtualization networking unit 13 receives virtualization networking messages from QP0 and retrieves information such as the operation type, operation fields, and operation data. For query operations, it requests the corresponding information from the virtual network card management unit 12. For configuration operations, it first performs a validity check and then issues the corresponding configuration command to the virtual network card management unit 12. After completing all operations on the message, it returns a response message to the network card hardware function module 20.

[0058] The multicast management unit 14 is configured to receive multicast type messages from the virtual QP1 of each virtual network card, obtain the MLID, MGID (multicast global identifier) ​​and other information of the multicast record, save the multicast record locally, and send a configuration command to the forwarding management unit 11.

[0059] In the embodiment of the present application, the network card hardware function module 20 includes a unicast lookup table 21 , a multicast forwarding lookup table 22 , a multicast forwarding unit 23 , and a unicast forwarding unit 24 .

[0060] The unicast lookup table 21 is used to store a one-to-one mapping relationship between LIDs and virtual network card numbers.

[0061] The multicast forwarding lookup table 22 is used to store a one-to-many mapping relationship between MLIDs and virtual network card numbers.

[0062] The multicast forwarding unit 23 queries the multicast forwarding lookup table 22 for a message whose DLID is a multicast type, and forwards the message to all nodes in the multicast group except the source node.

[0063] The unicast forwarding unit 24 obtains the DLID (destination local identifier) ​​and DQPN (destination queue pair number) information of the unicast message with a unicast DLID, queries the unicast lookup table to obtain the virtual network card number, and then calculates the corresponding logical QP number according to the virtual QP mapping formula, and forwards the message to the logical QP number.

[0064] Furthermore, when the network card hardware function module 20 receives a message, it obtains its DLID information. If it is of MLID (multicast) type, it is processed by the multicast forwarding unit 23; if it is of unicast type, it is processed by the unicast forwarding unit 24.

[0065] like Figure 4 FIG. 2 is a schematic diagram of a receiving process of the multicast forwarding unit 23, and the process is as follows:

[0066] S11: The multicast forwarding unit receives a multicast message, which may come from the network card port or from its own virtual network card.

[0067] S12: The multicast forwarding unit searches for the DLID of the message in the multicast forwarding lookup table to determine whether there is a corresponding record for the DLID. If there is a record, the process jumps to S13; if not, the process jumps to S14.

[0068] S13: Search the multicast forwarding lookup table to obtain the corresponding virtual network card list, exclude the source node of the message from the list, and obtain the target virtual network card list to be forwarded.

[0069] S14: There is no record for this DLID, so the message is discarded directly and the process jumps to S18.

[0070] S15: For each target virtual network card, calculate the corresponding logical QP number according to its virtual network card number and virtual QP number.

[0071] S16: Copy the message and forward it to the logical QP calculated in S15.

[0072] S17: Determine whether the request has been forwarded to all target virtual network cards in the forwarding list. If all requests have been forwarded, jump to S18; if not, jump to S15.

[0073] S18: The multicast forwarding unit completes the processing flow of a single multicast message.

[0074] like Figure 5 FIG. 2 is a schematic diagram of a receiving process of the unicast forwarding unit 24, and the process is as follows:

[0075] S21: The unicast forwarding unit receives a unicast message, which may come from the network card port or from its own virtual network card.

[0076] S22: The unicast forwarding unit searches the unicast lookup table for the DLID of the message and determines whether there is a corresponding record for the DLID. If there is a record, the process jumps to S23; if not, the process jumps to S24.

[0077] S23: Search the unicast lookup table to obtain the corresponding target virtual network card number.

[0078] S24: There is no record for this DLID, so the message is discarded directly.

[0079] S25: Calculate the corresponding logical QP number according to the target virtual network card number and the virtual QP number.

[0080] S26: Forward the message to the logical QP calculated in S25.

[0081] S27: The unicast forwarding unit completes the processing flow of a single unicast message.

[0082] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solutions obtained by equivalent replacement or equivalent transformation fall within the scope of protection of the present invention.

Claims

1. An InfiniBand network card virtualization networking system, characterized in that: Contains driver software module, network card hardware function module and virtual QP mapping relationship module; The driver software module is located in the host InfiniBand network card driver, and is used to forward the configuration information of the upper layer interface to the virtual network card to the network card hardware function module, and is also used to receive and parse the InfiniBand virtualization networking message and generate a corresponding response message; The network card hardware function module is set in the InfiniBand network card and is used to correctly forward the messages sent and received by each virtual network card according to the virtual network card configuration information issued by the driver software module; The virtual QP mapping module is used to implement a one-to-one mapping relationship between logical QP and virtual QP number, ensuring the correctness and efficiency of virtualized networking functions; The mapping method of the virtual QP mapping relationship module is: The InfiniBand physical network card is split into k virtual network cards. The virtual network card numbered 0 is assigned m virtual QPs, whose virtual QP number is equal to the logical QP number and increases from 0. The virtual network cards numbered 1 to k-1 are each assigned n virtual QPs, whose virtual QP numbers are not equal to the logical QP number and increase from 1. For the virtual network card numbered 0, no QP number mapping is required. For the virtual network cards numbered 1 to k-1, the mapping formula between the logical QP number and the virtual QP number is: logical QP number = virtual QP number + (virtual network card number - 1) * n + m, thus ensuring a one-to-one mapping. The network card hardware function module includes: A unicast lookup table is used to store a one-to-one mapping relationship between the local identifier LID and the virtual network card number; A multicast forwarding lookup table is used to store a one-to-many mapping relationship between a multicast local identifier MLID and a virtual network card number; The unicast forwarding unit obtains the DLID and the target queue pair number (DQPN) of a unicast packet, queries the unicast lookup table to obtain the virtual network interface card (VNIC), calculates the corresponding logical QP number based on the virtual QP mapping formula, and forwards the packet to the logical QP number. The multicast forwarding unit queries the multicast forwarding table in the network card hardware for a message with a multicast DLID, and forwards the message to all nodes in the multicast group except the source node. The driver software module includes: a forwarding management unit, configured to receive configuration operations of the upper layer interface on the LID and MLID, establish a one-to-one mapping relationship between the LID and the virtual network card number, and a one-to-many mapping relationship between the MLID and the virtual network card number, and fill in the unicast lookup table and the multicast forwarding lookup table in the network card hardware function module respectively; The virtual network card management unit is used to receive the configuration operations such as enabling and disabling, resource quota, priority, etc. of the virtual network card from the upper layer interface, and forward them to the network card hardware function module to realize the management function of the virtual network card; The virtualization networking unit is configured to receive virtualization networking messages from QP0 and obtain information such as the operation type, operation field, and operation data of the message; for query operations, request corresponding information from the virtual network card management unit; for configuration operations, first perform a validity check and then issue a corresponding configuration command to the virtual network card management unit; and after completing all operations on the message, return a response message; The multicast management unit is configured to receive multicast type messages from the virtual QP1 of each virtual network card, obtain the MLID, multicast global identifier MGID and other information of the multicast record, save the multicast record locally, and send a configuration command to the forwarding management unit.

2. The InfiniBand network card virtualization networking system according to claim 1, wherein: The receiving process of the multicast forwarding unit is as follows: S11: The multicast forwarding unit receives a multicast message, which comes from a network card port or its own virtual network card; S12: The multicast forwarding unit searches for the DLID of the message in the multicast forwarding lookup table to determine whether there is a corresponding record for the DLID. If there is a record, the process jumps to S13; if not, the process jumps to S14. S13: Search the multicast forwarding lookup table to obtain the corresponding virtual network card list, exclude the source node of the message from the list, and obtain the target virtual network card list to be forwarded; S14: There is no record for this DLID, so the message is discarded directly and the process goes to S18. S15: For each target virtual network card, calculate the corresponding logical QP number according to its virtual network card number and virtual QP number; S16: Copy the message and forward it to the logical QP calculated in S15; S17: Determine whether the request has been forwarded to all target virtual network cards in the list. If all requests have been forwarded, jump to S18. If not, jump to S15. S18: The multicast forwarding unit completes the processing flow of a single multicast message.

3. The InfiniBand network card virtualization networking system according to claim 2, wherein: The receiving process of the unicast forwarding unit is as follows: S21: The unicast forwarding unit receives a unicast message, where the unicast message comes from a network card port or its own virtual network card; S22: The unicast forwarding unit searches the unicast lookup table for the DLID of the message and determines whether there is a corresponding record for the DLID. If there is a record, the process jumps to S23; if not, the process jumps to S24. S23: Search the unicast lookup table to obtain the corresponding target virtual network card number; S24: There is no record for this DLID, so the message is discarded directly and the process goes to S27. S25: For each target virtual network card, calculate the corresponding logical QP number according to the target virtual network card number and the virtual QP number; S26: Forward the message to the logical QP calculated in S25; S27: The unicast forwarding unit completes the processing flow of a single unicast message.

Images