InfiniBand network card virtualization networking system

Abstract

The invention discloses an InfiniBand network card virtualized networking system, which comprises a driving software module, a network card hardware function module, a network card hardware function module, an InfiniBand network card hardware function module and an InfiniBand network card hardware function module, wherein the driving software module is positioned in a host InfiniBand network card driving program, is used for forwarding configuration information of an upper layer interface on a virtual network card to the network card hardware function module, and is also used for receiving and analyzing an InfiniBand virtualized networking message and generating a corresponding response message; the network card hardware function module is arranged in an InfiniBand network card and is used for correctly forwarding messages transmitted and received by each virtual network card according to the virtual network card configuration information issued by the driving software module; and the virtual QP mapping relation module is used for realizing a one-to-one mapping relation from a logic QP to a virtual QP number, and ensuring the correctness and high efficiency of a virtualized networking function. According to the InfiniBand network card virtualization networking system provided by the invention, the hardware resource overhead of the network card for a virtualization function can be reduced, and the debugging and development period is shortened, so that the cost of the network card is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of network technology, and particularly relates to an InfiniBand network card virtualization networking system. Background Art

[0002] InfiniBand (IB) is a high-bandwidth and low-latency network technology, which is widely used in the fields of high-performance computing, cloud computing, large language models, etc. The InfiniBand network can be divided into an application layer, a transport layer, a network layer, a link layer, and a physical layer from top to bottom. InfiniBand and RoCE networks both belong to the RDMA network. They both use the IB application layer and transport layer, but their network layers, link layers, and physical layers are different. The RoCE network uses the Ethernet network layer and link layer, and uniquely identifies network nodes through IP and MAC addresses; while the InfiniBand network uses the IB network layer and link layer, and uniquely identifies network nodes through GID and LID. Therefore, the InfiniBand networking process is different from that of the RoCE network.

[0003] With the development of virtualization technology, network card virtualization technology has been widely applied. Network card virtualization technology virtualizes a physical network card into multiple virtual network cards. According to the IB network specification, the network card needs to exchange various information of the virtual network card with the SubnetManager to complete the networking process of the virtual network card.

[0004] Currently, the virtualization networking logic of mainstream IB network cards is implemented within the network card hardware, with poor flexibility, which is not conducive to debugging and development, and also occupies more hardware resources. Therefore, in order to optimize the flexibility of the virtualization function of IB network cards and reduce the hardware resource overhead, it is necessary to further study the virtualization networking method of InfiniBand network cards. Summary of the Invention

[0005] The present invention provides an InfiniBand network card virtualization networking system to solve the above-mentioned technical problems, and specifically adopts the following technical solutions: An InfiniBand network card virtualization networking system includes a driver software module, a network card hardware function module, and a virtual QP mapping relationship module; The driver software module is located in the host InfiniBand network card driver program, and is used to forward the configuration information of the virtual network card from the upper-layer interface to the network card hardware function module, and is also used to receive and parse the InfiniBand virtualization networking message and generate a corresponding response message; The network card hardware function module is set in the InfiniBand network card, and is used to correctly forward the messages sent and received by each virtual network card according to the virtual network card configuration information sent by the driver software module. The virtual QP mapping relationship module is used to implement a one-to-one mapping relationship between logical QPs and virtual QP numbers, ensuring the correctness and efficiency of the virtualized networking function.

[0006] Furthermore, the mapping method of the virtual QP mapping relationship module is as follows: The InfiniBand physical network card is split into k virtual network cards. Among them, the virtual network card numbered 0 is allocated m virtual QPs, and its virtual QP number is equal to the logical QP number. The virtual QP numbers start incrementing from 0. Each of the virtual network cards numbered 1 to k - 1 is allocated n virtual QPs, and its virtual QP number is not equal to the logical QP number. The virtual QP numbers start incrementing from 1. For the virtual network card numbered 0, no QP number mapping is required. For the virtual network cards numbered 1 to k - 1, the mapping calculation formula between the logical QP number and the virtual QP number is: logical QP number = virtual QP number + (virtual network card number - 1) * n + m, thus ensuring a one-to-one mapping.

[0007] Furthermore, the network card hardware function module includes: A unicast lookup table for storing the one-to-one mapping relationship between LIDs and virtual network card numbers; A multicast forwarding lookup table for storing the one-to-many mapping relationship between MLIDs and virtual network card numbers; A unicast forwarding unit. For a packet with a unicast type DLID, it obtains its DLID and DQPN information, queries the unicast lookup table to get the virtual network card number, and then calculates the corresponding logical QP number according to the virtual QP mapping formula, and forwards the packet to this logical QP number; A multicast forwarding unit. For a packet with a multicast type DLID, it queries the multicast forwarding lookup table in the network card hardware and forwards the packet to all nodes except the source node in the multicast group to which it belongs.

[0008] Furthermore, the driver software module includes: A forwarding management unit for receiving configuration operations of LIDs and MLIDs from the upper-layer interface, respectively establishing a one-to-one mapping relationship between LIDs and virtual network card numbers and a one-to-many mapping relationship between MLIDs and virtual network card numbers, and filling them into the unicast lookup table and the multicast forwarding lookup table in the network card hardware function module respectively; A virtual network card management unit for receiving configuration operations such as enabling / disabling, resource quota, and priority of virtual network cards from the upper-layer interface, and forwarding them to the network card hardware function module to implement the management function of virtual network cards; The virtualized networking unit is used to receive virtualized networking messages from QP0, and obtain information such as the operation type, operation fields, and operation data of the message from them; for query operations, it requests corresponding information from the virtual network card management unit, and for configuration operations, it first performs a legality check, and then issues corresponding configuration commands to the virtual network card management unit. After completing all operations of the message, it returns a response message. The multicast management unit is used to receive multicast type messages of virtual QP1 from each virtual network card, obtain information such as the MLID and MGID of the multicast record from them, save the multicast record locally, and issue a configuration command to the forwarding management unit.

[0009] Further, the receiving process of the multicast forwarding unit is as follows: S11: The multicast forwarding unit receives a multicast message, and the multicast message comes from a network card port or its own virtual network card; S12: The multicast forwarding unit looks up the DLID of the message in the multicast forwarding lookup table, and determines whether there is a corresponding record for the DLID. If there is a record, it jumps to S13; if there is no record, it jumps to S14; S13: Look up the multicast forwarding lookup table to obtain the corresponding virtual network card list, and exclude the source node of the message from it to obtain the target virtual network card list to be forwarded; S14: There is no record for this DLID, directly discard the message, and jump to S18; S15: For each target virtual network card, calculate the corresponding logical QP number according to its virtual network card number and virtual QP number; S16: Copy the message and forward it to the logical QP calculated in S15; S17: Determine whether it has been forwarded to all target virtual network card lists to be forwarded. If all have been forwarded, jump to S18; if not all have been forwarded, jump to S15; S18: The multicast forwarding unit completes the processing process of a single multicast message.

[0010] Further, the receiving process of the unicast forwarding unit is as follows: S21: The unicast forwarding unit receives a unicast message, and the unicast message comes from a network card port or its own virtual network card; S22: The unicast forwarding unit looks up the DLID of the message in the unicast lookup table, and determines whether there is a corresponding record for the DLID. If there is a record, it jumps to S23; if there is no record, it jumps to S24; S23: Look up the unicast lookup table to obtain the corresponding target virtual network card number; S24: There is no record for this DLID, directly discard the message, and jump to S27; S25: For each target virtual network card, calculate the corresponding logical QP number according to the target virtual network card number and the virtual QP number; S26: Forward the message to the logical QP calculated in S25; S27: The unicast forwarding unit completes the processing flow of a single unicast message.

[0011] The advantage of the present invention lies in the provided InfiniBand network card virtualization networking system, which moves most of the virtualization networking logic from the network card hardware to the driver software, reducing the network card hardware resource overhead for virtualization functions, shortening the debugging and development cycle, and thus reducing the network card cost. Description of the Drawings

[0012] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0013] Figure 1 It is a schematic diagram of an InfiniBand network card virtualization networking system of the present application; Figure 2 It is a schematic diagram of the virtual QP mapping relationship module of the present application; Figure 3 It is an internal structure schematic diagram of the driver software module and the network card hardware function module of the present application; Figure 4 It is a schematic diagram of the receiving process of the multicast forwarding unit of the present application; Figure 5 It is a schematic diagram of the receiving process of the unicast forwarding unit of the present application. Detailed Embodiments

[0014] The following details the embodiments of the present application. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application and should not be construed as a limitation to the present application.

[0015] As Figure 1 shown, the InfiniBand virtualization networking method and system of the present application include a driver software module 10, a network card hardware function module 20, and a virtual QP (queue pair) mapping relationship module 30.

[0016] The drive software module 10 is located in the host InfiniBand network card driver, and is used to forward the configuration information of the virtual network card by the upper-layer interface to the network card hardware function module 20, and is also used to receive and parse the InfiniBand virtualized networking packets, and then generate appropriate response packets.

[0017] The network card hardware function module 20 is set in the InfiniBand network card, and is used to correctly forward the packets sent and received by each virtual network card according to the virtual network card configuration information sent by the drive software module 10.

[0018] The virtual QP mapping relationship module 30 is used to implement a one-to-one mapping relationship between the real QP number (hereinafter referred to as the logical QP) and the virtual QP number, and ensure the correctness and efficiency of the virtualized networking function.

[0019] As Figure 2 shown is the mapping relationship schematic diagram of the virtual QP mapping relationship module 30 of the present application. The InfiniBand physical network card is split into k virtual network cards. Among them, the virtual network card numbered 0 is assigned m virtual QPs, and its virtual QP number is equal to the logical QP number, and the virtual QP numbering starts from 0 and increases. Each of the virtual network cards numbered 1 to k-1 is assigned n virtual QPs, and its virtual QP number is not equal to the logical QP number, and the virtual QP numbering starts from 1 and increases. For the network card numbered 0, the QP number does not need to be mapped; for the virtual network cards numbered 1 to k-1, the mapping calculation formula between the logical QP number and the virtual QP number is as follows, which can ensure a one-to-one mapping: Logical QP number = virtual QP number + (virtual network card number - 1) * n + m As Figure 3 shown is the internal structure schematic diagram of the drive software module and the network card hardware function module of the present application.

[0020] In the embodiment of the present application, the drive software module 10 includes a forwarding management unit 11, a virtual network card management unit 12, a virtualized networking unit 13, and a multicast management unit 14.

[0021] The forwarding management unit 11 is used to receive the configuration operations of the upper-layer interface on the LID (local identifier) and MLID (multicast local identifier), respectively establish a one-to-one mapping relationship between the LID and the virtual network card number, a one-to-many mapping relationship between the MLID and the virtual network card number, and fill them into the unicast lookup table 21 and the multicast forwarding lookup table 22 in the network card hardware function module 20 respectively.

[0022] The virtual network card management unit 12 is used to receive the configuration operations of the upper-layer interface on the virtual network card such as enabling and disabling, resource quota, and priority, and forward them to the network card hardware function module 20 to implement the management function of the virtual network card.

[0023] The virtual networking unit 13 is used to receive virtual networking messages from QP0, and obtain information such as the operation type, operation field, and operation data of the message. For query operations, it requests corresponding information from the virtual network card management unit 12; for configuration operations, it first performs a legality check and then issues corresponding configuration commands to the virtual network card management unit 12. After completing all operations on the message, it returns a response message to the network card hardware function module 20.

[0024] The multicast management unit 14 is used to receive multicast type messages of virtual QP1 from each virtual network card, obtain information such as the MLID and MGID (multicast global identifier) of the multicast record from them, save the multicast record locally, and issue a configuration command to the forwarding management unit 11.

[0025] In an embodiment of the present application, the network card hardware function module 20 includes a unicast lookup table 21, a multicast forwarding lookup table 22, a multicast forwarding unit 23, and a unicast forwarding unit 24.

[0026] The unicast lookup table 21 is used to save the one-to-one mapping relationship between the LID and the virtual network card number.

[0027] The multicast forwarding lookup table 22 is used to save the one-to-many mapping relationship between the MLID and the virtual network card number.

[0028] For a message with a DLID of the multicast type, the multicast forwarding unit 23 queries the multicast forwarding lookup table 22 and forwards the message to all nodes in the multicast group except the source node.

[0029] For a message with a DLID of the unicast type, the unicast forwarding unit 24 obtains its DLID (destination local identifier) and DQPN (destination queue pair number) information, queries the unicast lookup table to obtain the virtual network card number, and then calculates the corresponding logical QP number according to the virtual QP mapping formula, and forwards the message to this logical QP number.

[0030] Further, when the network card hardware function module 20 receives a message, it obtains its DLID information. If it is of the MLID (multicast) type, it is processed by the multicast forwarding unit 23; if it is of the unicast type, it is processed by the unicast forwarding unit 24.

[0031] As Figure 4 is a schematic diagram of the receiving process of the multicast forwarding unit 23, and the process is as follows: S11: The multicast forwarding unit receives a multicast message, which may come from a network card port or its own virtual network card.

[0032] S12: The multicast forwarding unit looks up the DLID of the packet in the multicast forwarding lookup table and determines whether there is a corresponding record for the DLID. If there is a record, it jumps to S13; if there is no record, it jumps to S14.

[0033] S13: Look up the multicast forwarding lookup table to obtain the corresponding virtual network card list, and exclude the source node of the packet from it to obtain the target virtual network card list to be forwarded.

[0034] S14: Since there is no record for this DLID, the packet is directly discarded and it jumps to S18.

[0035] S15: For each target virtual network card, calculate the corresponding logical QP number according to its virtual network card number and virtual QP number.

[0036] S16: Duplicate the packet and forward it to the logical QP calculated in S15.

[0037] S17: Determine whether it has been forwarded to all target virtual network card lists to be forwarded. If all have been forwarded, it jumps to S18; if not all have been forwarded, it jumps to S15.

[0038] S18: The multicast forwarding unit has completed the processing flow of a single multicast packet.

[0039] Such as Figure 5 is a schematic diagram of the receiving process of the unicast forwarding unit 24, and the process is as follows: S21: The unicast forwarding unit receives a unicast packet, which may come from a network card port or its own virtual network card.

[0040] S22: The unicast forwarding unit looks up the DLID of the packet in the unicast lookup table and determines whether there is a corresponding record for the DLID. If there is a record, it jumps to S23; if there is no record, it jumps to S24.

[0041] S23: Look up the unicast lookup table to obtain the corresponding target virtual network card number.

[0042] S24: Since there is no record for this DLID, the packet is directly discarded.

[0043] S25: Calculate the corresponding logical QP number according to the target virtual network card number and virtual QP number.

[0044] S26: Forward the packet to the logical QP calculated in S25.

[0045] S27: The unicast forwarding unit has completed the processing flow of a single unicast packet.

[0046] The basic principles, main features and advantages of the present invention have been shown and described above. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and all technical solutions obtained by means of equivalent replacement or equivalent transformation fall within the protection scope of the present invention.

Claims

1. An InfiniBand network card virtualization networking system, characterized in that: Contains driver software module, network card hardware function module and virtual QP mapping relationship module; The driver software module is located in the host InfiniBand network card driver, and is used to forward the configuration information of the upper layer interface to the virtual network card to the network card hardware function module, and is also used to receive and parse the InfiniBand virtualization networking message, and generate a corresponding response message; The network card hardware function module is arranged in the InfiniBand network card, and is used to correctly forward the messages received and sent by each virtual network card according to the virtual network card configuration information issued by the driver software module; The virtual QP mapping relationship module is used to implement a one-to-one mapping relationship between a logical QP and a virtual QP number, thereby ensuring the correctness and efficiency of the virtualized networking function.

2. The InfiniBand network card virtualization networking system according to claim 1, characterized in that: The mapping method of the virtual QP mapping relationship module is: The InfiniBand physical network card is split into k virtual network cards, where the virtual network card numbered 0 is allocated m virtual QPs, whose virtual QP number is equal to the logical QP number, and the virtual QP number increases from 0. The virtual network cards numbered 1 to k-1 are each allocated n virtual QPs, whose virtual QP numbers are not equal to the logical QP number, and the virtual QP number increases from 1. For the virtual network card numbered 0, the QP number does not need to be mapped. For the virtual network cards numbered 1 to k-1, the mapping calculation formula between the logical QP number and the virtual QP number is: logical QP number = virtual QP number + (virtual network card number - 1) * n + m, thereby ensuring a one-to-one mapping.

3. The InfiniBand network card virtualization networking system according to claim 1, characterized in that: The network card hardware function module includes: Unicast lookup table, used to store the one-to-one mapping relationship between LID and virtual network card number; Multicast forwarding lookup table, used to store the one-to-many mapping relationship between MLID and virtual network card number; The unicast forwarding unit obtains the DLID and DQPN information of the unicast type message, queries the unicast lookup table to obtain the virtual network card number, and then calculates the corresponding logical QP number according to the virtual QP mapping formula, and forwards the message to the logical QP number; The multicast forwarding unit queries the multicast forwarding lookup table in the network card hardware for a message whose DLID is a multicast type, and forwards the message to all nodes in the multicast group except the source node.

4. The InfiniBand network card virtualization networking system according to claim 3, characterized in that: The driver software module includes: A forwarding management unit, used to receive the configuration operation of the upper layer interface on the LID and the MLID, respectively establish a one-to-one mapping relationship between the LID and the virtual network card number, and a one-to-many mapping relationship between the MLID and the virtual network card number, and fill in the unicast lookup table and the multicast forwarding lookup table in the network card hardware function module respectively; A virtual network card management unit is used to receive configuration operations such as enabling and disabling, resource quota, priority, etc. of the virtual network card from the upper layer interface, and forward it to the network card hardware function module to implement the management function of the virtual network card; The virtualization networking unit is used to receive the virtualization networking message from QP0, and obtain the operation type, operation field, operation data and other information of the message; for the query operation, request the corresponding information from the virtual network card management unit; for the configuration operation, first perform a legality check, and then send the corresponding configuration command to the virtual network card management unit, and return a response message after completing all operations of the message; The multicast management unit is used to receive multicast type messages from virtual QP1 of each virtual network card, obtain MLID, MGID and other information of the multicast record, save the multicast record locally, and send a configuration command to the forwarding management unit.

5. The InfiniBand network card virtualization networking system according to claim 3, characterized in that: The receiving process of the multicast forwarding unit is: S11: The multicast forwarding unit receives a multicast message, where the multicast message comes from a network card port or its own virtual network card; S12: The multicast forwarding unit searches for the DLID of the message in the multicast forwarding lookup table to determine whether there is a corresponding record for the DLID. If there is a record, the process jumps to S13; if there is no record, the process jumps to S14; S13: Look up the multicast forwarding lookup table, obtain the corresponding virtual network card list, exclude the source node of the message, and obtain the target virtual network card list to be forwarded; S14: There is no record for this DLID, so the message is discarded directly and the process goes to S18; S15: For each target virtual network card, the corresponding logical QP number is calculated according to its virtual network card number and virtual QP number; S16: Copy the message and forward it to the logical QP calculated in S15; S17: Determine whether the forwarding has been completed to all target virtual network cards to be forwarded. If all have been forwarded, jump to S18; if not all have been forwarded, jump to S15; S18: The multicast forwarding unit completes the processing flow of a single multicast message.

6. The InfiniBand network card virtualization networking system according to claim 3, characterized in that: The receiving process of the unicast forwarding unit is: S21: The unicast forwarding unit receives a unicast message, where the unicast message comes from a network card port or its own virtual network card; S22: the unicast forwarding unit searches the unicast lookup table for the DLID of the message, and determines whether there is a corresponding record for the DLID. If there is a record, the process jumps to S23; if there is no record, the process jumps to S24; S23: Search the unicast lookup table to obtain the corresponding target virtual network card number; S24: There is no record for this DLID, so the message is discarded directly and the process goes to S27; S25: For each target virtual network card, calculate the corresponding logical QP number according to the target virtual network card number and the virtual QP number; S26: forward the message to the logical QP calculated in S25; S27: The unicast forwarding unit completes the processing flow of a single unicast message.

Images