RDMA network card single-root virtualization hardware implementation method supporting Ethernet

Abstract

The invention relates to a single-root virtualization hardware implementation method for an RDMA (Remote Direct Memory Access) network card supporting Ethernet. The method comprises the following steps of: offsetting an input read-write address through offset information; tlp header framing is carried out on the request information through the VF configuration information; the framing of tlp header and the distribution of an interrupt number are carried out on the request information; and maintaining a reqquery register of the rcq descriptor through a qpnum register, maintaining a target MAC register through a destmac register, carrying out message forwarding on the obtained target MAC address and the queue number information through the target MAC address, and reporting the corresponding queue number information. According to the method, the change of data and control paths is small, the resource consumption is low, the resource allocation is flexible, and the flexibility and compatibility are extremely high.

Description

Technical Field

[0001] The present invention relates to the technical field of network card virtualization, and in particular to a hardware implementation method for single-root virtualization of an RDMA network card supporting Ethernet. Background Art

[0002] With the rapid development of technologies such as artificial intelligence, cloud computing, and distributed computing, the scale and amount of data in data centers have increased significantly, which has put forward huge data transmission requirements for the distributed and parallel processing of big data applications. However, when processing large-scale data transmission, traditional TCP / IP (Transmission Control Protocol / Internet Protocol) technology needs to go through the operating system and software layers, which causes a large amount of server resources and memory bus bandwidth to be occupied, which in turn brings a heavy burden to the CPU (Central Processing Unit) and memory, especially when the network bandwidth, processor speed and memory bandwidth do not match, which further aggravates the network delay effect.

[0003] To address this problem, Remote Direct Memory Access (RDMA) technology has been introduced and widely used in modern data centers. RDMA technology uses CPU bypass and zero-copy mechanisms to effectively reduce the latency of data transmission, significantly reduce CPU load and memory bandwidth bottlenecks, and thus improve bandwidth utilization. Through RDMA, applications can directly perform read and write operations on the virtual memory of the remote server without going through the complex processing of the operating system and software layers. The core advantages of RDMA technology include extremely low transmission latency, high throughput, and minimal dependence on CPU resource requirements. This technology provides an efficient I / O channel that is particularly suitable for big data applications, enabling them to quickly and efficiently transmit and process data, thereby significantly improving the overall performance and efficiency within the data center. When dealing with large-scale parallel computing and data analysis tasks, the excellent performance of RDMA technology is particularly significant, providing important technical support and optimization solutions for the high-performance computing needs of modern data centers. In summary, RDMA technology not only improves the efficiency and response speed of data transmission, but also reduces the consumption of system resources. It has become one of the indispensable key technologies in modern data centers, bringing significant performance improvements and operational efficiency to various data-intensive application scenarios.

[0004] Therefore, an RDMA network card compatible with standard Ethernet has significant advantages in terms of performance, latency, throughput, CPU load, compatibility, flexibility, simplified network architecture, and data center optimization. It can work seamlessly in a standard network environment while taking advantage of RDMA in scenarios that require high-performance network transmission. This dual function makes the design and deployment of network infrastructure more flexible and adaptable to different application requirements and network environments. In addition, using a network card compatible with standard Ethernet and RDMA can support multiple application requirements on the same network infrastructure, reducing the procurement and maintenance costs of different network hardware. It simplifies the network architecture and reduces management complexity while ensuring high performance and high reliability of the network. These advantages not only improve the efficiency and performance of network transmission, but also provide more flexible and efficient solutions for complex and changing application scenarios. By rationally utilizing the characteristics of this network card, the data transmission and processing capabilities in the network environment can be greatly optimized to meet the needs of modern high-performance computing and data centers.

[0005] In addition, hardware-based virtualization technology currently mainly adopts the single-root virtualization standard. Single-root I / O Virtualization (SR-IOV) is a hardware virtualization technology that has been widely studied and applied in academia and industry, and is mainly used to improve the performance and efficiency of I / O devices in virtualized environments. In traditional software virtualization, network packets must pass through the network protocol stack of the host operating system, which causes additional CPU overhead and transmission delay. In contrast, SR-IOV allows VF to directly access the hardware resources of physical devices, avoiding the processing of these intermediate levels, so it can achieve performance levels close to the hardware itself. This is especially important for applications with high throughput, low latency and large-scale data transmission. The core concept of SR-IOV technology is to virtualize a single physical I / O device (such as a network card or storage controller) into multiple virtual functions (VFs) through hardware-level support. These VFs can be directly assigned to virtual machines or containers, and each VF has its own independent PCI configuration space and DMA (Direct Memory Access) engine. The basic functions of physical devices are managed and controlled by physical functions (PF), which are responsible for allocating and reclaiming VFs and providing management interfaces. SR-IOV technology uses a hardware-level resource isolation mechanism to ensure that data and resources between different VFs are completely isolated to prevent data leakage and conflicts. In addition, SR-IOV technology supports dynamic creation and destruction of VFs, and administrators can flexibly configure and allocate device resources to different virtual machines or containers according to actual needs. This flexibility makes resource management more efficient and can support the expansion needs in large-scale data centers and cloud computing environments. With the rapid development of emerging technologies such as big data, artificial intelligence, and cloud computing, the demand for efficient, flexible, and scalable data center solutions continues to increase. As a technology based on hardware virtualization, SR-IOV can provide a solid technical foundation and support for future technological development, and help the academic community achieve more innovative results in fields such as high-performance computing and large-scale data processing.

[0006] In summary, the hardware implements single-root virtualization function and supports RDMA network cards of standard Ethernet, which has extremely high flexibility and compatibility, can improve data center performance and efficiency, meet the high-performance requirements of virtualized environments, reduce system resource consumption and costs, enhance security and isolation, and provide necessary technical support for future technological development. This makes it have important application prospects and practical value in academic research, industrial applications and large-scale computing environments.

[0007] However, the difficulty of implementing single-root virtualization based on hardware on Ethernet-supported RDMA network cards is that it requires changes to the existing control path and data path, which will introduce new hardware processing delays. At the same time, it may increase resource consumption and is not friendly to hardware development platforms with limited resources. Summary of the invention

[0008] In order to solve the above problems existing in the prior art, the present invention provides a hardware implementation method for single-root virtualization of an RDMA network card supporting Ethernet.

[0009] According to a first aspect of an embodiment of the present invention, a method for implementing single-root virtualization hardware of an RDMA network card supporting Ethernet is provided, the method comprising:

[0010] Maintain the offset register through the VF_offset register to obtain the offset information;

[0011] By using the offset information to allocate MMIO resources, the input read and write addresses are offset to achieve allocation and isolation of MMIO resources of PF and VF;

[0012] Maintain the requester_id register of the tlp header through the VF_config register to obtain the VF configuration information;

[0013] By using the VF configuration information to allocate DMA resources, the requester_id information is framed by tlp header, so as to allocate and isolate the DMA resources of the PF and the VF;

[0014] Maintaining the requester_id register of the tlp header through the requester_id register to obtain the requester_id information;

[0015] By using the requester_id information to allocate interrupt resources, framing the requester_id information with a tlp header and allocating an interrupt number, the interrupt resources of the PF and the VF are allocated and isolated;

[0016] The req_queue register of the rcq descriptor is maintained through the qp_num register, and the destination MAC register is maintained through the dest_mac register to obtain the destination MAC address for forwarding RX traffic and the queue number information for calling the RQ queue;

[0017] The message is forwarded through the destination MAC address and the corresponding queue number information is reported, so as to realize correct forwarding and splitting of the RX traffic of the PF and the VF and reporting of the queue number.

[0018] Optionally, allocating MMIO resources by using the offset information to offset the input read and write addresses includes:

[0019] After the read / write address is input, judging whether it is VF according to the device function number carried by the read / write address;

[0020] When the read / write address is a VF, determining whether the read / write address is within a processable range of the VF;

[0021] When the read / write address is not within the processable range, the read / write address is discarded;

[0022] When the read / write address is within the processable range, the read / write address is offset and output according to the corresponding preset offset value, so as to allocate and isolate the MMIO resources of the PF and the VF.

[0023] Optionally, the VF configuration information includes: VF_en and VF_function; the VF_en is a 1-bit register, when the flag bit is 1, it indicates that the VF is enabled, when the flag bit is 0, it indicates that the VF is not enabled, and the initial value is 0; the VF_function is an 8-bit register, indicating the device function number corresponding to the VF, and the initial value is 0.

[0024] Optionally, allocating DMA resources by using the VF configuration information and performing tlp header framing on the requester_id information includes:

[0025] Obtain the first VF enable bit in the VF configuration information corresponding to the standard Ethernet mode;

[0026] When the first VF enable bit is not pulled high, the device function number corresponding to the PF is used as the information for tlp header framing;

[0027] When the first VF enable bit is pulled high, the device function number corresponding to the VF is used as information for TLP header framing;

[0028] The device function number, which is used as information for tlp header framing, is output to the DMA engine and the framing module of the tlp header to allocate and isolate the DMA resources of the PF and the VF.

[0029] Optionally, the allocating DMA resources by using the VF configuration information and performing tlp header framing on the requester_id information further includes:

[0030] Get the second VF enable bit in the VF configuration information corresponding to the RDMA mode;

[0031] When the second VF enable bit is not pulled high, the device function number of the PF is used as information for tlp header framing;

[0032] When the second VF enable bit is pulled high, the device function number corresponding to the VF is used as the information for TLP header framing;

[0033] The device function number, which is used as information for tlp header framing, is output to the DMA engine and the framing module of the tlp header to allocate and isolate the DMA resources of the PF and the VF.

[0034] Optionally, allocating interrupt resources by using the requester_id information, framing the requester_id information into a tlp header and allocating an interrupt number, includes:

[0035] Determine whether the interrupt is a VF interrupt according to the device function number in the requester_id information;

[0036] If the interrupt is not a VF interrupt, the interrupt number corresponding to the interrupt and the pcie device function number of the PF in the requester_id information are output to the PF interrupt processing module, and the first tlp data returned by the PF interrupt processing module is obtained;

[0037] If the interrupt is a VF interrupt, the interrupt number corresponding to the interrupt and the pcie device function number of the VF in the requester_id information are output to the VF interrupt processing module, and the second tlp data returned by the VF interrupt processing module is obtained;

[0038] Output valid data of the first tlp data or the second tlp data to allocate and isolate interrupt resources of the PF and the VF.

[0039] Optionally, forwarding the message by using the destination MAC address and reporting corresponding queue number information includes:

[0040] When the destination MAC address is the MAC address of the PF, forwarding the first queue number information corresponding to the message and the PF to the PF port, and reporting the first queue number information to the queue management module;

[0041] When the destination MAC address is the MAC address of the VF, the second queue number information corresponding to the message and the VF is forwarded to the VF port, and the second queue number information is reported to the queue management module.

[0042] According to a second aspect of an embodiment of the present invention, there is provided a hardware implementation device for implementing single-root virtualization of an RDMA network card supporting Ethernet, the device comprising:

[0043] A first information acquisition module is used to maintain an offset register through a VF_offset register to obtain offset information;

[0044] A first resource allocation module, configured to allocate MMIO resources by utilizing the offset information and offset the input read and write addresses to achieve allocation and isolation of MMIO resources of PF and VF;

[0045] The second information acquisition module is used to maintain the requester_id register of the tlp header through the VF_config register to obtain the VF configuration information;

[0046] A second resource allocation module, configured to allocate DMA resources by using the VF configuration information and perform tlp header framing on the requester_id information, so as to allocate and isolate the DMA resources of the PF and the VF;

[0047] A third information acquisition module is used to maintain the requester_id register of the tlp header through the requester_id register to obtain the requester_id information;

[0048] A third resource allocation module, configured to allocate interrupt resources by utilizing the requester_id information, perform tlp header framing and interrupt number allocation on the requester_id information, so as to allocate and isolate the interrupt resources of the PF and the VF;

[0049] The fourth information acquisition module is used to maintain the req_queue register of the rcq descriptor through the qp_num register, and maintain the destination MAC register through the dest_mac register, to obtain the destination MAC address for forwarding RX traffic, and the queue number information for calling the RQ queue;

[0050] The information processing module is used to forward the message through the destination MAC address and report the corresponding queue number information, so as to realize the correct forwarding and splitting of the RX traffic of the PF and the VF and the reporting of the queue number.

[0051] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; a memory for storing instructions executable by the processor;

[0052] The processor is configured to: execute the executable instructions to implement the steps of the method for hardware implementation of single-root virtualization of an RDMA network card supporting Ethernet as described in any one of the embodiments of the first aspect above.

[0053] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the hardware implementation method for single-root virtualization of an RDMA network card supporting Ethernet described in the first aspect of the present invention are implemented.

[0054] The technical solution provided by the present invention may include the following beneficial effects:

[0055] Through the above technical solution, SR-IOV virtualization can be realized by hardware means. Compared with traditional software virtualization, it does not cause additional CPU overhead and transmission delay, improves the performance and efficiency of I / O devices in the virtualized environment, and at the same time, by using the offset idea to allocate MMIO resources, the allocation of MMIO resources is achieved with minimal changes to the data and control paths. By calling existing signals, such as the function number and destination MAC address of the PCIe device, the allocation of DMA and interrupt resources and the forwarding of RX traffic are realized using fewer hardware resources. The present invention has the effects of small changes to the data and control paths, low resource consumption, high isolation, flexible resource allocation, and basically no introduction of new hardware processing delays. It has extremely high flexibility and compatibility, and has important application prospects and practical value in academic research, industrial applications and large-scale computing environments.

[0056] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention but do not constitute a limitation of the present invention. In the accompanying drawings:

[0058] Figure 1 The present invention is a flowchart of a method for hardware implementation of a single-root virtualization RDMA network card supporting Ethernet according to an exemplary embodiment.

[0059] Figure 2 It is a system framework diagram shown according to an exemplary embodiment.

[0060] Figure 3 The invention is a flowchart showing allocation and isolation of MMIO resources of PF and VF according to an exemplary embodiment.

[0061] Figure 4 is a schematic diagram showing a VF_config register format according to an exemplary embodiment.

[0062] Figure 5 The invention is a flowchart showing a method for allocating and isolating DMA resources of PF and VF according to an exemplary embodiment.

[0063] Figure 6 The flowchart of another method for allocating and isolating DMA resources of PF and VF is shown according to an exemplary embodiment.

[0064] Figure 7 The invention is a flowchart showing a method for allocating and isolating interruption resources of PF and VF according to an exemplary embodiment.

[0065] Figure 8 The invention is a flowchart showing a method for allocating and isolating interruption resources of PF and VF according to an exemplary embodiment.

[0066] Fig. 9 The present invention is a block diagram of a hardware implementation device for implementing single-root virtualization of an RDMA network card supporting Ethernet according to an exemplary embodiment.

[0067] Fig.10 The invention is a block diagram of an electronic device showing a method for implementing hardware of a single root virtualization of an RDMA network card supporting Ethernet according to an exemplary embodiment. DETAILED DESCRIPTION

[0068] Figure 1 FIG. 1 is a flowchart of a method for implementing single-root virtualization hardware of an Ethernet RDMA network card according to an exemplary embodiment. Figure 1 As shown, the method may include the following steps:

[0069] S101. Maintain the offset register through the VF_offset register to obtain offset information.

[0070] S102: Allocate MMIO resources by using the offset information and offset the input read and write addresses to achieve allocation and isolation of MMIO resources of PF and VF.

[0071] Figure 2is a system framework diagram according to an exemplary embodiment, and the present invention can be applied to the system, such as Figure 2 As shown, the system may include PEIe IP, MMIO resource allocation module, interrupt resource allocation module, DMA resource allocation module, queue management module, QPC module, RDMA ip module, interface module, RX module, RX (receiving side) traffic forwarding module, TX (transmitting side) module and MAC (media access control) module;

[0072] It is understandable that PCIe IP can generate IP through software. When generating IP, it is necessary to ensure that single-root virtualization enable and interrupt enable are turned on. In the MMIO resource allocation module, the offset register (offset register) related to MMIO resource allocation can be maintained through the VF_offset register (VF offset register) to provide offset information for the allocation of MMIO resources. Through MMIO resource allocation, the read and write addresses are offset according to the offset information, thereby realizing the allocation and isolation of MMIO resources for PF and virtual function VF.

[0073] Optionally, S102 may include:

[0074] When the read / write address is input, determine whether it is VF based on the device function number carried in the read / write address;

[0075] When the read / write address is VF, it is determined whether the read / write address is within the processable range of VF;

[0076] When the read / write address is not within the processable range, the read / write address is discarded;

[0077] When the read / write address is within the processable range, the read / write address is offset and output according to the corresponding preset offset value, so as to allocate and isolate the MMIO resources of the PF and VF.

[0078] In one embodiment, Figure 3 FIG. 1 is a flowchart showing allocation and isolation of MMIO resources of PF and VF according to an exemplary embodiment. Figure 3 As shown, there may be multiple processable intervals. When the read / write address is not in processable interval 1, it is determined whether the read / write address is in processable interval 2. When the read / write address is not in any processable interval, the read / write address is discarded. When the read / write address is in any processable interval, the read / write address is offset and output according to the corresponding preset offset value to achieve allocation and isolation of MMIO resources of PF and VF.

[0079] S103. Maintain the requester_id register of the tlp header through the VF_config register to obtain VF configuration information.

[0080] Optionally, Figure 4 is a schematic diagram of a VF_config register format according to an exemplary embodiment. The VF configuration information may include: VF_en and VF_function; VF_en is a 1-bit register, when the flag bit is 1, it indicates that the VF is enabled, when the flag bit is 0, it indicates that the VF is not enabled, and the initial value is 0; VF_function is an 8-bit register, indicating the first device function number corresponding to the VF, and the initial value is 0.

[0081] S104: DMA resources are allocated by using the VF configuration information, and tlp header framing is performed on the requester_id information to allocate and isolate the DMA resources of the PF and the VF.

[0082] It can be understood that the DMA resource allocation module can perform tlp header framing on the requester_id information according to the information provided by the VF_config register, thereby allocating and isolating the DMA resources of the physical function (PF) and the virtual function (VF);

[0083] In addition, the queue management module can maintain the five queue states of transmission, transmission completion, reception, reception completion and event queue of the corresponding network interface. At the same time, in this embodiment, it is also used to store the VF_config configuration information required for DMA resource allocation when the standard Ethernet card side hardware implements single-root virtualization;

[0084] It is worth mentioning that the RMDA ip module is used to implement the RMDA function, and the QPC module is placed in this module; the QPC module belongs to the RDMA ip module used to implement the RDMA function, and is used to maintain queue information for the network protocol stack module to perform framing and sending of RoCEV2 frames / Ethernet frames. At the same time, in this embodiment, it is also used to store the VF_config configuration information required for DMA resource allocation when the RDMA network card side hardware implements single-root virtualization;

[0085] In one implementation, an interface module is created, and the queue management module is connected to the RMDA ip containing the QPC module and placed in the interface module.

[0086] Optionally, S104 may include:

[0087] Obtain the first VF enable bit in the VF configuration information corresponding to the standard Ethernet mode;

[0088] When the first VF enable bit is not pulled high, the device function number corresponding to the PF is used as the information for TLP header framing;

[0089] When the first VF enable bit is pulled high, the device function number corresponding to the VF is used as the information for TLP header framing;

[0090] The device function number, which is used as information for tlp header framing, is output to the DMA engine and the framing module of the tlp header to allocate and isolate the DMA resources of the PF and VF.

[0091] In one embodiment, Figure 5 FIG. 1 is a flowchart showing how to allocate and isolate DMA resources of a PF and a VF according to an exemplary embodiment. Figure 5 As shown, in the standard Ethernet mode, the first VF enable bit in the VF configuration information written by the driver into the register in the standard Ethernet IP logic is obtained from the queue management module, and the device function number corresponding to the PF or the device function number corresponding to the VF is selected according to the specific situation of the first VF enable bit to perform TLP header (transaction layer data packet header) framing information.

[0092] Optionally, S104 may further include:

[0093] Get the second VF enable bit in the VF configuration information corresponding to the RDMA mode;

[0094] When the second VF enable bit is not pulled high, the device function number of the PF is used as the information for TLP header framing;

[0095] When the second VF enable bit is pulled high, the device function number corresponding to the VF is used as the information for TLP header framing;

[0096] The device function number, which is used as information for tlp header framing, is output to the DMA engine and the framing module of the tlp header to allocate and isolate the DMA resources of the PF and VF.

[0097] In one embodiment, Figure 6 FIG. 1 is another flowchart showing how to allocate and isolate DMA resources of PF and VF according to an exemplary embodiment. Figure 6 As shown, in RDMA mode, the first VF enable bit in the VF configuration information written by the driver into the register in the RDMA ip logic is obtained from the QPC (Queue Pair Context) module, and the device function number corresponding to the PF or the device function number corresponding to the VF is selected according to the specific situation of the first VF enable bit to perform tlp header framing information.

[0098] S105. Maintain the requester_id register of the tlp header through the requester_id register to obtain the requester_id information.

[0099] S106 , by using the requester_id information to allocate interrupt resources, tlpheader framing and interrupt number allocation are performed on the requester_id information, so as to allocate and isolate the interrupt resources of the PF and VF.

[0100] It can be understood that the interrupt resource allocation module can perform tlp header framing and interrupt number allocation on the requester_id (requester number) information according to the information provided by the requester_id register through interrupt resource allocation, thereby allocating and isolating the interrupt resources of the physical function (PF) and the virtual function (VF).

[0101] Optionally, S106 may include:

[0102] Determine whether the interrupt is a VF interrupt based on the device function number in the requester_id information;

[0103] If the interrupt is not a VF interrupt, the interrupt number corresponding to the interrupt and the pcie device function number of the PF in the requester_id information are output to the PF interrupt processing module, and the first tlp data returned by the PF interrupt processing module is obtained;

[0104] If the interrupt is a VF interrupt, the interrupt number corresponding to the interrupt and the pcie device function number of the VF in the requester_id information are output to the VF interrupt processing module, and the second tlp data returned by the VF interrupt processing module is obtained;

[0105] Output valid data in the first tlp data or the second tlp data to allocate and isolate interrupt resources of the PF and the VF.

[0106] In one embodiment, Figure 7 FIG. 1 is a flowchart showing how to allocate and isolate interrupt resources of PF and VF according to an exemplary embodiment. Figure 7 As shown, the interrupt is judged to be a PF interrupt or a VF interrupt, and then the corresponding pcie device function number is output to the corresponding interrupt processing module. Then the tlp data returned by the corresponding module is obtained, and the valid data in the first tlp data or the second tlp data is output to realize the allocation and isolation of the interrupt resources of PF and VF.

[0107] S107. Maintain the req_queue (request queue) register of the rcq (receive completion queue) descriptor through the qp_num (queue number) register, and maintain the destination MAC register through the dest_mac register to obtain the destination MAC address for forwarding RX traffic and the queue number information for calling the RQ queue.

[0108] It can be understood that the RX module can manage the receiving data path, including receiving the message returned by the MAC, dequeuing and acquiring the DMA descriptor, receiving the DMA data packet, writing back the DMA data, recording and writing back the completion of the DMA enqueue, and processing the PTP timestamp. In this embodiment, it is used to connect the eSwitch module and the interface module;

[0109] The eSwitch module, i.e., the RX traffic forwarding module, can maintain the req_queue register of the rcq descriptor through the qp_num register, maintain the destination MAC register through the dest_mac register, provide the destination MAC information for the forwarding of RX traffic, and provide the queue number information for calling the RQ queue. Through RX traffic forwarding, packets are forwarded according to the destination MAC and reported according to the corresponding queue number, thereby realizing the correct forwarding, diversion, and queue number reporting of the RX traffic of the physical function (PF) and the virtual function (VF);

[0110] The TX module can manage the transmit data path, including the transmission of messages to the MAC module, the dequeuing and acquisition of DMA descriptors, the DMA data and packet transfers, the recording and writing back of the enqueuing completion through DMA, and the processing of timestamps;

[0111] The MAC module can provide functions such as addressing mechanism, data frame construction, data error checking, transmission control, and providing a standard data interface to the network layer.

[0112] S108. Forward the message by the destination MAC address and report the corresponding queue number information, so as to correctly forward and split the RX traffic of PF and VF and report the queue number.

[0113] Optionally, S108 may include:

[0114] When the destination MAC address is the MAC address of the PF, the first queue number information corresponding to the message and the PF is forwarded to the PF port, and the first queue number information is reported to the queue management module;

[0115] When the destination MAC address is the MAC address of the VF, the second queue number information corresponding to the message and the VF is forwarded to the VF port, and the second queue number information is reported to the queue management module.

[0116] In one embodiment, Figure 8 FIG. 1 is a flowchart showing how to allocate and isolate interrupt resources of PF and VF according to an exemplary embodiment. Figure 8 As shown, by judging whether the destination MAC address belongs to a virtual function, the message is forwarded and the corresponding queue number information is reported, so as to realize correct forwarding and splitting of the RX traffic of PF and VF and reporting of the queue number.

[0117] The single-root virtualization hardware implementation method of the Ethernet-supporting RDMA network card provided by the present invention can realize SR-IOV virtualization by hardware means, and compared with the traditional software virtualization, it does not cause additional CPU overhead and transmission delay, and improves the performance and efficiency of I / O devices in the virtualized environment. At the same time, it is compatible with standard Ethernet cards and RDMA network cards, and can provide a seamless experience in various environments.

[0118] The present invention allocates MMIO resources through the offset concept, realizes the allocation of MMIO resources with minimal changes to the data and control paths, and uses fewer hardware resources to realize the allocation of DMA and interrupt resources and the forwarding of RX traffic by calling existing signals, such as device function numbers and destination MAC addresses, with the beneficial effects of small changes to data and control paths, low resource consumption, high isolation, flexible resource allocation, and basically no introduction of new hardware processing delays. In addition, the present invention realizes both compatibility with standard Ethernet and support for RDMA functions in the same network card, and realizes single-root virtualization functions based on hardware, has extremely high flexibility and compatibility, can improve data center performance and efficiency, and meet the high-performance requirements of virtualized environments, and has important application prospects and practical value in academic research, industrial applications, and large-scale computing environments.

[0119] Fig. 9 is a block diagram of a hardware implementation device for implementing a single-root virtualization of an RDMA network card supporting Ethernet according to an exemplary embodiment. Fig. 9 As shown, the apparatus 900 may include:

[0120] A first information acquisition module 901 is used to maintain an offset register through a VF_offset register to obtain offset information;

[0121] A first resource allocation module 902, configured to allocate MMIO resources by using offset information, and offset the input read and write addresses, so as to allocate and isolate the MMIO resources of the PF and the VF;

[0122] The second information acquisition module 903 is used to maintain the requester_id register of the tlp header through the VF_config register to obtain the VF configuration information;

[0123] The second resource allocation module 904 is used to allocate DMA resources by using VF configuration information and perform tlp header framing on the requester_id information to achieve allocation and isolation of DMA resources of PF and VF;

[0124] The third information acquisition module 905 is used to maintain the requester_id register of the tlp header through the requester_id register to obtain the requester_id information;

[0125] The third resource allocation module 906 is used to allocate interrupt resources by using the requester_id information, perform tlp header framing and interrupt number allocation on the requester_id information, so as to allocate and isolate the interrupt resources of the PF and VF;

[0126] The fourth information acquisition module 907 is used to maintain the req_queue register of the rcq descriptor through the qp_num register, and maintain the destination MAC register through the dest_mac register, to obtain the destination MAC address for forwarding RX traffic, and the queue number information for calling the RQ queue;

[0127] The information processing module 908 is used to forward the message through the destination MAC address and report the corresponding queue number information, so as to realize the correct forwarding and splitting of the RX traffic of PF and VF and the reporting of the queue number.

[0128] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0129] Fig.10 1 is a block diagram of an electronic device for supporting a hardware implementation method of a single root virtualization of an Ethernet RDMA network card according to an exemplary embodiment. Fig.10 As shown, the electronic device 1000 may include: a processor 1001 and a memory 1002 .

[0130] The processor 1001 is used to control the overall operation of the electronic device 1000 to complete all or part of the steps in the above-mentioned hardware implementation method for supporting Ethernet RDMA network card single root virtualization. The memory 1002 is used to store various types of data to support the operation of the electronic device 1000, and these data may include, for example, instructions for any application or method operating on the electronic device 1000, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 1002 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0131] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned method for implementing hardware of a single-root virtualization RDMA network card supporting Ethernet are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 1002 including program instructions, and the above-mentioned program instructions may be executed by the processor 1001 of the electronic device 1000 to complete the above-mentioned method for implementing hardware of a single-root virtualization RDMA network card supporting Ethernet.

[0132] In another exemplary embodiment, a computer program product is also provided, which includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the above-mentioned hardware implementation method of single-root virtualization of an RDMA network card supporting Ethernet when executed by the programmable device.

[0133] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

[0134] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0135] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A method for implementing single-root virtualization hardware of an RDMA network card supporting Ethernet, characterized in that: The method comprises: Maintain the offset register through the VF_offset register to obtain the offset information; By using the offset information to allocate MMIO resources, the input read and write addresses are offset to achieve allocation and isolation of MMIO resources of PF and VF; Maintain the requester_id register of the tlp header through the VF_config register to obtain the VF configuration information; By using the VF configuration information to allocate DMA resources, the requester_id information is framed by tlp header, so as to allocate and isolate the DMA resources of the PF and the VF; Maintaining the requester_id register of the tlp header through the requester_id register to obtain the requester_id information; By using the requester_id information to allocate interrupt resources, tlpheader framing and interrupt number allocation are performed on the requester_id information, so as to allocate and isolate the interrupt resources of the PF and the VF; The req_queue register of the rcq descriptor is maintained through the qp_num register, and the destination MAC register is maintained through the dest_mac register to obtain the destination MAC address for forwarding RX traffic and the queue number information for calling the RQ queue; The message is forwarded through the destination MAC address and the corresponding queue number information is reported, so as to realize correct forwarding and splitting of the RX traffic of the PF and the VF and reporting of the queue number.

2. The method for implementing single-root virtualization hardware of an Ethernet-supporting RDMA network card according to claim 1, characterized in that: The step of allocating MMIO resources by utilizing the offset information to offset the input read and write addresses includes: After the read / write address is input, judging whether it is VF according to the device function number carried by the read / write address; When the read / write address is a VF, determining whether the read / write address is within a processable range of the VF; When the read / write address is not within the processable range, the read / write address is discarded; When the read / write address is within the processable range, the read / write address is offset and output according to the corresponding preset offset value, so as to allocate and isolate the MMIO resources of the PF and the VF.

3. The method for implementing single-root virtualization hardware of an Ethernet-supporting RDMA network card according to claim 1, characterized in that: The VF configuration information includes: VF_en and VF_function; the VF_en is a 1-bit register, when the flag bit is 1, it indicates that the VF is enabled, when the flag bit is 0, it indicates that the VF is not enabled, and the initial value is 0; the VF_function is an 8-bit register, indicating the device function number corresponding to the VF, and the initial value is 0.

4. The method for implementing single-root virtualization hardware of an Ethernet-supporting RDMA network card according to claim 3, characterized in that: The method of allocating DMA resources by using the VF configuration information and performing tlp header framing on the requester_id information includes: Obtain the first VF enable bit in the VF configuration information corresponding to the standard Ethernet mode; When the first VF enable bit is not pulled high, the device function number corresponding to the PF is used as the information for tlp header framing; When the first VF enable bit is pulled high, the device function number corresponding to the VF is used as the information for tlp header framing; The device function number, which is the information for tlp header framing, is output to the DMA engine and the framing module of the tlp header to allocate and isolate the DMA resources of the PF and the VF.

5. The method for implementing single-root virtualization hardware of an Ethernet-supporting RDMA network card according to claim 1, characterized in that: The method of allocating DMA resources by using the VF configuration information and performing TLP header framing on the requester_id information further includes: Get the second VF enable bit in the VF configuration information corresponding to the RDMA mode; When the second VF enable bit is not pulled high, the device function number of the PF is used as information for tlp header framing; When the second VF enable bit is pulled high, the device function number corresponding to the VF is used as the information for tlp header framing; The device function number, which is the information for tlp header framing, is output to the DMA engine and the framing module of the tlp header to allocate and isolate the DMA resources of the PF and the VF.

6. The method for implementing single-root virtualization hardware of an Ethernet-supporting RDMA network card according to claim 1, characterized in that: The method of allocating interrupt resources by using the requester_id information, framing the requester_id information into a tlpheader and allocating an interrupt number includes: Determine whether the interrupt is a VF interrupt according to the device function number in the requester_id information; If the interrupt is not a VF interrupt, the interrupt number corresponding to the interrupt and the pcie device function number of the PF in the requester_id information are output to the PF interrupt processing module, and the first tlp data returned by the PF interrupt processing module is obtained; If the interrupt is a VF interrupt, the interrupt number corresponding to the interrupt and the pcie device function number of the VF in the requester_id information are output to the VF interrupt processing module, and the second tlp data returned by the VF interrupt processing module is obtained; Output valid data of the first tlp data or the second tlp data to allocate and isolate interrupt resources of the PF and the VF.

7. The method for implementing single-root virtualization hardware of an Ethernet-supporting RDMA network card according to claim 1, characterized in that: The forwarding of the message by the destination MAC address and reporting the corresponding queue number information includes: When the destination MAC address is the MAC address of the PF, forwarding the first queue number information corresponding to the message and the PF to the PF port, and reporting the first queue number information to the queue management module; When the destination MAC address is the MAC address of the VF, the second queue number information corresponding to the message and the VF is forwarded to the VF port, and the second queue number information is reported to the queue management module.

8. A hardware implementation device for single-root virtualization of an RDMA network card supporting Ethernet, characterized in that: The device comprises: A first information acquisition module is used to maintain an offset register through a VF_offset register to obtain offset information; A first resource allocation module, configured to allocate MMIO resources by utilizing the offset information and offset the input read and write addresses to achieve allocation and isolation of MMIO resources of PF and VF; The second information acquisition module is used to maintain the requester_id register of the tlp header through the VF_config register to obtain the VF configuration information; A second resource allocation module, configured to allocate DMA resources by using the VF configuration information and perform tlp header framing on the requester_id information, so as to allocate and isolate the DMA resources of the PF and the VF; A third information acquisition module is used to maintain the requester_id register of the tlp header through the requester_id register to obtain the requester_id information; A third resource allocation module, configured to allocate interrupt resources by utilizing the requester_id information, perform tlp header framing and interrupt number allocation on the requester_id information, so as to allocate and isolate the interrupt resources of the PF and the VF; The fourth information acquisition module is used to maintain the req_queue register of the rcq descriptor through the qp_num register, and maintain the destination MAC register through the dest_mac register, to obtain the destination MAC address for forwarding RX traffic, and the queue number information for calling the RQ queue; The information processing module is used to forward the message through the destination MAC address and report the corresponding queue number information, so as to realize the correct forwarding and splitting of the RX traffic of the PF and the VF and the reporting of the queue number.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the hardware implementation method of the Ethernet-supporting RDMA network card single root virtualization described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for implementing single-root virtualization hardware of an RDMA network card supporting Ethernet described in any one of claims 1-7 are implemented.

Images