Apparatus and method for pooling network interface cards in cloud network

Abstract

A network interface component is disclosed having a CPU and a pool of physical NIC for providing network interface resources to one or more virtual NIC. An operating system of the network interface component creates one or more virtual NIC to facilitate data flow to and from virtual machines running on one or more servers coupled to the network interface component. The operating system dynamically creates and manages as many virtual NICs as needed to process traffic flows to / back to one or more virtual machines running on one or more servers coupled to the network interface component. The operating system also indicates which of the physical NICs should be used or which of the virtual NICs should be shared. Any physical NIC from the pool can be shared by more than one virtual NIC, and can also be reallocated to another virtual NIC according to traffic flow over a cloud computing network.

Description

Technical Field

[0001] This application relates to apparatus and methods for pooling network interface cards in a cloud network. Background Technology

[0002] Figure 1 A traditional cloud network system is illustrated. All hardware servers 101 to 103 have one or more dedicated network interface cards (NICs) among their own 104 to 106. A server may have multiple NICs, depending on how many slots are available on the motherboard of these NICs. All data processed by servers 101 to 103 (whether inbound or outbound) is required to pass through its corresponding physical NIC. NICs 104 to 106 are connected to one or more switches 107. Therefore, all data transfers between servers 101 to 103 are performed through switch 107.

[0003] For example, if virtual machine 2 (VM2) 109 transmits data to virtual machine 108, it needs to bridge its virtual NIC to physical NIC 2 105, as shown in data path 111. The data is then sent from NIC 2 105 to switch 107, as shown in data path 112. Switch 107 then sends the data to physical NIC 1 104 of server 104 via data path 113. Finally, NIC 1 104 transmits the data to virtual machine 1 (VM1) 108. Similarly, data transmission from VM1 108 to VM3 110 follows data paths 115 to 118. The data is first sent to physical NIC 1 104 of server 1 101, then from NIC 1 104 to switch 107, then from switch 107 to NIC 3 106 of server 3 103, and finally to virtual machine 3 (VM3) 110.

[0004] Because all data transmission must pass through physical NICs, traditional cloud networks are limited by the throughput of these NICs. Each individual virtual machine can only use its local physical NIC to transmit data. Virtual machines cannot pool or combine physical NICs from different servers.

[0005] The limitations imposed by the way physical NICs are implemented in cloud network systems are becoming increasingly harmful. Current computing trends favor higher speeds and larger volumes of data transfer, particularly for artificial intelligence (AI) applications. Data transfers typically occur within the cloud network (e.g., between virtual machines). This is known as east-west data transfer or intranet data transfer. For example, AI systems like OpenAI typically perform billions of bits of computation per second for processing within the cloud network. This scale of computation requires thousands of AI semiconductor chips to work collaboratively across multiple virtual machines. This places a massive data transfer load between virtual machines and AI chips. The problem is that, due to the physical limitations of the available physical ports on server motherboards, only a limited number of NICs can be installed on any given server. This limits the speed and volume of data that can be achieved in east-west data transfers. Furthermore, often, the NICs on a particular server are not fully utilized, while other servers lack NIC capacity. This results in a severe underutilization and inefficient use of limited and expensive NIC resources. Summary of the Invention

[0006] This invention discloses a network interface component having a CPU and a physical NIC pool, the physical NIC pool being used to provide network interface resources to one or more virtual NICs. The operating system of the network interface component creates one or more virtual NICs to facilitate data flow to and from virtual machines running on one or more servers coupled to the network interface component. The operating system dynamically creates and manages as many virtual NICs as are needed to process the traffic flow to / from one or more virtual machines running on one or more servers coupled to the network interface component. The operating system also instructs which physical NICs should be used or shared among which virtual NICs. Any physical NIC from the pool can be shared by more than one virtual NIC and can also be reallocated to another virtual NIC based on traffic flow over a cloud computing network.

[0007] A better understanding of the features and advantages of the invention will be obtained by referring to the following detailed description and accompanying drawings, which illustrate illustrative embodiments utilizing the principles of the invention. Attached Figure Description

[0008] Figure 1 This illustrates a traditional cloud network system.

[0009] Figure 2 A block diagram is shown, illustrating an example of a cloud computing network with pooled network interface components according to an embodiment of the present invention.

[0010] Figure 3An embodiment of the pooled network interface component of the present invention is shown.

[0011] Figure 4 An exemplary data flow diagram according to an embodiment of the present invention is shown.

[0012] Figure 5 A computer network system according to an exemplary embodiment of the present invention is shown. Detailed Implementation

[0013] Figure 2 A block diagram is shown illustrating an example of a cloud computing network with a pooled network interface component 216 according to an embodiment of the present invention. As described in more detail below, virtual circuitry in the network device reformats one or more incoming data streams at a non-predetermined data rate into one or more outgoing data streams at a predetermined fixed data rate. This, in turn, allows multiple data streams with non-predetermined data rates (which are lower than the predetermined fixed data rate) to be combined and output from a single network port, and allows a single data stream with non-predetermined data rates (which are higher than the predetermined fixed data rate) to be segmented and output from two or more network ports.

[0014] like Figure 2 As shown, multiple virtual machines 201 to 206 are coupled to servers 207 to 209. These virtual machines can be coupled to any specific server. Servers 207 to 209 are coupled to a pooled NIC component 216. The servers and their corresponding virtual machines share the pooled NIC resources available in the pooled NIC component 216. The pooled NIC component 216 includes, but is not limited to, multiple virtual NICs (e.g., virtual NICs 210 to 212) and multiple physical NICs (e.g., NIC1 213, NIC2 214, and NIC3 215). Physical NICs 213 to 215 form a NIC group 217. NIC group 217 can have many different physical NICs and can add more physical NICs as needed. Virtual NIC operating system software is used to operate the pooled NIC component 216. The pooled NIC component 216 is coupled to the Internet backbone 218.

[0015] In one embodiment, a set of physical NICs (e.g., NICs 213 to 215 of NIC group 217) are separate from their corresponding servers (e.g., servers 207 to 209). In other words, these physical NICs do not reside on any single server. Instead, they reside within NIC group 217 of pooled NIC component 216. It is possible that one or more servers may have their own physical NICs. However, the pooled physical NICs of the NIC group are shared among the various servers. Therefore, virtual machines (e.g., VMs 201 to 206) created in the physical servers (e.g., servers 207 to 209) can connect to virtual NICs (e.g., virtual NICs 210 to 212) within the pooled NIC component 216. In one embodiment, NIC group 217 is connected to the physical servers via PCIe cables.

[0016] The pooled NIC component 216 has an operating system, namely the virtual NIC OS 219, which can generate one or more virtual NICs through Network Element Virtualization (NEV) and standard virtual NIC software. The operating system can dynamically create and delete virtual NICs as needed. NEV is a technology for abstracting / virtualizing underlying network resources. This is consistent with VMware's concept of abstracting and virtualizing x86 server resources. NEV essentially allows the virtualization of network elements, thereby enabling the virtualization of networking elements, cables, routers, switches, gateways, and existing NICs. Multiple virtual connections can be transmitted over the same physical connection, while bandwidth and latency can be dynamically adjusted as needed. Therefore, the physical network interface can carry many NEV service virtual interfaces. The virtual NIC OS follows the PCIe protocol to connect to physical servers. Unlike other virtual NICs, it should generate multiple virtual NICs for multiple server connections based on the PCIe protocol. This is a fully meshed connection. On the other hand, in one embodiment of the invention, the virtual NIC is based on NEV technology and can generate an unlimited number of physical-like ports, rather than the logical ports typically found in virtual NICs.

[0017] In one embodiment, each virtual NIC supports NEV and can provide unlimited bandwidth ports via NEV. Each virtual NIC can span and cover multiple physical NICs via NEV technology. Multiple virtual NICs can be merged to form a higher-level virtual NIC. This means that virtual NICs can have additional layers. Furthermore, multiple servers can use a single virtual NIC, and similarly, a single server can use multiple virtual NICs. Virtual machines 201 to 206 can also be directed to connect to virtual NICs 210 to 212 via bridged connections.

[0018] This system primarily pools or shares physical NICs, enabling numerous servers to utilize an unlimited number of physical NICs. It also generates customized NICs for each server via virtual NICs. If any one or more virtual NICs 210 to 212 are currently unloaded, they can be migrated to any server. The Virtual NIC OS 219 uses PCIe to connect the pooled NIC components 216 to each server 207 to 209. It follows the standard PCIe protocol, registering one or more virtual NICs for each server. For each server, a virtual NIC will have multiple IDs, allowing each server to determine whether a virtual NIC is available or unused / unallocated. The Virtual NIC OS 219 also creates one or more virtual NICs based on the resources of the physical NIC group 217 and uses NEV technology to create an unlimited number of NIC port interfaces. Therefore, it implements the functionality of physical NICs without the limitations of physical NICs. Depending on data traffic, the pooling of virtual and physical NICs can be dynamically controlled to handle increases and / or decreases in data traffic over the network more efficiently and effectively. For example, if data traffic increases for a specific server, the pooled network interface component can allocate additional virtual and physical NICs to handle the increased data flow for that server. Similarly, if data flow to / from a specific server decreases, the pooled network interface component can proportionally redirect virtual and physical NICs from that server to support other parts of the network. Pooled NICs / virtual NICs are used not only for VMs but also for physical servers. This means that within a physical server, it can also see the elastic / dynamic NICs connected to it. Therefore, any physical server can utilize virtual NICs (in the NIC pool) to connect to other physical servers, without being limited by (multiple) local physical NICs.

[0019] according to Figure 2The block diagram shows that data from VM1 201 on server 1 207 can be sent via virtual NIC 1 210, and then via physical NIC1 213 of NIC group 217 to virtual NIC2 211 on server 2 208, and then to VM3 203. No switch or backbone is required. VM1 can transmit data to VM2 via pooled NIC component 216. Similarly, any VM from 201 to 206 can directly transmit and receive data between any other VMs from 201 to 206, which are coupled to the pooled NIC component 216 via their respective virtual and physical NICs. Virtual NIC OS 219 allocates virtual and physical NIC resources to efficiently and effectively handle internal east-west data transfers, all within the cloud computing network. Other advantages include providing alternative compute network interconnect solutions besides unlimited bandwidth or NVLink. Furthermore, support for arbitrary-to-arbitrary ultra-high-speed interconnects is implemented through virtualized network port pooling. For computing networks in data center and cloud environments, each host achieves a throughput of 1Tbps. Therefore, this invention provides features for a low-cost, scalable, and integrated NEV virtualization product.

[0020] Figure 3 An embodiment of the pooled network interface component of the present invention is illustrated. Multiple host computers or servers 301 to 303 are shown. These computers are capable of running multiple virtual machines. Host computers 301 to 303 include operating systems 304 to 306, drivers 307 to 309, and NIC buffers 310 to 312. NIC buffers 310 to 312 are PCIe 6 endpoint devices. They have two PCIe endpoints (EPs); one connected to the host and the other connected to a NIC delivery point (POD) device 313. The memory (e.g., cache) chip within the NIC buffer has a read / write speed of 2 Tbit / s or higher. The NIC buffer is capable of supporting up to and more than 1024 virtual endpoints (vEPs), with a total support of 1 Tbps and a throughput of 1 Tbs. It is capable of managing bandwidth allocation for each virtual endpoint. It is exposed to the operating system as a standard EP / netdev, not the NIC. Furthermore, it is exposed to the NIC POD as a storage EP (e.g., DRAM memory). Synchronization for IRQ notifications was implemented between NIC buffers 310 to 312 and NIC POD 313.

[0021] The PCIe bus connects host computers 301 to 303 to NIC POD device 313. NIC POD device 313 includes a PCIe switch 314, a CPU 315, and physical NICs 316 to 320. It has a capacity of 16 to 24 Tb and features 1:1 non-blocking capability. The integrated PCIe switch 314 connects NIC buffers 310 to 312 to the physical NIC cards 316 to 320 of NIC POD 313. 16 to 24 hosts can be connected via the PCIe switch. The ability to read / write data in the NIC buffers at allocated bandwidth rates for specific virtual endpoints and services is distributed across the NIC pool by implementing appropriate algorithms. L2 Ethernet switch 321 switches between the different hosts 301 to 303.

[0022] Figure 4 An exemplary data flow diagram according to an embodiment of the present invention is shown. Host 1 CPU 401 uses a standard Linux protocol stack to process data on the endpoint EP (netdev) at a read / write speed of 1 terabits per second (Tbps). Host 1 has a throughput of 1 Tbps. Host 1 is capable of creating up to 1024 or more virtual EPs on the NIC buffer. Each virtual endpoint (vEP) is allocated a specific bandwidth. Therefore, the buffer is allocated in association with the bandwidth. The CPUs on the host and POD synchronize IRQ notifications across devices. The POD CPU processes data on the endpoint (EP) (e.g., a storage device). The POD CPU routes traffic from the vEP on the host via PCIe switch 402 and across the NIC port pool 403. An external L2 switch 404 performs traffic forwarding to Host 2. Host 2 CPU 405 uses a standard Linux protocol stack to process data on the EP (netdev). Data flow between Host 1, the EP (storage device), and the PCIe switch is via PCIe. Similarly, data flow between Host 2, EP (Storage Device), and the PCIe switch is via PCIe. Data flow between the PCIe switch, the NIC port pool, and the endpoint (EP) is via PCIe. Data flow between the endpoint (EP) and the L2 switch is via Ethernet.

[0023] Figure 5A computer network system according to an exemplary embodiment of the present invention is illustrated. NIC buffer 1 502 is included in x86 server host 1 501. It has a throughput of 80 gigabits per second (GBps) and has a PCIe connector connected to the CPU of host 1 and another PCIe cable connected to the NIC POD. Furthermore, it creates two virtual EPs: vEP1 with a bandwidth of 76 GBps for a total of 80 Gbps, and vEP2 with a bandwidth of 4 Gbps. NIC buffer 2 504 also runs on x86 server host 2 503. It has a throughput of 80 Gbps. It has a PCIe connector connected to the CPU of host 2 and another PCIe cable connected to the NIC POD. It creates two virtual EPs: vEP3 with a bandwidth of 76 Gbps for a total of 80 Gbps, and vEP2 with a bandwidth of 4 Gbps. The NIC POD has a throughput of 160 Gbps. It features a PCIe switch 505 to connect the POD CPU, NIC buffer 1, NIC buffer 2, and four 4×10G standard conventional NICs 506 to 509. The four 4×10G standard conventional NICs are connected to an external switch 510 (e.g., an Arista switch). Traffic flow from vEP1 on host 1 to vEP3 on host 2 has an iPerf transmit / receive rate of 76 bps. Traffic flow from vEP4 on host 1 to vEP1 has an iPerf transmit / receive rate of 4 Gbps. Furthermore, because this invention enables pooling of all NICs, the network can utilize NEV NICs to create customized NICs for each host, and not just provide standard NICs such as 10G or 200G (e.g., aggregating two 100G NICs). Conversely, using this invention, 123G NICs can be implemented using 100G standard NICs and 23G NEV NICs.

[0024] Reference will now be made in detail to various embodiments of this disclosure, examples of which are illustrated in the accompanying drawings. While various embodiments have been described in conjunction with this disclosure, it will be understood that these various embodiments are not intended to limit this disclosure. Rather, this disclosure is intended to cover alternatives, modifications, and equivalents that may be included within the scope of this disclosure as interpreted in the claims. Furthermore, numerous specific details have been set forth in the foregoing detailed description of various embodiments of this disclosure in order to provide a thorough understanding of this disclosure. However, those skilled in the art will understand that this disclosure may be practiced without these specific details or with their equivalents. In other instances, well-known methods, processes, components, and circuits have not been described in detail so as not to obscure aspects of the various embodiments of this disclosure unnecessarily.

[0025] Some parts of the detailed description are presented in terms of programs, logic blocks, processes, and other symbolic representations of operations on data bits within computer memory. These descriptions and representations are used by those skilled in the art of data processing to effectively communicate the substance of their work to others skilled in the art. In this disclosure, programs, logic blocks, processes, etc., are conceived as self-consistent sequences of operations or instructions that achieve a desired result. An operation is an operation that utilizes the physical manipulation of physical quantities. Typically (but not always), these quantities take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated in a computing system. Primarily for reasons of common use, referring to these signals as transactions, bits, values, elements, symbols, characters, samples, pixels, etc., has proven convenient in some cases.

[0026] However, it should be remembered that all these and similar terms should be associated with appropriate physical quantities and are merely convenient labels applied to those quantities. Unless otherwise specified (as will be apparent from the following discussion), it should be understood that throughout this disclosure, discussions using terms such as “generate,” “determine,” “allocate,” “aggregate,” “utilize,” “virtualize,” “process,” “access,” “execute,” and “store” refer to the actions and processes of a computer system or similar electronic computing device or processor. A computing system or similar electronic computing device or processor manipulates data represented as physical (electronic) quantities within computer system memory, registers, other such information storage devices and / or other computer-readable media, and converts it into other data, which are similarly represented as physical quantities within computer system memory or registers or other such information storage, transmission, or display devices.

[0027] The technical solutions in the embodiments of this application have been clearly and completely described in the preceding sections with reference to the accompanying drawings of the embodiments of this application. It should be noted that terms such as "first," "second," etc., in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that these numbers can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in an order other than that illustrated or described herein.

[0028] The functions described in the methods of this embodiment, if implemented as software functional units and sold or used as independent products, can be stored in a computing device-readable storage medium. Based on this understanding, portions of the embodiments of this application that benefit from the prior art or technical solutions can be embodied in the form of software products stored in the storage medium, including multiple instructions for causing a computing device (which may be a personal computer, server, mobile computing device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB drives, portable hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, etc., which are capable of storing program code.

[0029] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A network device, comprising: Network interface components, including: A plurality of physical ports, the plurality of physical ports coupling a plurality of servers to the network interface component, wherein data from virtual machines running on the plurality of servers coupled to the network interface component can transmit data to the physical ports and receive data from the physical ports; Multiple virtual network interface cards with logical ports running within the network interface, the logical ports being coupled to the virtual machines running on the server, wherein a single virtual machine running on a server can use one or more of the virtual interface cards, and wherein one of the virtual network interface cards can be used by one or more virtual machines running on a server. Multiple physical network interface cards (PNICs) provide physical resources for creating and supporting the functions of the multiple virtual network interface cards (VNICs), wherein a particular VNIC can access the functions of one or more of the multiple PNICs. The second plurality of physical ports couple the network interface component to the switch; A processor having an operating system that creates, deletes, and controls the operations of the plurality of virtual network interface cards and the physical network interface cards.

2. The network device according to claim 1, wherein the physical network interface resides outside the plurality of servers.

3. The network device according to claim 1, wherein the first plurality of physical ports of the network interface component are coupled to the plurality of servers via PCIe.

4. The network device according to claim 1, wherein the operating system is based on network element virtualization and standard virtual NIC software, and creates the virtual network interface as needed.

5. The network device according to claim 1, wherein the plurality of virtual network interfaces are coupled to the plurality of servers via the PCIe protocol.

6. The network device according to claim 1, wherein the physical network interface card acts as a network interface resource pool, and the network interface resource pool can be shared among the plurality of virtual network interface cards.

7. The network device of claim 6, wherein one of the physical network interface cards can be assigned by the operating system to support a specific virtual network interface card, and then reassigned by the operating system to support different virtual network interface cards.

8. The network device of claim 6, wherein one of the physical network interface cards can be allocated by the operating system to support multiple virtual network interface cards.

9. The network device according to claim 1, wherein multiple virtual network interface cards are merged to form a higher-level virtual network interface card.

10. The network device of claim 1, wherein one of the servers is capable of using multiple of the virtual network interface cards.

11. The network device of claim 1, wherein the virtual network interface card and the physical network interface card can be allocated and reallocated among the plurality of servers according to the service flow through the network.

12. The network device of claim 11, wherein a portion of the virtual network interface card is allocated to a second server, the second server requiring additional bandwidth than the first server.

13. The network device of claim 1, wherein the plurality of virtual machines of the plurality of servers are able to transmit and receive data to each other through the network interface component without going through a switch.

14. The network device of claim 1, wherein the network interface component enables the plurality of servers to use a plurality of pooled physical NICs.

15. The network device according to claim 1, wherein the network interface component is customized for each server using the plurality of virtual network interface cards.

16. The network device of claim 1, wherein one of the virtual machines is coupled to one of the virtual network interface cards via a bridge, and east-west data flow between the plurality of servers passes through the network interface component without passing through a separate external switch.

17. A network comprising: Multiple servers, wherein the multiple servers have virtual machines for processing data; A network interface device coupled to the plurality of servers, wherein the network interface device includes a plurality of virtual NICs and a plurality of physical NICs, wherein the plurality of virtual NICs interface with the plurality of virtual machines, and the plurality of physical NICs are supplied to support the plurality of virtual NICs, and the plurality of physical NICs can be programmed to adaptively support one or more of the virtual NICs. A switch, coupled to the physical NIC, switches data services to and from the plurality of servers.

18. The network of claim 17, wherein the plurality of virtual NICs and the plurality of physical NICs provide a virtual endpoint pool that can be shared among the plurality of servers.

19. The network of claim 18, wherein the virtual endpoint pool comprises up to 1024 virtual endpoints.

20. The network of claim 17, wherein the network interface device provides a network interface resource pool, the network interface resource pool being allocated and reallocated among the plurality of servers according to the data service flows corresponding to the plurality of servers.

21. The network of claim 17, wherein the east-west data flow between the plurality of servers passes directly through the network interface device.

22. A method for transmitting data over a cloud computing network, comprising: Create the first virtual machine to run on the first server; Create a second virtual machine to run on the first server; Transmit data from the first virtual machine to the first virtual network interface; Receive data from the first virtual network interface to the first virtual machine; Transmit data from the second virtual machine interface to the second virtual network interface; Receive data from the second virtual network interface to the second virtual machine; The first virtual network interface and the second virtual network interface are created from a pool of multiple physical network interface cards, wherein the physical network interface cards are shared among the multiple virtual network interfaces.

23. The method of claim 22, further comprising: Multiple virtual endpoints are created and managed based on data flow conditions through the cloud computing network.

24. The method of claim 23, further comprising: Data is exchanged between the multiple virtual endpoints via the PCIe protocol.

25. The method of claim 22, further comprising: Create a third and a fourth virtual machine to run on the second server; Transmit data from the third virtual machine to the first virtual network interface; Receive data from the first virtual network interface to the third virtual machine; Transmit data from the fourth virtual machine to the second virtual network interface; Receive data from the second virtual network interface to the fourth virtual machine.

26. The method of claim 25, further comprising: Based on the service flow to and from the first virtual machine, the second virtual machine, the third virtual machine, and the fourth virtual machine, the pool of the plurality of network interface cards is allocated to the first virtual network interface and the second virtual network interface.

27. The method of claim 22, further comprising: Create a third virtual network interface for transmitting and receiving data from one or more virtual machines running on a third server; Network interface resources are allocated from the pool of the plurality of network interface cards to meet the data flow requirements of the virtual machine running on the third server.

Images