Smart network interface cards and arrangements for reducing routing requirements in rack systems

Abstract

The disclosure relates to an intelligent network interface card and arrangement for reducing wiring requirements in a rack system. A computing device has a number of smart network interface cards (NICs) with RJ45 connectors. The intelligent NIC is arranged in a master-slave mode, wherein the internal RJ45 connector of the master intelligent NIC is connected to the corresponding RJ45 connector of the slave intelligent NIC through an RJ45 cable. A front RJ45 connector of the main smart NIC is connected to a network switch mounted in the same rack as the computing device through an RJ45 cable.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to computing devices. BACKGROUND

[0002] Data centers and other computing facilities have multiple computing devices, such as server computers, storage units, etc., installed in racks. The racks can include so-called Top of Rack (TOR) switches, which are network switches that connect to network interface cards (NICs) of the computing devices in the rack. Despite their name, TOR switches can be installed at the top of the rack, at the bottom of the rack, or between racks. The TOR switches act as a central connection point, allowing the computing devices in the rack to communicate with each other and with external networks.

[0003] A rack can have one or more TOR switches. For example, the NICs of server computers in the rack can connect to a TOR switch, while the NICs of storage units in the rack can connect to another TOR switch. A rack can also have a single TOR switch to which all of the NICs in the rack connect. The NICs connect to the TOR switches through network interface cables, which are typically run along the front of the rack. Examples of these network interface cables include fiber optic cables that connect to small form-factor pluggable (SFP) modules and RJ45 cables that connect to RJ45 connectors. The number of network interface cables in a rack not only increases cost, but also creates operational problems in terms of maintenance, airflow, identification, etc. SUMMARY

[0004] In one embodiment, a rack system includes a rack, a top-of-rack (TOR) switch housed in the rack, and a computing device. The computing device includes a chassis housed in the rack, a motherboard in the chassis, a master intelligent network interface card (NIC) removably plugged into a corresponding bus slot of the motherboard, the master intelligent NIC including a small form-factor pluggable (SFP) module, an Ethernet switch, a front RJ45 connector connected to the Ethernet switch, and a plurality of internal RJ45 connectors connected to the Ethernet switch, and a plurality of slave intelligent NICs removably plugged into corresponding bus slots of the motherboard, each of the slave intelligent NICs including an SFP module and an RJ45 connector. The rack system further includes a first RJ45 cable connecting the front RJ45 connector of the master intelligent NIC to the TOR switch, a second RJ45 cable connecting a first internal RJ45 connector of the plurality of internal RJ45 connectors of the master intelligent NIC to an RJ45 connector of a first slave intelligent NIC of the plurality of slave intelligent NICs, and a third RJ45 cable connecting a second internal RJ45 connector of the plurality of internal RJ45 connectors of the master intelligent NIC to an RJ45 connector of a second slave intelligent NIC of the plurality of slave intelligent NICs.

[0005] In another embodiment, a computing device includes a chassis, a motherboard in the chassis, the motherboard including a plurality of bus slots and at least one processor, a master intelligent NIC removably plugged into a corresponding bus slot of the plurality of bus slots, the master intelligent NIC including an SFP module, an Ethernet switch, a front RJ45 connector connected to the Ethernet switch, and a plurality of internal RJ45 connectors connected to the Ethernet switch, and a plurality of slave intelligent NICs removably plugged into corresponding bus slots of the plurality of bus slots, each of the slave intelligent NICs including an SFP module and an RJ45 connector, wherein a first RJ45 cable connects a first internal RJ45 connector of the plurality of internal RJ45 connectors of the master intelligent NIC to an RJ45 connector of a first slave intelligent NIC of the plurality of slave intelligent NICs, and a second RJ45 cable connects a second internal RJ45 connector of the plurality of internal RJ45 connectors of the master intelligent NIC to an RJ45 connector of a second slave intelligent NIC of the plurality of slave intelligent NICs.

[0006] In yet another embodiment, a smart network interface card (NIC) comprises: a small form-factor pluggable (SFP) module; an Ethernet switch; a front RJ45 connector connected to the Ethernet switch, wherein the front RJ45 connector is mounted on a circuit board facing a front of the smart NIC; at least one internal RJ45 connector connected to the Ethernet switch, wherein the at least one internal RJ45 connector is mounted on a circuit board facing a back of the smart NIC or a top of the smart NIC; and a bus connector on the circuit board, the bus connector configured to removably connect to a bus slot of a motherboard.

[0007] These and other features of the present disclosure will be apparent to those skilled in the art from the following disclosure, including the appended claims and accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0008] A more complete understanding of the subject matter can be obtained by referring to the following detailed description and claims, considered in connection with the accompanying drawings, in which like reference numbers refer to like elements throughout. The drawings are not intended to be drawn to scale.

[0009] Figure 1 An electrical configuration of a computing device according to an embodiment of the present invention is schematically illustrated.

[0010] Figure 2 A physical configuration of a computing device according to an embodiment of the present invention is schematically illustrated.

[0011] Figure 3 A rack system according to an embodiment of the present invention is shown.

[0012] Figure 4 A side view of a smart NIC according to an embodiment of the present invention is shown.

[0013] Figure 5 A side view of a smart NIC according to another embodiment of the present invention is shown.

[0014] Figure 6 A side view of a smart NIC according to yet another embodiment of the present invention is shown.

[0015] Figure 7 Multiple smart NICs in a master-slave arrangement according to an embodiment of the present invention are shown.

[0016] Figure 8 Multiple smart NICs in a master-slave arrangement according to another embodiment of the present invention are shown.

[0017] Figure 9 Multiple smart NICs in a master-slave arrangement according to yet another embodiment of the present invention are shown. DETAILED DESCRIPTION

[0018] In the disclosure, numerous specific details are provided, such as examples of components, structures, and methods, to provide a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application can be practiced without one or more of the specific details. In other instances, well-known details are not shown or described in order not to obscure aspects of the application.

[0019] Figure 1 An electrical configuration of a computing device 100 according to embodiments of the application is schematically illustrated. The computing device 100 can comprise a server computer, a storage unit, or other network-connected device that can removably accept a smart network interface card (NIC) 110. For illustration clarity, only two computing devices 100 are shown. In practice, a data center can have multiple racks, with each rack housing multiple computing devices 100.

[0020] The computing device 100 includes at least one processor, memory, and other electrical components. The computing device 100 further includes an input / output (I / O) bus, which in one embodiment is a Peripheral Component Interconnect Express (PCIe) bus. The computing device 100 has multiple PCIe bus slots, each slot accepting an I / O card, such as a smart NIC 110. In Figure 1 In the example, for illustration clarity, only two smart NICs 110 are shown. However, four, eight, etc. smart NICs 110 can be connected to the I / O bus of the computing device 100.

[0021] Each smart NIC 110 has a network interface controller 113 connected to a main network 130 and a network interface controller 112 connected to a management network 120. For security reasons, the main network 130 and the management network 120 are separate computer networks.

[0022] The main network 130 is a high-speed Ethernet network for normal network communications. The network interface controller 113 can be a network interface controller for a small form-factor pluggable (SFP) module such as a quad small form-factor pluggable (QSFP) module or an octal small form-factor pluggable (OSFP) module (not shown in the example). A fiber optic cable connects the SFP module to the main network 130. For example, the fiber optic cable can be compatible with a multi-fiber termination push-on (MTP) and multi-fiber push-on (MPO) connector or a Lucent (LC) connector. Figure 1

[0023] ​Management network 120 is an Ethernet network used to communicate with smart NICs 110 for configuration, troubleshooting, or other management related operations. Network interface controllers 112 can be a baseboard management controller (BMC), a microcontroller, or other circuitry that interfaces with management network 120 through an RJ45 connector Figure 1 Although all controllers 112 on computing device 100 can be connected to management network 120, only one of the RJ45 connectors needs to be connected to a TOR switch through an RJ45 cable. This is because, as will be made clearer in the following, the RJ45 cable connection can be routed from other smart NICs in computing device 100 to smart NIC 110.

[0024] Figure 2 A physical configuration of computing device 100 according to embodiments of the present application is schematically illustrated. Computing device 100 includes a chassis 150, which in one embodiment is rack mountable. Housed within chassis 150 is a motherboard 152 having a plurality of PCIe bus slots 151. Each smart NIC 110 is removably connected to a bus slot 151. Depending on the type of computing device 100, installed on motherboard 152 are at least one processor 154, memory 153, and other electronic components.

[0025] Figure 3 A rack system according to embodiments of the present application is shown. In Figure 3 the rack system includes a rack 200, at least one TOR switch 210, and a plurality of computing devices 100. Rack 200 can be a 19-inch rack commonly used in data centers. For example, rack 200 can be a 19-inch 48U rack. Chassis 150 of computing devices 100 are installed in rack 200. For example, computing devices 100 can be 4U server computers, meaning that chassis 150 of the server computers occupy 4 rack units of rack 200.

[0026] In Figure 3 the TOR switch 210 is an Ethernet switch installed at the top of rack 200. As previously noted, TOR switch 210 need not be housed at the top of the rack. For illustration clarity, only one TOR switch 210 is shown in Figure 3 Typically, rack 200 can have a plurality of TOR switches 210, one for each type of computing device housed in rack 200. For example, rack 200 can have a first TOR switch 210 for all storage units, a second TOR switch 210 for all server computers, etc.

[0027] Figure 3The front of the rack 200 is shown. In a typical data center, the network interface cables to the TOR switches 210 are routed in the front of the rack 200. As an example, Figure 3 Shown are RJ45 cables 201 (e.g., twisted pair cables) connecting the RJ45 connectors of the smart NICs 110 of the computing device 100 to corresponding connectors on the TOR switch 210, and fiber optic cables 202 connecting the SFP modules of the smart NICs 110 to corresponding connectors on the TOR switch 210. The number of network interface cables from the computing device 100 to the TOR switch 210 depends on the number of smart NICs 110 in the computing device 100.

[0028] Figure 4 A side view of a smart NIC 110A according to an embodiment of the present invention is shown. Smart NIC 110A is a specific embodiment of smart NIC 110. Smart NIC 110A includes circuit board 304 and bus connector 303. Smart NIC 110A has a half-length, half-height form factor.

[0029] In one embodiment, the bus connector 303 is a PCIe bus connector that is connected to the bus slot 151 ( Figure 2 100) is detachably connected to the PCIe bus of the computing device 100. The smart NIC 110A includes an SFP module 302, which may be, for example, a QSFP or OSFP module. The smart NIC 110A further includes a plurality of RJ45 connectors connected to the multi-port Ethernet switch 310. The RJ45 connector located on the front of the smart NIC 110 (i.e., facing the front of the smart NIC 110 / rack 200) is labeled 321, and the RJ45 connector located on the back of the smart NIC 110 (i.e., facing the back of the rack 200 / smart NIC 110) is labeled 322. Figure 4 In the example of FIG, smart NIC 110A has a single RJ45 connector 321 on the front and four RJ45 connectors 322 on the back that connect to Ethernet switch 310. Ethernet switch 310 may be a conventional multi-port Ethernet switch integrated circuit (IC), such as those available from Realtek Semiconductor Corp.

[0030] As will become more apparent in the following, the RJ45 connectors of the smart NIC 110 of the compute device 100 can be connected in a master-slave arrangement to minimize the number of RJ45 cables from the compute device 100 to the corresponding TOR switch 210. In the arrangement, the front RJ45 connectors of the master smart NIC 110 can be wired to the TOR switch 210, and the RJ45 connectors of the slave smart NIC 110 can be wired to other RJ45 connectors (e.g., rear RJ45 connectors or top RJ45 connectors) of the master smart NIC 110. The Ethernet switch 310 switches network traffic received at the other RJ45 connectors of the master smart NIC 110 to the front RJ45 connectors of the master NIC 110, thereby allowing the received network traffic to reach the TOR switch 210. In addition, the Ethernet switch 310 switches network traffic received at the front RJ45 connectors of the master smart NIC 110 to the RJ45 connectors of the master NIC 110 designated to receive the network traffic.

[0031] Figure 5 A side view of a smart NIC 110B according to an embodiment of the present application is shown. The smart NIC 110B is a particular embodiment of the smart NIC 110. The smart NIC 110B is the same as the smart NIC 110A except that the smart NIC 110B has a circuit board 401 and additional RJ45 connectors 323 at the top of the circuit board 401 (i.e., facing the top of the rack 200 / smart NIC 110B). In Figure 5 In an example, the smart NIC 110B has a single RJ45 connector 321 in the front, four RJ45 connectors 322 in the back, and four RJ45 connectors 323 at the top. The RJ45 connectors (e.g., the RJ45 connectors 322 and 323) that accept RJ45 cables from inside the chassis 150 of the compute device 100 are also referred to herein as internal RJ45 connectors.

[0032] The smart NIC 110B has a half-length full-height form factor and thus has more circuit board space than the smart NIC 110A. This allows the smart NIC 110B to accommodate the RJ45 connectors 321 and the RJ45 connectors 322 in addition to the RJ45 connectors 323 at the top of the circuit board 401. The RJ45 connectors 323 are connected to the Ethernet switch 310. The smart NIC 110B is otherwise the same as the smart NIC 110A.

[0033] Figure 6A side view of a smart NIC 110C is shown according to an embodiment of the present invention. Smart NIC 110C is a specific embodiment of smart NIC 110. Smart NIC 110C is identical to smart NIC 110A, except that smart NIC 110C has circuit board 501 and fewer RJ45 connectors. Figure 6 In the example of FIG, the smart NIC 110C has a single RJ45 connector 321 on the front and a single RJ45 connector 322 on the back.

[0034] Like the smart NIC 110A, the smart NIC 110C has a half-length, half-height form factor. However, unlike the smart NIC 110A, the smart NIC 110C has only two RJ45 connectors. However, this is not a problem and can actually be cost-effective when used as a slave device in a master-slave arrangement. The smart NIC 110C is otherwise identical to the smart NIC 110A.

[0035] Figure 7 Multiple smart NICs are shown in a master-slave arrangement according to an embodiment of the present invention. Figure 7 In the example of FIG, in a master-slave arrangement, multiple smart NICs 600 are configured as slave devices and smart NIC 110A is configured as a master device. Smart NICs 600 can be conventional smart NICs that do not have an internal RJ45 connector and an Ethernet switch. In one embodiment, master smart NIC 110A and slave smart NICs 600 are all plugged into corresponding bus slots 151 of the motherboard 152 of the computing device 100.

[0036] exist Figure 7 In the example shown, RJ45 cable 601 connects the front RJ45 connector of the primary smart NIC 110A (see Figure 4 , RJ45 connector 321) to the TOR switch 210. The RJ45 cable connects the internal RJ45 connector of the primary smart NIC 110A (see Figure 4 , RJ45 connector 322) to the front RJ45 connector of the corresponding slave smart NIC 600. More specifically, RJ45 cables 602 to 605 connect the rear RJ45 connector of the master smart NIC 110A to the corresponding RJ45 connector of the slave smart NIC 600. Fiber optic cables connect the SFP modules of the master smart NIC 110A and the slave smart NIC 600 to the TOR switch 210.

[0037] Figure 8 A plurality of intelligent NICs in a master-slave arrangement according to another embodiment of the present invention is shown. Figure 8In an example, in a master-slave arrangement, multiple smart NICs 110C are configured as slave devices and smart NIC 110A is configured as a master device. In one embodiment, the master smart NIC 110A and the slave smart NIC 110C are all plugged into corresponding bus slots 151 of the motherboard 152 of the computing device 100. The use of smart NICs 110C allows RJ45 cables that are not plugged into the TOR switch 210 to remain within the chassis 150 of the computing device 100.

[0038] exist Figure 8 In the example of FIG, RJ45 cable 701 connects the front RJ45 connector of the master smart NIC 110A to the TOR switch 210. The RJ45 cable connects the internal RJ45 connector of the master smart NIC 110A to the rear RJ45 connector of the corresponding slave smart NIC 110C (see FIG. Figure 6 , RJ45 connector 322). More specifically, RJ45 cables 702-705 connect the rear RJ45 connector of the master smart NIC 110A to the corresponding rear RJ45 connector of the slave smart NIC 110C. Fiber optic cables connect the SFP modules of the master smart NIC 110A and the slave smart NIC 110C to the TOR switch 210.

[0039] Figure 9 A plurality of intelligent NICs in a master-slave arrangement according to another embodiment of the present invention is shown. Figure 9 In the example of FIG, in a master-slave arrangement, multiple smart NICs 110C are configured as slave devices and smart NIC 110B is configured as a master device. Since smart NIC 110B may have eight internal RJ45 connectors (see FIG. Figure 5 , RJ45 connector 323 and RJ45 connector 322), so Figure 9 The master-slave arrangement can accommodate eight slave smart NICs. In one embodiment, the master smart NIC 110B and the slave smart NIC 110C are all plugged into corresponding bus slots 151 of the motherboard 152 of the computing device 100. The use of the smart NIC 110C allows the RJ45 cables that are not plugged into the TOR switch 210 to remain within the chassis 150 of the computing device 100.

[0040] exist Figure 9In the example, RJ45 cable 801 connects the front RJ45 connector of the master intelligent NIC 110B to the TOR switch 210. The RJ45 cable connects the internal RJ45 connector of the master intelligent NIC 110B to the corresponding back RJ45 connector of the slave intelligent NIC 110C. More specifically, RJ45 cables 802-809 connect the internal RJ45 connector of the master intelligent NIC 110B to the corresponding back RJ45 connectors of the slave intelligent NIC 110C. Fiber optic cables connect the SFP modules of the master intelligent NIC 110B and the slave intelligent NIC 110C to the TOR switch 210.

[0041] As can be appreciated, other master-slave arrangements involving the intelligent NICs 110 are possible. For example, depending on the number of slave intelligent NICs, any of the intelligent NICs 110A, 110B, or 110C can be used as the master intelligent NIC. Similarly, any of the intelligent NICs 110A, 110B, or 110C can be used as the slave device. Furthermore, the number of intelligent NICs 110 and / or the number of RJ45 connectors switched within the intelligent NIC 110 can be increased to increase the number of intelligent NICs in the computing device 100.

[0042] Embodiments of the present invention advantageously reduce the number of RJ45 cables that extend out of the computing device. As a particular example, assume that the computing device has 4 intelligent NICs. In the example, 4 RJ45 cables and 8 fiber optic cables are needed to connect the computing device to the TOR switch in a conventional manner. In comparison, for embodiments of the present invention, the computing device would only need 1 RJ45 cable for the front RJ45 connector and 8 fiber optic cables for the SFP modules to connect the computing device to the TOR switch. As additional intelligent NICs are installed in the computing device and as additional computing devices are installed in the rack (e.g., in artificial intelligence (AI) / machine learning applications), the reduction in the number of RJ45 cables to connect to the TOR switch becomes more significant.

[0043] While specific embodiments of the present invention have been provided, it is to be understood that these embodiments are for illustration purposes only and are not limiting. Those skilled in the art reading the present disclosure will appreciate many additional embodiments.

Claims

1. A rack system comprising: a rack; a top-of-rack switch mounted in the rack; a computing device separate from the top-of-rack switch, the computing device comprising: a chassis mounted in the rack; a motherboard in the chassis, the motherboard comprising a plurality of bus slots and at least one processor; a master intelligent network interface card removably inserted into a corresponding bus slot of the plurality of bus slots, the master intelligent network interface card comprising a small form-factor pluggable module, an Ethernet switch, a front RJ45 connector connected to the Ethernet switch, and a plurality of internal RJ45 connectors connected to the Ethernet switch; and a plurality of slave intelligent network interface cards removably inserted into a corresponding bus slot of the plurality of bus slots, each of the slave intelligent network interface cards comprising a small form-factor pluggable module and an RJ45 connector; a first RJ45 cable connecting the front RJ45 connector of the master intelligent network interface card to the top-of-rack switch; a second RJ45 cable connecting a first internal RJ45 connector of the plurality of internal RJ45 connectors of the master intelligent network interface card to an RJ45 connector of a first slave intelligent network interface card of the plurality of slave intelligent network interface cards; and a third RJ45 cable connecting a second internal RJ45 connector of the plurality of internal RJ45 connectors of the master intelligent network interface card to an RJ45 connector of a second slave intelligent network interface card of the plurality of slave intelligent network interface cards.

2. The rack system of claim 1, wherein the second RJ45 cable and the third RJ45 cable are disposed within the chassis of the computing device.

3. The rack system of claim 1, wherein the small form-factor pluggable modules of the master intelligent network interface card and the plurality of slave intelligent network interface cards are quad small form-factor pluggable modules or octal small form-factor pluggable modules.

4. The rack system of claim 1, wherein the RJ45 connector of each of the plurality of slave intelligent network interface cards faces a rear of the rack.

5. The rack system of claim 1, wherein the computing device is a server computer.

6. The rack system of claim 1, wherein each of the plurality of bus slots is a peripheral component interface express bus slot.

7. A computing device comprising: a chassis; a motherboard in the chassis, the motherboard comprising a plurality of bus slots and at least one processor; a master intelligent network interface card removably inserted into a corresponding bus slot of the plurality of bus slots, the master intelligent network interface card comprising a small form-factor pluggable module, an Ethernet switch, a front RJ45 connector connected to the Ethernet switch, and a plurality of internal RJ45 connectors connected to the Ethernet switch; and a plurality of slave intelligent network interface cards removably inserted into a corresponding bus slot of the plurality of bus slots, each of the slave intelligent network interface cards comprising a small form-factor pluggable module and an RJ45 connector. a plurality of slave intelligent network interface cards removably inserted into corresponding ones of the plurality of bus slots, each of the slave intelligent network interface cards including a small form-factor pluggable module and an RJ45 connector, wherein a first RJ45 cable connects a first one of the plurality of internal RJ45 connectors of the master intelligent network interface card to an RJ45 connector of a first one of the plurality of slave intelligent network interface cards, and a second RJ45 cable connects a second one of the plurality of internal RJ45 connectors of the master intelligent network interface card to an RJ45 connector of a second one of the plurality of slave intelligent network interface cards.

8. The computing device of claim 7, wherein a third RJ45 cable connects the front RJ45 connector of the master intelligent network interface card to a top-of-rack switch mounted in the same rack as the computing device.

9. The computing device of claim 7, wherein the first and second RJ45 cables are disposed within the chassis of the computing device.

10. The computing device of claim 7, wherein the small form-factor pluggable modules of the master intelligent network interface card and the plurality of slave intelligent network interface cards are quad small form-factor pluggable modules or octal small form-factor pluggable modules.

11. The computing device of claim 7, wherein the RJ45 connector of each of the slave intelligent network interface cards faces a rear of the slave intelligent network interface card.

12. The computing device of claim 7, wherein the computing device is a server computer.

13. The computing device of claim 7, wherein each of the plurality of bus slots is a peripheral component interface express bus slot.

14. An intelligent network interface card, comprising: a small form-factor pluggable module; an Ethernet switch; a front RJ45 connector connected to the Ethernet switch, wherein the front RJ45 connector is mounted on a circuit board facing a front of the intelligent network interface card; at least one internal RJ45 connector connected to the Ethernet switch, wherein the at least one internal RJ45 connector is mounted on the circuit board facing a rear of the intelligent network interface card or facing a top of the intelligent network interface card; and a bus connector on the circuit board configured to be removably connected to a bus slot of a motherboard.

15. The intelligent network interface card of claim 14, wherein a first RJ45 cable connects the front RJ45 connector of the intelligent network interface card to a top-of-rack switch mounted in the same rack as a computing device that includes the intelligent network interface card.

16. The intelligent network interface card of claim 15, wherein a second RJ45 cable connects the at least one internal RJ45 connector of the intelligent network interface card to an RJ45 connector of another intelligent network interface card located in the same computing device as the intelligent network interface card.

17. The intelligent network interface card of claim 16, wherein each of the intelligent network interface card and the other intelligent network interface card are plugged into a peripheral component interface express bus slot of a motherboard of the same computing device.

18. The intelligent network interface card of claim 14, wherein the small form-factor pluggable module is a quad small form-factor pluggable module or an octal small form-factor pluggable module.

19. The intelligent network interface card of claim 14, wherein the at least one internal RJ45 connector is one of a plurality of internal RJ45 connectors mounted on the circuit board facing a back of the intelligent network interface card or facing a top of the intelligent network interface card.

Images