Hardware load balancer gateway on commodity switch hardware

Abstract

A top-of-rack (TOR) switch operating with the Open Networking Software in the Cloud (SONiC) platform is implemented using commodity switch hardware and is configured to maintain or retrieve flow state information for incoming data flows in a load balancer. In one embodiment, an application-specific integrated circuit (ASIC) notifies a user mode container of flow state information for each incoming data flow. The user mode container notifies the ASIC of any affected flows that may be generated by a modified distributed system (e.g., a server that is added, updated, or removed). In other embodiments, the ASIC may utilize remote direct memory access (RDMA) to retrieve flow state information maintained by a remote device, or may utilize RDMA to retrieve network address translation (NAT) information for incoming traffic. In each of the implementations, the integrity of the load balancing system is maintained when the distributed system of servers changes.

Description

Background Art

[0001] Servers set up as part of a distributed system can host data such as dynamic web applications and websites, and a load balancer can direct incoming traffic to these servers. In some implementations, since consistent hashing is used to organize data on the servers, the flow of data from the load balancer to the servers may be affected when servers are added, updated, or removed from the distributed system. In addition, the bottleneck of the load balancer may occur at the multiplexer (MUX), which handles inbound network traffic and maps the virtual Internet Protocol (IP) addresses exposed to the public to the dynamic IP addresses where the virtual machines are located. Summary of the invention

[0002] A top-of-rack (TOR) switch operating with the Open Networking Software in the Cloud (SONiC) platform is implemented using commodity switch hardware and is configured to maintain or retrieve flow state information for incoming data flows to a load balancer that is configured to perform load balancing at the switch line rate. In one embodiment, an application-specific integrated circuit (ASIC) notifies a user mode container of flow state information for each incoming data flow. The user mode container notifies the ASIC of any affected flows that may be generated regarding a modified distributed system (e.g., a server that is added, updated, or removed). In other embodiments, the ASIC may utilize remote direct memory access (RDMA) to retrieve flow state information maintained by a remote device, or may utilize RDMA to retrieve network address translation (NAT) information for incoming traffic. In each of the implementations, the integrity of the load balancing system is maintained when the distributed system of the server changes.

[0003] This summary is provided to introduce in simplified form a series of concepts that will be further described in the detailed description below. This summary is not intended to identify the key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. In addition, the claimed subject matter is not limited to implementations that address any or all of the shortcomings noted in any part of this disclosure. It should be understood that the above-mentioned subject matter can be implemented as a computer-controlled device, a computer process, a computing system, or as an article such as one or more computer-readable storage media. These and various other features will be apparent by reading the following detailed description and reviewing the associated drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 An illustrative environment is shown in which incoming network traffic is processed by a top-of-rack (TOR) switch;

[0005] Figure 2An illustrative environment is shown in which an application specific integrated circuit (ASIC) communicates with at least one user mode container and a stream cache;

[0006] Figure 3 An illustrative environment is shown in which an ASIC checks for flow state information in a flow cache;

[0007] Figure 4 An illustrative environment is shown in which an ASIC determines flow state information and notifies a user mode container of the flow state information;

[0008] Figure 5 An illustrative environment for a consistent hashing implementation in a data center is shown;

[0009] Figure 6 An illustrative environment showing an updated consistent hashing implementation after adding a server to a data center;

[0010] Figure 7 An illustrative environment is shown in which a user-mode container transmits information related to an affected data flow to a flow cache of an ASIC;

[0011] Fig. 8A An illustrative environment is shown in which an ASIC uses remote direct memory access (RDMA) to a memory resource to check a flow table;

[0012] Figure 8B An illustrative environment is shown in which a remote switch uses RDMA to manage flow information to an ASIC of a TOR switch;

[0013] Fig. 9 An illustrative environment is shown in which an ASIC uses RDMA to a remote device to check a network address translation (NAT) table;

[0014] Figure 10-12 is a flow chart of an illustrative method performed by one or more computer servers within a data center;

[0015] Fig.13 is a block diagram of an illustrative data center that may be used, at least in part, to implement the present hardware load balancer gateway on commodity switch hardware;

[0016] Fig.14 is a simplified block diagram of an illustrative architecture of a computing device that may be used, at least in part, to implement the present hardware load balancer gateway on commodity switch hardware; and

[0017] Fig.15is a simplified block diagram of an illustrative remote computing device, remote service, or computer system that can be used in part to implement the present hardware load balancer gateway on commodity switch hardware.

[0018] Like reference numerals in the drawings indicate like elements. Unless otherwise noted, elements are not drawn to scale. DETAILED DESCRIPTION

[0019] SONiC (Software Open Networking in the Cloud) networking switch software is built on the Switch Abstraction Interface (SAI), which defines a standardized application programming interface (API). Network load balancing is implemented using programmable commodity switch hardware that leverages SAI to provide a consistent interface to hardware ASICs (Application Specific Integrated Circuits) that typically have a small memory footprint. Rather than implementing load balancing in dedicated servers or edge routers, SONiC is adapted to provide load balancing as a containerized TOR (Top of Rack) switch by managing flow state transformations as a user-mode function for newly affected flows routed by ASICs in TOR switches.

[0020] As new network nodes come online, a hash-based distribution model is used to map new flows to available servers. However, since the hardware load balancer ASIC has reduced memory resources compared to the SONiC TOR switch (which can run on a relatively resource-rich platform), the flow state for the newly affected flows is maintained by the SONiC user mode. This flow state maintenance enables a low-cost / overhead solution to extend the capabilities of commodity hardware by transferring some load balancing responsibilities to the SONiC TOR switch. When the DIP (destination IP) pool changes, for example due to demand elasticity or network maintenance, the SONiC TOR switch enables commodity load balancing hardware to handle new flows. Additional SONiC containers can also be added in the same chassis / rack to provide some scalability for load balancing needs.

[0021] In a highly scalable alternative implementation, SONiC is configured to support RDMA (Remote Direct Memory Access) to provide access to external memory resources suitable for storing computed hash tables for distribution patterns and / or flow states that can be pushed down to commodity load balancing hardware. ASIC memory can then be utilized for caching. NAT gateway functionality can also be supported in RDMA implementations by storing NAT flows and tables in external memory resources.

[0022] Turning now to the accompanying drawings Figure 1, computing device 105 may transmit network traffic 120 such as data packets 125 to data center 130, which is composed of at least host devices 140 and virtual machines 145. Rack 110 may include at least a top-of-rack (TOR) switch 115, a server, and a software load balancing (SLB) multiplexer (MUX) 150, which is configured to load balance incoming network traffic to the host devices. The TOR switch may use a hardware load balancer configured to perform equal cost multi-path routing (ECMP) 165 of incoming data packets to the host devices. Network 155 may be composed of any one or more of a local area network, a wide area network, the Internet, and the World Wide Web.

[0023] SLB is utilized to evenly distribute tenant and tenant customer network traffic among virtual network resources using software defined networking (SDN). SLB maps a virtual IP address (VIP) to a dynamic IP address (DIP) 170 that is part of a cloud service resource set in a data center 130, which may be a public IP address 160 exposed to a client computing device 105. In a typical implementation, a VIP is a single IP address that provides public access to a pool of load-balanced virtual machines 145. For example, a VIP is an IP address exposed on the Internet so that tenants and tenant customers operating computing devices 105 can connect to tenant resources in a cloud data center. DIP 170 is the IP address of a member virtual machine of the load-balanced pool behind the VIP. DIPs are assigned to tenant resources within the cloud infrastructure.

[0024] Figure 1 An illustrative environment is shown in which a computing device 105 transmits a data packet 125 destined for a host device 140 in a data center 130 and the data packet 125 is intercepted by a TOR switch 115. The data packet is transmitted through a data flow 205 ( Figure 2 ) traverse, the data flow 205 may be, for example, a TCP / IP (Transmission Control Protocol / Internet Protocol) connection. Although a TOR switch is depicted in the figures for clarity of illustration, the switch may be placed in various locations relative to the servers on which it operates. For example, the switch may be an end-of-row (EOR) switch or placed in the middle or at another location within the rack 110. In addition, the switch may be a router or other networking / gateway device having at least some of the functionality described herein for maintaining or accessing and retrieving flow state information. For example, a TOR switch may include commodity hardware operable to perform hardware load balancing and configured as Figure 2 Abstractly shown in FIG.

[0025] Figure 2A simplified layered architecture of components of a TOR switch 115, such as a line card or backplane, is shown, wherein the components include at least a hardware layer 210, a switch abstraction interface (SAI) layer 215, and a network application layer 220. Although not shown, in a typical implementation, other layered components such as a kernel space for an ASIC driver and a user space for an ASIC SDK (software development kit) may be utilized with the TOR switch 115.

[0026] The hardware layer may include an application specific integrated circuit (ASIC) 230 and a flow cache 225, which may be, for example, an L1 or L2 cache associated with the ASIC. The flow cache may store, for example, flow state information, including source IP, source port, destination IP, destination port, and the protocol used. In general, the flow state information may identify a data flow for incoming network services. The ASIC may be configured as a hardware load balancer on a TOR switch to calculate or otherwise determine how to distribute data packets to virtual machines on a host device, such as using consistent hashing as discussed in more detail below. The ASIC is also configured to perform ECMP 165, which also enables the execution of load balancing tasks that may typically be performed by a software load balancer (e.g., SLB MUX) to be transferred to the TOR switch 115. This may save resources for the SLB MUX and thereby increase its bandwidth and throughput.

[0027] The switch abstraction interface layer 215 may provide a standard interface that enables integration and disaggregation between commodity hardware and commodity software components and the network application layer 220, such as containers 235 and user mode containers 240, as representatively shown by reference numeral 245. The SAI may include a set of application programming interfaces (APIs) to support communication between containers and network operating systems and other network applications on commodity hardware.

[0028] Networking applications may operate using a network operating system (OS) 255, such as a 's Open Networking Software in the Cloud (SONiC) platform. In addition to the ASIC hardware on which the network applications operate, the TOR switch 115 can also utilize other hardware components. For example, the TOR switch 115 can utilize a separate processor, such as a CPU (central processing unit) and a hardware-based memory device (e.g., random access memory (RAM) or other flash memory) to perform application and operating system functions associated with the network application layer 220. The flow cache 225 and any other memory associated with the ASIC 230 are typically configured with a small memory footprint to perform discrete operations. Additional hardware used for containers and user mode applications enables the performance of additional system operations related to the hardware load balancing functionality of the ASIC.

[0029] The network application layer 220 can be instantiated on the network OS 255 and includes a container 235, which can include network applications and management tools that provide users with control over the operation of commodity-based networked devices. The user mode container 240 can maintain flow state information received from the ASIC so that the TOR switch 115 remains aware of data flow forwarding operations and information. The ASIC 230 is configured to communicate with the user mode container to transmit the flow state information, as illustratively shown by the double arrows.

[0030] Figure 3 An illustrative environment is shown in which TOR switch 115 receives data packet 125 via data flow 205. ASIC 230 receives the data packet and checks flow cache 225 for flow state information corresponding to input data flow 205. If corresponding data flow information exists, the ASIC routes the data according to the flow state information contained in the flow cache.

[0031] Figure 4 An illustrative environment is shown in which the ASIC 230 performs some operations in response to no existing or applicable flow state information being available within the flow cache 225. The ASIC uses a consistent hashing method to determine to which host device and virtual machine to forward the transmission data packet, as representatively shown at reference numeral 410, and may further utilize ECMP to route the data packet. Shifting some of the gateway functionality from other components in the SLB environment, such as the SLB MUX, to the ASIC may reduce some of the workload on the SLB MUX and thus alleviate its bottleneck, thereby improving overall system performance.

[0032] Consistent hashing is a distributed hashing method that utilizes an abstract circle with 360° to allow servers and objects to scale without affecting the overall system. The consistent hashing principle operates independently of the number of servers or objects in the distributed hash table, thereby achieving elasticity to server demand or maintenance without affecting the entire distributed system. The ASIC is configured to perform consistent hashing to route data packets to the correct virtual machines and host devices.

[0033] The ASIC communicates with the user mode container using the SAI 215. The ASIC synchronously transmits the determined flow state 405 for the data flow 205 to the user mode container 240. The user mode container stores the received determined flow state information in a memory device associated with the network application layer 220, thereby maintaining a list of flow state information for incoming flows.

[0034] Figure 5An abstraction of a server 505 and a data stream 510 on a circle having 360° and an illustrative environment of a data center 130 is shown, the circle representing a consistent hashing implementation 515 for data objects. In a typical implementation, based on the arrangement of the data streams on the circle, the data streams or objects on the circle belong to the data center 130 ( Figure 1 ). For example, a data stream or object might belong to the first server encountered when tracing counterclockwise from the data stream, but other configurations and uses of consistent hashing methods are possible.

[0035] Figure 6 An illustrative environment is shown in which data center 130 adds server 610, which affects the consistent hashing calculation of at least a portion of the data flow. Figure 5 , the addition of server 610 affects data flow 605 because the added server is now the first server encountered by the affected data flow when tracing counterclockwise on the circle. According to consistent hashing, the affected data flow belongs to the added server 610, not the previously used server 615. The affected data flow or objects may therefore cause data storage problems when components of the system (e.g., servers and networking devices) determine where data objects belong. If the TOR switch continues to send data to the previously used server when the updated consistent hash indicates that certain data objects belong to the added server, data errors may occur, which may damage the integrity of the system. Although in Figure 6 , but other modifications to the distributed system may also affect the data flow when using consistent hashing, such as removing or updating a server, temporarily taking a server offline, or providing maintenance to a server.

[0036] Figure 7 240 is shown in which the user mode container 240 is aware of the distributed system ( Figure 6 ) and identifies an illustrative environment of the affected flows 715. The user mode container of the TOR switch 115 can be notified of changes to the distributed system based on input from a user with remote control over the switch operation. For example, a system administrator can notify all devices in the system of a data center change. The user mode container can perform consistent hashing and use the flow state information ( Figure 4 ) to identify the affected flow status. That is, the user mode container can calculate a consistent hash for the updated system and identify which flows received from the ASIC are affected by the updated system.

[0037] Because only a portion of the flows are affected after the modification to the distributed system, the user mode container pushes information 705 identifying the affected flow states to the flow cache 225. Given the small memory footprint typically associated with an ASIC, it is desirable to store the affected flow states rather than all possible flow states. Figure 3 and Figure 4 ), the ASIC checks the flow cache 225 for each incoming data flow before it is sent to the server, allowing the ASIC to correctly route the data flow and adapt to the elasticity of demand or server maintenance.

[0038] Fig. 8A An alternative embodiment is shown in which the ASIC 230 utilizes remote direct memory access (RDMA) 810 to an external memory 815 of a remote computing device 820 to check the flow table 805. Utilization of external memory resources can provide increased scalability because the external source does not face hardware (e.g., memory) limitations like the TOR switch. The flow table can store relevant information for multiple data flows (affected or unaffected) to enable the ASIC to direct the data flow to the correct host device each time a new TCP / IP connection is established. The remote computing device 820 can keep the memory 815 up to date to reflect any modifications (e.g., additions, updates, removals) to the distributed system. If there is no stored or applicable flow state information for the data flow, the ASIC determines the route for the data flow, such as using consistent hashing.

[0039] By allowing applications on the TOR switch 115 to read data on the memory of a remote server or device with minimal demand on memory bus bandwidth and CPU processing overhead, the utilization of the RDMA protocol enables the removal of data copy operations and enables reduced latency while preserving memory protection semantics. The RDMA protocol is described in the RFC 5040 specification published by the Internet Engineering Task Force (IETF) and is built on the Direct Data Placement (DDP) protocol described in the RFC 5041 specification.

[0040] Figure 8BAn illustrative environment is shown in which a remotely instantiated user mode container 240 is configured to remotely control a local TOR switch 115, as representatively shown at reference numeral 860. The remote user mode container may be instantiated at a remote TOR switch 850 operating externally, such as within a remote data center 855. The remote TOR switch may utilize RDMA 810 to transmit flow state information 865, which may include current flow information and affected flow information, to the flow cache 225 for utilization by the ASIC 230. Remote implementation may improve scalability and performance by preventing bottleneck scenarios that may occur in local CPU scenarios.

[0041] Fig. 9 An alternative embodiment is shown in which the ASIC 230 utilizes RDMA to the external memory 815 of the remote computing device 820 to check the network address translation (NAT) table 905 to perform NAT functionality. The use of external memory enables the ASIC to store and access large amounts of information to perform NAT routing. NAT can be used to retrieve private IP addresses and other information for each virtual machine to which incoming flows can be distributed. The ASIC can use RDMA to retrieve this information from the external memory to route data packets. Since changes to the data center configuration (e.g., adding or removing servers) may affect the NAT table, the use of external memory resources enables the ASIC to maintain and access information to appropriately route data packets. The use of RDMA for flow tables or NAT tables can advantageously provide larger memory resources to the TOR switch 115, thereby preventing bad actors from potentially exploiting the small amount of memory footprint typically associated with the ASIC.

[0042] Figure 10-12 1000, 1100, and 1200 are flow charts of illustrative methods 1000, 1100, and 1200 that can be performed using a networking device such as a TOR switch 115 or a router. Unless otherwise specified, the methods or steps shown in the flow charts and described in the accompanying text are not limited to a particular order or sequence. In addition, some of these methods or steps may occur or be performed simultaneously, and not all methods or steps must be performed in a given implementation, depending on the requirements of such implementation, and some methods or steps may be optionally utilized.

[0043] In step 1005, Fig.10In step 1010, the networking device uses a load balancer associated with the networking device to load balance the data packets. In step 1015, the networking device checks whether flow state information for the data packets is stored in a memory associated with the networking device. In step 1020, depending on whether the flow state information is in the memory, the networking device calculates where to transmit the data packets or uses the flow state information to transmit the data packets.

[0044] In step 1105, Fig.11 In step 1110, the networking device receives a data packet destined for a host device. In step 1115, the networking device uses the determination of the host device to which the received data packet is to be forwarded to notify a user mode container of an application container of a flow state associated with the data packet, wherein the user mode container includes the flow state in a flow state table.

[0045] In step 1205, Fig.12 In step 1210, an application specific integrated circuit (ASIC) associated with a networking device intercepts a data packet before the data packet reaches a software load balancing (SLB) multiplexer (MUX). In step 1210, the ASIC uses remote direct memory access (RDMA) to query an external memory device associated with a remote computing device. In step 1215, the ASIC routes at least a portion of the intercepted data packet to a host device using the information queried from the external memory device.

[0046] Fig.13 1 is a high-level block diagram of an illustrative data center 1300 that provides cloud computing services or distributed computing services that can be used to implement the present hardware load balancer gateway on commodity switch hardware. Multiple servers 1301 are managed by a data center management controller 1302. Load balancer 1303 distributes requests and computing workloads to servers 1301 to avoid situations where a single server may become overwhelmed. Load balancer 1303 maximizes the available capacity and performance of resources in data center 1300. Routers / switches 1304 support data traffic between servers 1301 and between data center 1300 and external resources and users (not shown) via external network 1305, which can be, for example, a local area network (LAN) or the Internet.

[0047] The servers 1301 may be stand-alone computing devices, and / or they may be configured as individual blades in a rack of one or more server devices. The servers 1301 have input / output (I / O) connectors 1306 that manage communications with other database entities. One or more host processors 1307 on each server 1301 run a host operating system (O / S) 1308 that supports multiple virtual machines (VMs) 1309. Each VM 1309 may run its own O / S, such that each VM O / S 1310 on a server may be different, the same, or a mixture of the two. The VM O / S 1310 may be, for example, different versions of the same O / S (e.g., running Additionally or alternatively, the VM O / S 1310 may be provided by different manufacturers (e.g., some VMs run different current and legacy versions of the operating system). operating system, while other VMs run Each VM 1309 may also run one or more applications (Apps) 1311. Each server 1301 also includes a storage device 1312 (e.g., a hard disk drive (HDD)) and a memory 1313 (e.g., RAM) 1307, which may be accessed and used by the host processor and the VM 1309 for storing software code, data, etc. In one embodiment, the VM 1309 may employ the data plane API disclosed herein.

[0048] Data center 1300 provides pooled resources on which customers can dynamically provision and scale applications as needed without adding servers or additional networking. This allows customers to obtain the computing resources they need without having to purchase, provision, and manage infrastructure on a per-application basis. Cloud computing data center 1300 allows customers to dynamically scale up or down resources to meet the current needs of their business. In addition, data center operators can provide customers with usage-based services so that customers only pay for the resources they use when they need to use them. For example, a customer may initially use server 1301. 1 A VM 1309 on the same server 1301 can be used to run their application 1311. When the demand for application 1311 increases, data center 1300 can run the same VM 1309 on the same server 1301 as needed. 1 and / or new server 1301 N Additional VMs 1309 may be activated at a later time. If the demand for the application decreases later, these additional VMs 1309 may be deactivated.

[0049] Data center 1300 can provide guaranteed availability, disaster recovery, and backup services. For example, the data center can 1One VM 1309 on the server is designated as the primary location for the customer application, and a second VM 1309 on the same or different server can be activated as a standby or backup in case the first VM or server 1301 1 Failure occurs. The data center management controller 1302 automatically transfers incoming user requests from the primary VM to the backup VM without customer intervention. Although the data center 1300 is illustrated as a single location, it should be understood that the servers 1301 can be distributed to multiple locations around the world to provide additional redundancy and disaster recovery capabilities. In addition, the data center 1300 can be a resident private system that provides services to a single enterprise user, or it can be a publicly accessible distributed system that provides services to multiple unrelated customers, or it can be a combination of both.

[0050] Domain Name System (DNS) servers 1314 resolve domain names and host names into IP (Internet Protocol) addresses for all roles, applications, and services in data center 1300. DNS logs 1315 maintain records of which domain names have been resolved by role. It should be understood that DNS is used as an example herein, and other name resolution services and domain name record services may be used to identify dependencies.

[0051] Datacenter health monitoring 1316 monitors the health of the physical systems, software, and environment in datacenter 1300. Health monitoring 1316 provides feedback to datacenter management when problems are detected with servers, blades, processors, or applications in datacenter 1300, or when network bandwidth or communication issues arise.

[0052] Fig.1414 is a simplified block diagram of an illustrative computer system 1400, such as a PC or server, with which the present hardware load balancer gateway on commodity switch hardware can be implemented. The computer system 1400 includes a processor 1405, a system memory 1411, and a system bus 1414 that couples various system components including the system memory 1411 to the processor 1405. The system bus 1414 can be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, or a local bus using any of a variety of bus architectures. The system memory 1411 includes a read-only memory (ROM) 1417 and a random access memory (RAM) 1421. A basic input / output system (BIOS) 1425 containing basic routines such as those that help transfer information between elements within the computer system 1400 during startup is stored in the ROM 1417. The computer system 1400 may also include a hard disk drive 1428 for reading and writing to an internally disposed hard disk (not shown), a disk drive 1430 for reading or writing to a removable disk 1433 (e.g., a floppy disk), and an optical drive 1438 for reading or writing to a removable optical disk 1443 (such as a CD (compact disk), a DVD (digital versatile disk), or other optical media). The hard disk drive 1428, the disk drive 1430, and the optical drive 1438 are connected to the system bus 1414 via a hard disk drive interface 1446, a disk drive interface 1449, and an optical drive interface 1452, respectively. The drives and their associated computer-readable storage media provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computer system 1400. Although this illustrative example includes a hard disk, a removable disk 1433, and a removable optical disk 1443, other types of computer-readable storage media that can store computer-accessible data, such as magnetic tapes, flash memory cards, digital video disks, data cartridges, random access memory (RAM), read-only memory (ROM), etc., may also be used in some applications of the present hardware load balancer gateway on commodity switch hardware. In addition, as used herein, the term computer-readable storage medium includes one or more instances of a media type (e.g., one or more disks, one or more CDs, etc.). For purposes of this specification and claims, the phrase "computer-readable storage medium" and its variants are intended to cover non-transient embodiments and do not include waves, signals, and / or other transient and / or intangible communication media.

[0053] Many program modules including operating system 1455, one or more application programs 1457, other program modules 1460, and program data 1463 may be stored on a hard disk, disk 1433, optical disk 1443, ROM 1417, or RAM 1421. A user may enter commands and information into computer system 1400 through input devices such as keyboard 1466 and pointing devices 1468 such as a mouse. Other input devices (not shown) may include microphones, joysticks, game controllers, satellite dishes, scanners, trackballs, touch pads, touch screens, touch-sensitive devices, voice command modules or devices, user motion or user gesture capture devices, and the like. These and other input devices are often connected to processor 1405 via a serial port interface 1471 coupled to system bus 1414, but may also be connected via other interfaces, such as a parallel port, a game port, or a universal serial bus (USB). A monitor 1473 or other type of display device is also connected to system bus 1414 via an interface such as a video adapter 1475. In addition to the monitor 1473, personal computers typically include other peripheral output devices (not shown) such as speakers and printers. Fig.14 The illustrative example shown in FIG. 14 also includes a host adapter 1478 , a small computer system interface (SCSI) bus 1483 , and an external storage device 1476 connected to the SCSI bus 1483 .

[0054] The computer system 1400 is operable in a networked environment using logical connections to one or more remote computers, such as remote computer 1488. Remote computer 1488 may be selected as another personal computer, a server, a router, a network PC, a peer device, or other common network node, and typically includes many or all of the elements described above with respect to computer system 1400, although Fig.14 Only a single representative remote memory / storage device 1490 is shown. Fig.14 The logical connections depicted in 1490 include a local area network (LAN) 1493 and a wide area network (WAN) 1495. Such networking environments are commonly deployed, for example, in offices, enterprise-wide computer networks, intranets, and the Internet.

[0055] When used in a LAN networking environment, the computer system 1400 is connected to the local area network 1493 through a network interface or adapter 1496. When used in a WAN networking environment, the computer system 1400 typically includes a broadband modem 1498, a network gateway, or other components for establishing communications over the wide area network 1495 (e.g., the Internet). The broadband modem 1498, which may be internal or external, is connected to the system bus 1414 via the serial port interface 1471. In a networking environment, program modules related to the computer system 1400, or portions thereof, may be stored in the remote memory storage device 1490. It should be noted that Fig.14 The network connections shown in are illustrative, and other means of establishing a communication link between the computers may be used depending on the specific requirements of the application of the present hardware load balancer gateway on commodity switch hardware.

[0056] Fig.15 An illustrative architecture 1500 is shown for a computing device, such as a laptop or personal computer, that a customer can use to communicate with a data center of the present hardware load balancer gateway on commodity switch hardware. Fig.15 The illustrated architecture 1500 includes one or more processors 1502 (e.g., central processing units, dedicated artificial intelligence chips, graphics processing units, etc.), system memory 1504 including RAM (random access memory) 1506 and ROM (read only memory) 1508, and a system bus 1510 that operatively and functionally couples the components in the architecture 1500. A basic input / output system containing basic routines that help transfer information between elements within the architecture 1500 (such as during startup) is typically stored in the ROM 1508. The architecture 1500 also includes a mass storage device 1512 for storing software code or other computer executable code that is utilized to implement applications, file systems, and operating systems. The mass storage device 1512 is connected to the processor 1502 through a mass storage controller (not shown) connected to the bus 1510. The mass storage device 1512 and its associated computer-readable storage media provide non-volatile storage for the architecture 1500. Although the descriptions of computer-readable storage media contained herein refer to mass storage devices, such as a hard disk or a CD-ROM drive, those skilled in the art will appreciate that computer-readable storage media may be any available storage media that may be accessed by architecture 1500 .

[0057] By way of example and not limitation, computer-readable storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. For example, computer-readable media include, but are not limited to, RAM, ROM, EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), flash memory or other solid-state memory technology, CD-ROM, DVD, HD-DVD (high-definition DVD), Blu-ray or other optical storage, cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by the architecture 1500.

[0058] According to various embodiments, the architecture 1500 can operate in a networked environment using logical connections to remote computers through a network. The architecture 1500 can be connected to the network through a network interface unit 1516 connected to the bus 1510. It can be appreciated that the network interface unit 1516 can also be utilized to connect to other types of networks and remote computer systems. The architecture 1500 can also include an input / output controller 1518 for receiving and processing input from a variety of other devices, including a keyboard, a mouse, a touch pad, a touch screen, a control device such as buttons and switches or an electronic stylus ( Fig.15 Similarly, the input / output controller 1518 can provide signals to a display screen, user interface, printer, or other type of output device (not shown in FIG. Fig.15 (not shown) provides an output.

[0059] It can be appreciated that the software components described herein can transform the processor 1502 and the overall architecture 1500 from a general-purpose computing system into a special-purpose computing system customized to support the functionality presented herein when loaded into the processor 1502 and executed. The processor 1502 can be constructed from any number of transistors or other discrete circuit elements, which can individually or collectively display any number of states. More specifically, the processor 1502 can operate as a finite state machine in response to executable instructions contained in the software modules disclosed herein. These computer executable instructions can transform the processor 1502 by specifying how the processor 1502 transitions between states, thereby transforming the transistors or other discrete hardware elements that constitute the processor 1502.

[0060] The software module encoding presented herein can also transform the physical structure of the computer-readable storage medium presented herein. The specific transformation of the physical structure may depend on various factors in the different implementations of this specification. Examples of such factors may include, but are not limited to, the technology used to implement the computer-readable storage medium, whether the computer-readable storage medium is characterized as a primary storage device or a secondary storage device, etc. For example, if the computer-readable storage medium is implemented as a semiconductor-based memory, the software disclosed herein can be encoded on the computer-readable storage medium by transforming the physical state of the semiconductor memory. For example, the software can transform the state of the transistors, capacitors, or other discrete circuit elements that constitute the semiconductor memory. The software can also transform the physical state of such components in order to store data thereon.

[0061] In another example, the computer-readable storage medium disclosed herein can be implemented using magnetic or optical technology. In such an implementation, when the software presented herein is encoded in a magnetic or optical medium, the software can transform the physical state of the magnetic or optical medium. These transformations can include changing the magnetic properties of a specific location in a given magnetic medium. These transformations can also include changing the physical characteristics or characteristics of a specific location in a given optical medium to change the optical properties of those locations. Without departing from the scope and spirit of this description, other transformations of physical media are possible, and the aforementioned examples are provided just for ease of discussion.

[0062] The architecture 1500 may also include one or more sensors 1514 or a battery or power source 1520. Sensors may be coupled to the architecture to pick up data about the environment or components, including temperature, pressure, etc. Exemplary sensors may include thermometers, accelerometers, smoke or gas sensors, pressure sensors (barometric or physical), light sensors, ultrasonic sensors, gyroscopes, etc. The power source may be adapted with an AC power cord or a battery, such as a rechargeable battery for portability.

[0063] In view of the above, it can be appreciated that many types of physical transformations occur in architecture 1500 in order to store and execute the software components presented herein. It can also be appreciated that architecture 1500 can include other types of computing devices, including wearable devices, handheld computers, embedded computer systems, smart phones, PDAs, and other types of computing devices known to those skilled in the art. It is also contemplated that architecture 1500 may not include Fig.15 All components shown in the components can include Fig.15 Other components not explicitly shown in the drawings or may be used in conjunction with Fig.15 A completely different architecture than the one shown in .

[0064] Various exemplary embodiments of the present hardware load balancer gateway on commodity switch hardware are now presented by way of illustration and not as an exhaustive list of all embodiments. One example includes a method performed by a networking device associated with a load balancer operating on one or more servers, wherein the networking device employs a SONiC (Software for Open Networking in the Cloud) environment on a commodity hardware device, comprising: forwarding a data packet using a load balancer associated with the networking device; intercepting a data packet on a flow path toward the load balancer at the networking device; checking, by the networking device, whether flow state information for the data packet exists in a memory associated with the networking device; depending on whether the flow state information exists in the memory: in response to the flow state information not being present in the memory, calculating where to transmit the data packet; and in response to the flow state information being present in the memory, using the flow state information to transmit the data packet.

[0065] In another example, the networking device includes an ASIC (application-specific integrated circuit) that checks its cache for flow state information. As another example, the ASIC also uses RDMA (remote data memory access) to access flow state information stored on a memory device that is remotely operated for the networking device, wherein the memory device that supports RDMA stores flow state information for each load balancer. In a further example, the method also includes: checking the memory device that supports RDMA after checking the cache of the ASIC. In another example, the networking device uses RDMA (remote data memory access) to access a NAT (network address translation) table stored on a memory device that is remotely operated for the networking device, wherein the memory device that supports RDMA stores a NAT table for each host device and virtual machine. In another example, the networking device includes an ASIC (application-specific integrated circuit) that intercepts data packets. As a further example, the method also includes: synchronizing the flow state information calculated by the ASIC with a user mode container instantiated on the networking device, the user mode container being depolymerized with the ASIC so that the user mode container is operable by a hardware component different from the ASIC. As a further example, the method further includes: receiving data center configuration information at a user mode container indicating that a server within the data center has been updated, removed from a group of servers, or added to a group of servers; and determining which flow states within a hash table are affected within a networking device based on the data center configuration information; and pushing the determined affected flow states from the user mode container to a memory associated with the ASIC. In another example, the user mode container interfaces with a switch abstraction interface (SAI), and the SAI interfaces with a component associated with the ASIC. In another example, the networking device is a switch or a router. In a further example, the method further includes: performing equal-cost multipath routing on the intercepted data packets by the networking device.

[0066] Another exemplary embodiment includes a networking device configured to utilize flow state information to manage forwarding of incoming data traffic affected by a dynamically sized data center, comprising: one or more processors; one or more non-transitory hardware-based memory devices associated with the one or more processors, the memory devices storing computer-readable instructions for operating an application container on the networking device; an application-specific integrated circuit (ASIC); and one or more caches associated with the ASIC, wherein the ASIC is configured to: receive a data packet destined for a host device; determine to which host device the received data packet is to be forwarded; and use the decision as to which host device the received data packet is to be forwarded to notify a user mode container of the application container of a flow state associated with the data packet, wherein the user mode container stores the flow state in a flow state table.

[0067] In a further example, the determination of which host device to forward the received data packet to is performed using a consistent hashing method. In another example, the ASIC is further configured to check a cache for information describing an affected data stream, and wherein the determination of which host device to forward the received data packet to is performed in response to the absence of the affected data stream in the cache. In another example, the networking device receives a change to a data center at a user mode container, the change comprising removing or adding a server within the data center; determining the affected data streams based on the data center change; and transmitting information describing the affected data streams to a cache associated with the ASIC. As another example, the determination of which data streams are affected by the data center change is performed using a consistent hashing method.

[0068] Another example includes one or more hardware-based non-transitory computer-readable memory devices storing instructions that, when executed by one or more processors disposed in a networking device associated with a load balancer, cause the networking device to: intercept data packets at an application-specific integrated circuit (ASIC) associated with the networking device before the data packets reach a load-balancing multiplexer (MUX), wherein the load-balancing MUX transmits the data packets to a virtual machine instantiated on a host device; query, by the ASIC, an external memory device associated with a remote computing device using remote direct memory access (RDMA), wherein the external memory hosts information that can be used by the networking device to route the intercepted data packets to the host device; and route at least a portion of the intercepted data packets to the host device using the queried information from the external memory device.

[0069] In another example, the information hosted on the external storage device is flow state information contained in a hash table. In a further example, the information is network address translation (NAT) information for a virtual machine on the host device. As a further example, the ASIC further utilizes consistent hashing to calculate where to route a portion of the intercepted data packet.

[0070] Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method performed by a networking device including an application specific integrated circuit (ASIC), the networking device being associated with a load balancer operating on one or more servers, wherein the networking device employs an open networking software SONiC environment in the cloud on a commodity hardware device, the method comprising: forwarding data packets using the load balancer associated with the networking device; intercepting, at the ASIC in the networking device, data packets on a flow path toward the load balancer; determining, by the ASIC in the networking device, whether flow state information for the data packet exists in a memory associated with the networking device; Depends on whether the stream state information exists in the memory: In response to the flow state information not being present in the memory, calculating where to transmit the data packet; In response to the flow state information being present in the memory, transmitting the data packet using the flow state information; synchronizing the flow state information for the data packet with a user mode container instantiated on the networking device, the user mode container being deaggregated from the ASIC such that the user mode container is operable by a hardware component distinct from the ASIC; as well as Pushing flow state from the user mode container to a memory associated with the ASIC, the user mode container determining that the flow state is affected based on a configuration of the one or more servers.

2. The method of claim 1, wherein the networking device comprises an application specific integrated circuit (ASIC) that checks its cache for the flow state information.

3. The method of claim 2, wherein the ASIC further uses remote direct memory access (RDMA) to access flow state information stored on a memory device that is remote to the networking device, wherein the memory device that supports the RDMA stores flow state information for each load balancer. 4 . The method of claim 3 , further comprising checking the RDMA-capable memory device after checking the cache of the ASIC.

5. The method of claim 2, wherein the networked device uses remote direct memory access (RDMA) to access a network address translation (NAT) table stored on a memory device that is remote to the networked device, wherein the memory device that supports the RDMA stores the NAT table for each host device and virtual machine.

6. The method according to claim 1, further comprising: receiving, at the user mode container, a configuration of the one or more servers, the configuration of the one or more servers indicating that a server within a data center has been updated, removed from a group of servers, or added to a group of servers; A determination is made based on the configuration of the one or more servers as to which flow states within the hash table are affected within the networking device. 7 . The method of claim 1 , wherein the user mode container interfaces with a switch abstraction interface (SAI), and the SAI interfaces with components associated with the ASIC. The method of claim 1 , wherein the networking device is a switch or a router.

9. The method of claim 1, further comprising performing, by the networking device, equal-cost multi-path routing on the intercepted data packets.

10. The method of claim 6, wherein the hash table is implemented independent of the number of servers utilized in the data center.

11. The method of claim 1 , further comprising using the ASIC to perform equal-cost multi-path routing when transmitting the data packet.

12. A networking device configured to utilize flow state information to manage forwarding of incoming data traffic subject to a dynamically resized data center, the networking device comprising: one or more processors; one or more non-transitory hardware-based memory devices associated with the one or more processors, the one or more non-transitory hardware-based memory devices storing computer-readable instructions for operating an application container on the network-connected device; Application-specific integrated circuit ASIC; as well as one or more caches associated with the ASIC, wherein the ASIC is configured to: receiving a data packet destined for a host device; using flow state information for the data packet present in the external memory device to determine to which host device the received data packet is to be forwarded; notifying a user mode container of the application container of the flow state information for the received data packet, wherein the user mode container stores the flow state information in a flow state table; as well as synchronizing the flow state information for the data packets with the user mode container, the user mode container being deaggregated from the ASIC so that the user mode container is operable by a hardware component different from the ASIC, and Pushing flow state from the user mode container to the one or more caches associated with the ASIC, the user mode container determining that the flow state is affected based on a configuration of the dynamically sizing data center.

13. The networking device of claim 12, wherein the determination of which host to forward the received data packet to is performed using a consistent hashing method.

14. The networking device of claim 13, wherein the ASIC is further configured to check the cache for information describing an affected data flow, and wherein the determining to which host device the received data packet is to be forwarded is performed in response to the absence of the affected data flow in the cache.

15. The networking device of claim 12, further comprising: receiving, at the user mode container, a change to a data center, the change comprising removing or adding a server within the data center; determining an affected data stream based on the data center change; as well as Information describing the affected data flow is transmitted to the cache associated with the ASIC.

16. The networking device of claim 15, wherein the determination of which data flows are affected by the data center change is performed using a consistent hashing method.

17. One or more hardware-based non-transitory computer-readable memory devices storing instructions that, when executed by one or more processors disposed in a networking device associated with a load balancer, cause the networking device to: intercepting the data packet at an application specific integrated circuit ASIC associated with the networking device before the data packet reaches a load balancing multiplexer, wherein the load balancing multiplexer transmits the data packet to a virtual machine instantiated on a host device; querying, by the ASIC, an external memory device associated with a remote computing device using remote direct memory access (RDMA), wherein the external memory device hosts flow state information that can be used by the networking device to route the intercepted data packet to a host device; as well as routing at least a portion of the intercepted data packets to a host device using the flow state information queried from the external memory device; synchronizing the flow state information of the intercepted data packet with a user mode container instantiated on the networking device, the user mode container being deaggregated from the ASIC such that the user mode container is operable by a hardware component distinct from the ASIC; as well as A flow state is pushed from the user mode container to a memory associated with the ASIC, the user mode container determining that the flow state is affected based on a configuration of the host device.

18. The one or more hardware-based non-transitory computer-readable memory devices of claim 17, wherein the information hosted on the external memory device is flow state information contained within a hash table.

19. The one or more hardware-based non-transitory computer-readable memory devices of claim 17, wherein the external memory device also hosts network address translation (NAT) information for a virtual machine on the host device.

20. The one or more hardware-based non-transitory computer-readable memory devices of claim 17, wherein the ASIC further utilizes consistent hashing to calculate where to route a portion of the intercepted data packet.

Images