Software-defined network operations for programmable connectivity devices

Abstract

A virtual network including virtual machines executing at a computing environment is implemented. A software defined network (SDN) appliance is configured to provide connectivity to computing resources via a virtual network of the virtual computing environment. The SDN appliance is configured to apply policies of the virtual computing environment to data traffic on the virtual network. The SDN appliance is operable to interact with a plurality of network devices configured to act as hardware acceleration devices for processing the data traffic. Virtual addresses are assigned to the network devices. The SDN appliance executes a virtual switch configured to identify data traffic sent to or received from the hosts and act as a proxy for the network devices and respond on their behalf.

Description

Background Technology

[0001] Data centers can house computer systems and various networks, storage, and other related components. For example, data centers are used by service providers to offer computing services to businesses and individuals as remote computing services or to provide "Software as a Service" (e.g., cloud computing). In some cases, such users may want to combine custom computing or storage assets with the cloud computing capabilities of a data center. However, custom assets may lack the ability to apply policies to user data. In some scenarios, cloud computing services may use a dedicated data plane injected into compute instances within a customer's virtual network to isolate data from other customers' data. However, security policies implemented on a customer's virtual network may compromise management plane connectivity to the injected compute instances, potentially disrupting resource deployment.

[0002] Taking these and other factors into account, this article presents the publicly available content. Summary of the Invention

[0003] Users of remote computing services can be offered the option to incorporate custom or third-party hardware, such as high-capacity processing machines or storage devices, into a virtual network within their cloud. For example, bare-metal servers can be deployed within the network. A bare-metal server can refer to a single-tenant physical server that is not shared among users. Such custom hardware and other hybrid cloud scenarios may require software-defined networking (SDN) policies to access cloud provider services. However, devices such as bare-metal servers may not possess all the functionalities of a host on SDN. For example, a bare-metal server may not be able to implement the host network stack.

[0004] The goal is to apply existing user policies to custom hardware without impacting the performance of the user's virtual machines within their virtual network. One way to achieve this is by programming network devices, such as top-of-rack (ToR) switches, to route traffic into the customer's virtual network. However, the options for programming network devices can be limited. For example, loading a software-defined network policy stack onto a network device may be impossible due to hardware constraints and limitations imposed by the network device manufacturer.

[0005] SDN policies typically cannot be applied to non-VM workloads. SDN is usually implemented on general-purpose compute nodes. The SDN control plane can be programmed to provide core networking functions such as security. For example, the network SDN stack can be coupled to the host. However, there may be no way to scale the host network stack. Therefore, capacity may be limited to the operations performed on a single host. Furthermore, traffic requiring additional SDN policies (such as private links) must be sent through several software hops. This can lead to performance degradation of network services, which can be an obstacle to migrating high-throughput workloads and applications to the cloud.

[0006] This document discloses systems and methods for achieving this integration of hardware services while maintaining a seamless user experience. In some embodiments, the above-described scenarios can benefit from implementations of systems and methods for decoupling policy processing from hosts. This decoupling also enables larger network scales to meet growing customer demands. In some embodiments, SDN can implement intermediate appliances, which may be referred to herein as SDN appliances. In some embodiments, as further described herein, SDN appliances can allow the use of an SDN control plane to manage network devices while providing high availability and fault tolerance. SDN appliances provide a model for decoupling the application of SDN policies and configurations into different computing environments. The operation of the appliances is transparent to the virtual network. Furthermore, SDN appliances offer an opportunity to amortize the computing environment over a greater number of virtual networks than previously possible.

[0007] In the example scenario shown, SDN capabilities can be enhanced by decoupling policy implementations from the host and moving them to SDN appliances strategically placed within the network. SDN appliances can be configured to implement SDN policies and execute associated transformation and load balancer policies. In some embodiments, to completely move host SDN policy implementations off-host, SDN appliances including FPGAs can be used to off-host SDN policy implementations in some implementations. The implementation of SDN appliances can free up workload / compute capacity for customer workloads and achieve more predictable performance. SDN appliances can be placed in the data center to dynamically provide any scale or feature that may be impossible or unavailable on the host. These features can include VM scaling, packet processing offloading, and flexible SDN policy application.

[0008] By incorporating SDN appliances, for example, when regional virtual network peering is established, bare metal servers can connect to virtual machines in another virtual network within the same region. Furthermore, outbound routing tables can be distributed on bare metal traffic. In this scenario, SDN appliances can provide connectivity to address prefixes of all regional peering virtual networks. Additionally, global peering can also be supported. This disclosure describes techniques for addressing various problems associated with the aforementioned scenarios.

[0009] Network devices with specific connections that include programmable ASICs (such as FPGAs) may lack a network stack capable of interacting with public network protocols (such as ARP and LACP) that are crucial for routing and monitoring. Such network devices may reside, for example, on servers containing network interfaces capable of accepting control traffic.

[0010] This disclosure describes embodiments for modifying an ARP framework running on a network device so that it can provide connectivity to programmable network devices that do not have a native Layer 3 network stack running on it (e.g., an FPGA). Such devices can share a public network interface on a server to bootstrap network connectivity (e.g., ARP). Agents and drivers on SDN appliances can be programmed and communicate with these connected devices to complete the bootstrap of network connectivity.

[0011] In this embodiment, a virtual IP address may be assigned to the FPGA. Additionally, in order to respond when a packet addresses the virtual IP address, a network driver running on the SDN appliance may act as a proxy for the FPGA and respond on its behalf.

[0012] More generally, virtual MAC and IP addresses can be assigned to connected devices, and an agent and a driver programmed by that agent can know the mapping of these connected devices to (virtual IP, virtual MAC) pairs, handle network boot requests, and send boot responses through these connected devices.

[0013] The described embodiments can support multiple FPGAs and other connected devices on SDN appliances in multiple different networks and physical topologies.

[0014] The described techniques enable virtual computing environments to support a variety of configurations, including custom hardware and hybrid architectures, while maintaining efficient use of computing resources such as processor cycles, memory, network bandwidth, and power. This summary is provided to present a selection of concepts in a simplified form, which will be further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that address any or all of the shortcomings pointed out in any part of this disclosure. Attached Figure Description

[0015] A detailed description is provided with reference to the accompanying drawings. In this detailed description, specific embodiments or examples are shown by way of illustration, with reference to the accompanying drawings, which form a part of this document. The drawings herein are not drawn to scale. In several figures, the same numbers denote the same elements.

[0016] Figure 1 This is a schematic diagram illustrating an example architecture according to this disclosure;

[0017] Figure 2 This is a schematic diagram illustrating a data center according to this disclosure;

[0018] Figure 3 This is a schematic diagram illustrating an example architecture according to this disclosure;

[0019] Figure 4 This is a schematic diagram illustrating an injection example according to this disclosure;

[0020] Figure 5 This is a schematic diagram illustrating an architecture for implementing virtual services according to this disclosure;

[0021] Figure 6 This is a schematic diagram illustrating an architecture for implementing virtual services according to this disclosure;

[0022] Figure 7 This is a schematic diagram illustrating an architecture for implementing virtual services according to this disclosure;

[0023] Figure 8 This is a schematic diagram illustrating the architecture according to this disclosure;

[0024] Figure 9A This is a schematic diagram illustrating the architecture according to this disclosure;

[0025] Figure 9B This is a schematic diagram illustrating the architecture according to this disclosure;

[0026] Figure 10 This is a schematic diagram illustrating the architecture according to this disclosure;

[0027] Figure 11 This is a schematic diagram illustrating an apparatus according to the present disclosure;

[0028] Figure 12 This is a schematic diagram illustrating an apparatus according to the present disclosure;

[0029] Figure 13 This is a schematic diagram illustrating an apparatus according to the present disclosure;

[0030] Figure 14 This is a schematic diagram illustrating an example virtual network according to this disclosure;

[0031] Figure 15 This is a schematic diagram illustrating an example virtual network according to this disclosure;

[0032] Figure 16 This is a schematic diagram illustrating an example virtual network according to this disclosure;

[0033] Figure 17 This is a schematic diagram illustrating the architecture according to this disclosure;

[0034] Figure 18 This is a schematic diagram illustrating the architecture according to this disclosure;

[0035] Figure 19 This is a schematic diagram illustrating the architecture according to this disclosure;

[0036] Figure 20 This is a schematic diagram illustrating the architecture according to this disclosure;

[0037] Figure 21 This is a schematic diagram illustrating the architecture according to this disclosure;

[0038] Figure 22 This is a schematic diagram illustrating the architecture according to this disclosure;

[0039] Figure 23 It is an example computing system based on this disclosure.

[0040] Figure 24 This is an example procedure based on this disclosure. Detailed Implementation

[0041] In some computing environments that provide virtual computing and storage services, various computing and networking services can be configured to allow users to combine a variety of configurations, including custom hardware and hybrid architectures, to extend the capabilities of the services. For example, an enterprise may want to deploy dedicated hardware (e.g., bare metal servers). Users of virtual computing resources can benefit in several ways by deploying resources such as virtual machines on dedicated resources. Consolidating dedicated hardware for all aspects of virtual computing services allows users to more effectively meet data residency, compliance, latency, and other requirements while continuing to benefit from the many advantages of leveraging remote and / or virtual computing services, such as scalability and flexibility.

[0042] The embodiments of this disclosure implement various scenarios for implementing dedicated hardware and other hybrid architectures. The embodiments of this disclosure also enable such extensions to be implemented in a way that reduces the cost and complexity of such networks, thereby allowing for more efficient use of computing, storage, and networking resources.

[0043] Cloud service providers can efficiently implement end-to-end capability services using a hybrid architecture, enabling a seamless and more consistent experience across regular and hybrid user footprints. By leveraging a comprehensive resource management approach to integrate multi-tenant and single-tenant resources, user overhead can be minimized, eliminating the need for users to handle policy enforcement issues and perform other complex management tasks. The efficient distribution of the described separation function can be determined based on its impact on various performance and security factors, such as latency and data security.

[0044] In some embodiments disclosed herein, a flexible network interface can be implemented. As used herein, such a flexible network interface may be referred to as a flexible network interface controller (NIC), a floating network interface controller, or an fNIC.

[0045] The fNIC associated with a virtual machine in a cloud computing network can be configured to elastically attach and detach from its parent NIC, enabling the virtual machine to connect simultaneously to multiple different virtual networks (VNets) and / or subnets associated with the same or different subscriptions. For example, an fNIC can allow a service provider to inject compute instances into an existing VNet, where the data plane uses a dedicated network interface to connect to the customer's VNet, while another dedicated network interface provides management plane connectivity to the service provider. This configuration provides data plane isolation for the customer's VNet to comply with applicable security policies without interrupting management traffic between the injected resources and the service provider. For example, using a cross-subscription architecture, the parent NIC can be associated with a service subscription used for management traffic to inject compute instances, and the attached fNIC can be associated with a customer subscription used for data traffic.

[0046] In addition to providing isolation between data and management traffic of injected compute instances, the use of fNICs offers additional flexibility to cloud computing customers and service providers. For example, compute instances can be simultaneously connected to different subnets within a customer's VNet (which may have different security policies). This capability provided by fNICs can facilitate the efficient organization and consumption of resources within a customer's enterprise.

[0047] The use of fNICs enables the implementation of multi-tenant architectures, providing access to a single shared VM by multiple tenants. Each fNIC attached to a parent NIC associated with a service provider can subscribe to a unique Network Partition Identifier (NPI) for each tenant. fNICs provide a flexible multi-tenant implementation, supporting granular network policies for specified discrete computing workloads, rather than across the entire VM. Virtual filtering platform extensions under the parent NIC can be configured to enforce specified network policies attached to each fNIC, including, for example, bandwidth metering, access control, VNet data encapsulation, and addressing. The data planes of each tenant on the VM can operate concurrently but isolated from each other to ensure that data processing by one tenant does not affect other tenants.

[0048] fNICs can also leverage associated policies to provide pre-provisioning of additional compute resources, which can be rapidly deployed on demand while reducing the time typically required to inject resources into a customer's VNet. In this "hot attach" architecture, for example, a service provider can have a pool of already active compute instances in a VM as a backup. The service can attach the fNIC to the parent NIC and associate it with the customer's subscription to provide access to the customer's VNet. The management plane and data plane operate independently to prevent disruption while adhering to applicable network and security policies.

[0049] Referring to the accompanying drawings, where the same numerals denote the same elements throughout several figures, aspects of various techniques used for network separation techniques and supporting techniques will be described. In the detailed description below, reference is made to the accompanying drawings, which form part of this document, and the drawings are illustrated by way of explaining specified configurations or examples.

[0050] Figure 1 An example computing environment in which the embodiments described herein can be implemented is shown. Figure 1A service provider 100 is illustrated, configured to provide computing resources to users at user site 140. User site 140 may have user computers that can access services provided by service provider 100 via network 130. The computing resources provided by service provider 100 may include various types of resources, such as computing resources, data storage resources, and data communication resources. For example, computing resources may be available as virtual machines. Virtual machines may be configured to run applications including web servers, application servers, media servers, and database servers. Data storage resources may include file storage devices, block storage devices, etc. Network resources may include virtual networks, software load balancers, etc.

[0051] Service provider 100 may have various computing resources, including servers, routers, and other devices that can provide remotely accessible computing and network resources, such as virtual machines. Other resources that may be provided include data storage resources. Service provider 100 may also perform functions such as managing and controlling network resource allocation, for example, a network manager 110.

[0052] For example, network 130 may be a publicly accessible network that links other networks and may be operated by various entities such as the Internet. In other embodiments, network 130 may be a private network, such as a dedicated network that is completely or partially inaccessible to the public. Network 130 may provide access to computers and other devices at user site 140.

[0053] Figure 2 An example computing environment in which the embodiments described herein can be implemented is shown. Figure 2A data center 200 is illustrated, configured to provide computing resources to users 200a, 200b, or 200c (which may be referred to as "user 200" individually or "multiple computers 202" in the case of multiple users) via a communication network 220 through user computers 202a, 202b, and 202c (which may be referred to as "computer 202" individually or "multiple users 200" in the case of multiple users). The computing resources provided by the data center 200 may include various types of resources, such as computing resources, data storage resources, data communication resources, etc. Each type of computing resource may be general-purpose or may be available in many specified configurations. For example, computing resources may be available as virtual machines. Virtual machines may be configured to execute applications including web servers, application servers, media servers, database servers, etc. Data storage resources may include file storage devices, block storage devices, etc. Each type or configuration of computing resources may be available in different configurations, such as the number of processors, the size of memory and / or storage capacity. In some embodiments, resources may be provided to clients in units called instances (e.g., virtual machine instances or storage instances). A virtual computing instance may be referred to as a virtual machine and may include, for example, one or more servers having specified computing capabilities (which may be specified by indicating the type and number of CPUs, main memory size, etc.) and specified software stacks (e.g., specified versions of operating systems that may run on top of a hypervisor).

[0054] Data Center 200 can correspond to Figure 1 and 2 Service provider 100 or Figure 2 Edge site 150. Data center 200 may include servers 226a, 226b, and 226c (which may be referred to as "server 226" individually or as "multiple servers 226" in the case of multiple servers), which may be standalone or installed in server racks, and provide available computing resources as virtual machines 228a and 228b (which may be referred to as "virtual machine 228" individually or as "multiple virtual machines 228" in the case of multiple servers). Virtual machine 228 may be configured to run applications such as web servers, application servers, media servers, database servers, etc. Other resources that may be provided include data storage resources ( Figure 2 (Not shown in the image), and may include file storage devices, block storage devices, etc. Server 226 may also perform functions such as managing and controlling resource allocation in the data center, for example, controller 225. Controller 225 may be another type of program that constructs a controller or is configured to manage the allocation of virtual machines on server 226.

[0055] refer to Figure 2 The communication network 220 may be, for example, a publicly accessible network of linked networks, and may be operated by various entities such as the Internet. In other embodiments, the communication network 220 may be a private network, such as a corporate network that is completely or partially inaccessible to the public.

[0056] Communication network 220 can provide access to computer 202. Computer 202 can be a computer used by user 200. Computers 202a, 202b, or 202c can be servers, desktop computers, laptops, tablets, smartphones, set-top boxes, or any other computing devices capable of accessing data center 200. User computer 202a or 202b can be directly connected to the Internet (e.g., via a cable modem). User computer 202c can be located within data center 200 and can be directly connected to resources within data center 200 via an internal network. Although only three user computers 202a, 202b, and 202c are depicted, it should be understood that multiple user computers can exist.

[0057] Computer 202 can also be used to configure aspects of the computing resources provided by data center 200. For example, data center 200 can provide a web interface through which various aspects of its operation can be configured using a web browser application running on user computer 202. Alternatively, a stand-alone application running on user computer 202 can be used to access application programming interfaces (APIs) exposed by data center 200 for performing configuration operations.

[0058] Server 226 can be configured to provide the aforementioned computing resources. One or more servers in server 226 can be configured to execute a manager 230a or 230b configured to execute virtual machines (which, in a single case, may be referred to as "manager 230", or in multiple cases, may be referred to as "multiple managers 230"). For example, manager 230 may be a virtual machine monitor (VMM), a structure controller, or another type of program configured to allow the execution of virtual machine 228 on server 226.

[0059] It should be understood that although the embodiments disclosed above are discussed in the context of a virtual machine, other types of implementations may be utilized using the concepts and techniques disclosed herein.

[0060] exist Figure 2In the example data center 200 shown, network device 222 can be used to interconnect servers 226a and 226b. Network device 222 may include one or more switches, routers, or other network devices. Network device 222 may also be connected to gateway 240, which is connected to communication network 220. Network device 222 can facilitate communication within the network in data center 200, for example, by forwarding packets or other data communications where appropriate, based on the characteristics of such communication (e.g., header information including source and / or destination addresses, protocol identifiers, etc.) and / or the characteristics of private networks (e.g., routing based on network topology, etc.). It should be understood that, for simplicity, various aspects of the computing systems and other devices in this example are shown, without showing specific general details. In other embodiments, additional computing systems and other devices may be interconnected, and may be interconnected in different ways.

[0061] It should be understood that Figure 2 The network topologies shown have been greatly simplified, and more networks and network devices can be used to interconnect the various computing systems disclosed herein. These network topologies and devices should be obvious to those skilled in the art.

[0062] It should also be understood that Figure 2 The data center 200 described herein is merely exemplary, and other implementations can be utilized. Additionally, it should be understood that the functionality disclosed herein can be implemented in software, hardware, or a combination of both. Other implementations will be apparent to those skilled in the art. It should also be understood that servers, gateways, or other computing devices may include any combination of hardware or software capable of interacting with and performing the functions of the type described, including, but not limited to, desktop computers or other computers, database servers, network storage devices and other network devices, handheld computers, tablet computers, smartphones, internet devices, television-based systems (e.g., using set-top boxes and / or personal / digital video recorders), and a variety of other consumer products including appropriate communication capabilities. Furthermore, in some embodiments, the functionality provided by the illustrated modules may be combined in fewer modules or distributed across additional modules. Similarly, in some embodiments, the functionality of some of the illustrated modules may not be provided and / or other available additional functionality may be provided.

[0063] Figure 3 An exemplary cloud computing environment 300 is illustrated, in which a customer network 305 comprises multiple parts, including an on-premises network 330 and a virtual network (VNet) 335. The customer network in this example is a hybrid network, but other network configurations can be utilized depending on the specific requirements of the user scenario. The VNet can be physically implemented using one or more master (machines) 320 operated by the cloud service provider 325. Note that, for clarity, Figure 3 The diagrams in the illustrations are simplified and typical network devices such as firewalls and routers are not shown.

[0064] On-premises networks and VNets are typically coupled via a communication network 335, which may include, for example, a private and / or public networking infrastructure using a combination of various connectivity services. A VNet may include multiple subnets 340, each subnet 340 comprising one or more instances of virtual machines 345 typically connected using a load balancer 350 and / or other network devices. Security network policies and other network policies (indicated by reference numeral 355) typically apply to each subnet. Network policies for each subnet are typically different, but may be the same and / or overlap in some cases.

[0065] Figure 4 An exemplary service from service provider 405 is illustrated, which injects a dedicated compute instance 410 into a customer's VNet 415. For example, in some implementations, such a service provider could be implemented using Platform as a Service (PaaS) to provide search, content delivery, etc. In some cases, the service provider could be associated with cloud service provider 405 or a third party. The service provider can inject compute instances or other resources into the VNet when providing a given cloud computing service that interacts with customer data traffic 415 entering and leaving gateway 330. Figure 4 As shown, the customer-implemented policy 455 to protect the VNet or subnet network is used to filter traffic and provide end-node control for all network traffic flows at the VM / VNet / subnet.

[0066] Network policies can block management traffic 440 by preventing management plane access from NRP 405, which would result in service interruption. Service interruption can be handled by the fNIC, which can be attached to and detached from the parent network interface controller fNIC in a resilient manner as described below. The separation of the management plane and data plane can be a feature of Software-Defined Networking (SDN). Therefore, as described below, in some implementations, a customer's network 405 can be entirely or partially based on SDN technology.

[0067] Figure 5An exemplary fNIC 505 is illustrated, which can be utilized to support various scenarios of attachment to a parent NIC 510. In this example, the parent NIC can be implemented using a container model to virtualize the network interface at the host supporting VM 345, although a physical implementation may be used in some scenarios. The fNIC can be implemented using container compartments as sub-network interface configurations. Essentially, compute instances can be created using placeholder network interfaces, allowing multiple fNICs to be dynamically placed and removed by being attached to and detached from the instance. This ability to attach and detach from the parent NIC provides flexibility for various VM deployment scenarios.

[0068] The parent NIC and fNIC provide identification, connectivity, and discoverability for VMs within a customer's VNet. The fNIC's ability to attach and detach from the parent NIC enables flexibility for various VM deployment scenarios. This flexibility allows for the rapid provisioning of various cloud computing features and services on demand, without altering the underlying workflow within a given VM / VNet / subnet while adhering to applicable network policies.

[0069] like Figure 5 As shown, exemplary use cases include, for example, cross-subscription and multi-VNet ownership (indicated by reference numeral 515), multi-tenancy and subnet sharing 520, and pre-provisioning or “hot-attaching” of resources 525.

[0070] Figure 6 An example Virtual Filtering Platform (VFP) 600 extension to VM switch 605 is shown, which enables data path isolation in the multitenant architecture discussed herein by enforcing specified network policies attached to each container 610. The VM switch may logically reside below a parent NIC 310 and may provide port 612 to each VM supported by the NIC. The VFP can layer network policies applicable to the ports, including rules governing SDN behavior and characteristics. The Virtual Filtering Platform can provide the ability to enforce policies and transform or tunnel data packets in a given compute workload entering and leaving VM 145. The Virtual Filtering Platform may include a central packet processor (not shown) that performs the processing of data packets.

[0071] In this example, the network policy layer may include those functions or features related to metering 615, access control lists (ACLs) 620, VNet addressing / routing 625, and various other SDN functions or features 630, which may include, for example, functions or features related to routing, tunneling, filtering, address translation, encryption, decryption, encapsulation, decapsulation, or Quality of Service (QoS). The packet processor in VFP 600 can evaluate packets of data traffic 635 as they pass through the network policy layer, performing actions in the previous layer and matching them to rules in each layer based on the packet's state. Returning packets may traverse the layer in the opposite direction and can be processed by the VFP to match applicable rules. Rules used to represent network policies may be entities that perform actions on matching packets (e.g., using a matching action table model) when the VFP processes the computational workload.

[0072] Figure 7 This is an exemplary layered cloud computing architecture 700, illustrating how the fNIC enables network functions to be decoupled from the provisioning of computing resources. As shown in the typical workflow illustrated by dashed line 705, the network between the service layer 710 and the host layer 715 is typically coupled to the provisioning of computing resources from a computing resource provider 720. In this example, the computing resource provider may be instantiated in a zone control layer 725, which can be utilized in some cloud computing scenarios involving multiple virtual networks. The computing resource provider interfaces with a computing agent 730 in the host layer, which in turn interfaces with a network agent 735 to complete the workflow.

[0073] Compared to the standard workflow 705, such as Figure 7 As shown by solid line 740, the workflow available to the fNIC enables the service layer 710 to implement network functions for management and other traffic independent of computing resources. As discussed herein, this independence allows for multi-VNet ownership, cross-subscription, hot-attachment, and other scenarios by flexibly attaching the fNIC to a parent NIC. Here, the resource manager 745 can interact with the network resource provider 750 to establish an fNIC that operates in conjunction with the network agent 735 in the host layer 715, thereby enabling computing resource-independent network functions.

[0074] refer to Figure 8 The diagram shows a bare metal server 830 that can route packets to a destination 850 via a routing table applied to a delegated subnet by a network virtual appliance virtual machine 860.

[0075] refer to Figure 9A and Figure 9BThis illustrates an example of bringing a first-party service into the client's Vnet via a private endpoint. A bare metal server 830 is shown, accessing blob storage via a service endpoint 910 set up on the bare metal subnet. A private endpoint (172.2.0.8) linked to the blob storage can be created. The storage account can be accessed by the bare metal server 830 via the private endpoint. The SDN appliance 920 can perform all the necessary conversions (e.g., V4 → V6) required to initiate the connection.

[0076] Figure 10 This illustrates cross-virtual network connectivity with Vnet peering (regional and global). A peering Vnet is shown with a bare-metal server 830 in Vnet 1 and a VM 1020 in Vnet 2.

[0077] In the example scenario shown, SDN capabilities can be enhanced by implementing a policy of separating SDN from the host and moving it to SDN appliances strategically placed in the network.

[0078] Software-defined networking (SDN) is typically implemented on general-purpose compute nodes. The SDN control plane can program hosts to provide core network functions such as security, virtual networking, and load balancer policies. Through the implementation of SDN appliances, when establishing regional or global virtual network peering, a bare-metal server (10.10.0.4) can connect to virtual machines in another virtual network (172.31.0.4) within the same region. Furthermore, outbound routing tables can be distributed on bare-metal traffic. In this scenario, SDN appliances can provide connectivity to address prefixes across all regional peering virtual networks.

[0079] refer to Figure 11 An example of a decoupled SDN appliance 1110 according to some embodiments is shown. In some embodiments, as further described herein, the SDN appliance can allow the use of an SDN control plane to manage network devices while providing high availability and fault tolerance. Figure 11 An example network optimization chassis including an SND agent 1120, a virtual filtering platform (VFP) 1130, a policy 1140, and an FPGA 1150 is shown. The SKU can be changed, and the host can be used as an SDN appliance if needed. The various embodiments described herein illustrate the use of SDN appliances as a general concept.

[0080] refer to Figure 12 This example demonstrates how to decouple the Virtual Filtering Platform (VFP) implementation from the hypervisor port and move it to a port with a custom Layer 2 identifier.

[0081] refer to Figure 13 This illustrates an example of remote flow offloading. If the source is a host appliance, the SDN appliance 1320 can act as a remote policy evaluator and offload the flow to the remote host 1310 after the initial TCP handshake.

[0082] In this mode, only the initial TCP handshake can be performed by the SDN appliance 1320. In an embodiment, if the source is not a host, the SDN appliance 1340 can offload the flow to a local offloading device, such as a field-programmable gate array (FPGA) device. In one embodiment, the described system and method may include an FPGA device configured as a hardware acceleration device such that data traffic is processed in hardware and the impact on software is minimized, thereby allowing end-to-end traffic to maintain its line rate in a way that reduces latency. In this mode, all packets pass through the SDN appliance.

[0083] refer to Figure 14 This illustrates an example of an SDN appliance running the full SDN stack and applying all SDN policies. The lowest level (typically rack-level) switch, which may be called a top-of-rack (ToR) switch, may not have the capability to support all SDN functions.

[0084] refer to Figure 15 This illustrates an example that allows connection to storage resource 1530 from a locally deployed network 1510 via a private endpoint through an SDN appliance 1520.

[0085] refer to Figure 16 This illustrates an example of how SDN policies can be applied to fNICs that are not bound to virtual machines and are therefore bound to the physical host.

[0086] As described above, various embodiments include data center network models that incorporate selectively placed network hops that can apply software-defined networking (SDN) policies as "collisions on the line" (i.e., network traversal points) before data traffic reaches its destination. This can provide improved overall performance, host decoupling, and the application of other SDN capabilities before the target receives the data traffic.

[0087] In the various embodiments described herein, a floating or flexible network interface can be implemented. As used herein, such a floating or flexible network interface may be referred to as a flexible network interface controller (fNIC) or commonly as a virtual port (vport). In embodiments, an fNIC may be inserted into the path from the host to the destination and may be configured to apply SDN policies before reaching the destination. In some embodiments, one or more fNICs may be implemented on an SDN appliance. The point at which SDN policies are implemented may float between the host and the SDN appliance depending on the flow conditions.

[0088] In an embodiment, criteria for determining where to implement an SDN policy for a flow may include, for example, the age of the flow, the rate of the flow, the total number of bytes transmitted on the flow, the total number of flows used at the corresponding host, etc. Since an SDN policy applied to a flow may include multiple aspects, different aspects of the policy can be implemented at different locations.

[0089] In conventional SDN applications, SDN policy application can be based on tuple processing. When implemented as a middlebox technology such as an SDN appliance, the traffic destination requiring SDN operation behind the middlebox can be pre-programmed as an fNIC. This fNIC resolves traffic consisting of combinations of custom identifiers such as VLANs, MAC addresses, IP addresses, and other information to uniquely identify the flow and apply appropriate policies. This programmable layer provides flexibility for applying policies in different network environments and scenarios.

[0090] In some embodiments, a Layer 2 address and a tenant network identifier can be used to identify an fNIC (e.g., a virtual port) in an SDN appliance. More generally, a virtual port can represent an entity that is not a virtual machine. For example, a virtual port can be associated with a bare metal server, a private endpoint, etc. Since these entities do not reside on the same device as the switch, a method is needed to identify packets sent to / from the physical entity represented by the virtual port, or more generally, to identify any endpoint that is not a host.

[0091] In some implementations using rack-level switches (such as top-of-rack (ToR) switches), these devices typically lack the capability to perform translation. SDN appliance gateways can be used to host these agents and provide switching functionality, as well as translation and connectivity. SDN appliances can accept policies for performing translation. In some embodiments, agents can be implemented by programming drivers that run on the SDN appliance. Traffic sent over workloads can be directed through the SDN appliance, which can apply policies and perform translation on the traffic, sending it to its destination. In some configurations, SDN appliances may include virtual switches, such as virtual filtering platforms.

[0092] For example, for customer networks using dedicated hardware, SDN appliances with fNICs or virtual ports can be used. Many switches have destination information, but they are route-based and typically lack the ability to apply policies. Furthermore, some destinations (such as private links) and destinations including first-party or third-party services within a customer's virtual network may not have the ability to apply policies.

[0093] refer to Figure 11An example SDN appliance 1110 is shown, which includes an SDN agent 1120, a network driver capable of performing network transformations such as a Virtual Filtering Platform (VFP) 1130, and a policy 1140 associated with the fNIC.

[0094] refer to Figure 12 The example shown is a host 1210 with two virtual machines and two virtual machine ports, using a custom Layer 2 identifier. Data traffic is sent to an SDN appliance 1220, which applies policies before sending it to its destination. In other examples, the decoupling shown can be performed for any workload and applicable policies.

[0095] refer to Figure 13 Examples of local and remote flow offloading capabilities are shown. For remote flow offloading, host 1320 can act as a remote policy evaluator and offload the flow to the SDN appliance after the initial TCP handshake. For local flow offloading, if the source is not a host, device 1340 can offload the flow to the local FPGA.

[0096] SDN appliances can become single points of failure for software-defined networks. Mitigation of SDN appliance failures must consider the preservation of transient states (e.g., TCP flow state) and the location of state within a single SDN appliance. If two appliances are cross-connected to two switches, the single point of failure can be avoided from a physical device perspective, but it may introduce state management and anticipated connectivity issues.

[0097] In network scenarios where the FPGA is located in the network path between the NIC and the ToR switch, in some cases, the network can be implemented by adding support for multiple FPGAs for each NIC. When the FPGA is no longer located between the NIC and the ToR, packets must still be able to be sent from the FPGA to the NIC. Since there is only one NIC for each multiple FPGA, and the FPGA is connected to multiple ToRs, a new Layer 2 network path needs to be established between the FPGA and the NIC using the ToRs. To avoid extending the Layer 2 domain to higher-level switches, the FPGA can have the ability to pipe packets into a single port and outflow packets from different ports associated with different pipes.

[0098] In an embodiment, SDN appliances can utilize transient states (e.g., TCP flow states) when processing packets. If this state is not replicated across the set of devices, the network surrounding these devices must attempt to send packets from the same flow to the same device without overloading the device or breaking clients in the event of hardware failure or recovery.

[0099] An extension to ECMP can be implemented to support hierarchical hashing, enabling switches to meet these requirements without requiring switches to maintain flow state mappings. The hierarchical hashing method allows switches to select SDN appliance links (potentially represented as a set of links) from multiple SDN appliances and continue sending flows to the same SDN appliance, even if the availability of individual links changes. If a link fails, the switch can then select a different link to the same SDN appliance, allowing the SDN appliance to still utilize existing transient states and act appropriately. If the entire SDN appliance fails, the switch can select a different SDN appliance to restore connectivity as quickly as possible to maintain high availability. If the link recovers, flows can be returned to that link. Flows returned to the link should originate from the same SDN appliance and should not result in flow switching between appliances.

[0100] In this embodiment, hierarchical hashing can be deterministic between the two switches in such a way that if switch 1 chooses to send a flow to SDN appliance 1 in both SDN appliance 1 and SDN appliance 2, then switch 2 must choose to send it to SDN appliance 1 in the same manner. Furthermore, hierarchical hashing behavior should be maintained between the two switches regardless of the state of individual ports, as these ports may fail individually. This behavior is necessary because, for fault-tolerant and redundant systems, flows from higher-level network topologies can arrive at any switch at any time, and any appliance link can be open or closed on either switch. The two switches should be able to achieve this in a stateless manner without requiring continuous state sharing.

[0101] During recovery, when using buckets and / or consistent hashing for ECMP and fault recovery updates to the mapping of bucket sets, applying this update over time may reduce the scope of changes at any given time (while extending the total time required to complete recovery). This can be useful if the impact of distributed recovery benefits the overall system functionality (giving client applications with multiple connections a graceful transition to a valid connection and giving network devices more time to adapt to sudden changes in traffic behavior). Since the transition is from a healthy link to a different healthy link, slowing down the transition will not affect availability.

[0102] refer to Figure 17Example implementations of dual redundant ToRs 1710 and 1730 with cross-connection to a dual redundant SDN appliance 1740 are shown. To provide full fault tolerance, state must be maintained to determine whether any given packet is part of an existing request. In the example implementation, the VFP is the controller for the data plane. The two VFPs may not be communicatively coupled, and therefore the TORs should be kept fully stateful. The two ToRs should not have any conflicting information in order to send responses to the correct device.

[0103] The hierarchical hashing described above should be deterministic across the two TORs in such a way that if TOR1 chooses to send some flow to SDN appliance 1 in both SDN appliance 1 and SDN appliance 2, then TOR2 must choose to send it to SDN appliance 1 in the same manner. Furthermore, the hierarchical hashing behavior should be consistent across the two TORs regardless of the state of the individual ports, as these ports may fail individually. This behavior is necessary in this way because, for fault-tolerant and redundant systems, flows from higher-level network topologies can arrive at any TOR at any point in time, and any of the six links can be on or off on any TOR.

[0104] In the example shown, using ECMP via ToR involves treating all ports as equal. However, instead of treating all 12 ports as equal (which could lead to loss of state information in the event of a failure), it is better to treat the two sets of ports as two sets of six ports each. In this embodiment, hierarchical hashing can be implemented by first selecting one of two port libraries. Then, a port is selected from that library. Thus, if a link fails, the port library that includes the failed link is selected, allowing the appropriate VFP to be chosen.

[0105] In this example, if a port fails and then returns to service, traffic may have been rerouted during the outage. If the regenerated port is brought back immediately, the corresponding cache will not be filled with active data, potentially leading to more cache misses and data lookup delays. In this embodiment, the regenerated port can be brought back to full operation in a staggered manner within the port library over a period of time. Since the library will still maintain healthy operation through five ports, the gradual return of a sixth port will not impair healthy system operation.

[0106] In this embodiment, the VFP can determine the pipeline jump to ensure that data traffic is sent to the corresponding FPGA port on the redundant ToR.

[0107] Network devices with specific connections, including programmable ASICs (such as FPGAs), may lack a network stack capable of interacting with public network protocols (such as ARP and LACP) that are critical for routing and monitoring. Such network devices may reside, for example, on a server containing a network interface capable of accepting control traffic. This disclosure describes embodiments for modifying the ARP framework running on a network device so that it can provide connectivity to programmable network devices (such as FPGAs) that do not have a native Layer 3 network stack running on it. Such devices may share a public network interface on a server to bootstrap network connectivity (such as ARP). Agents and drivers on SDN appliances can program and communicate with these connected devices to complete the bootstrapping of network connectivity.

[0108] In scenarios where traffic needs to be directly routed to these connected network devices, in some embodiments, the proxy and driver running on the server can be implemented as follows:

[0109] 1. Announce routes from the server's network interfaces to these connected network devices. In one example, the same address prefix with multiple next hops can be announced to such connected network devices. Not all physical devices support BGP path addition, so different next hops can be announced to the upstream ToR on different sessions to enable ECMP to be performed equally across all these connected network devices.

[0110] 2. Generate ARP response packets for the connected devices and send them out through the appropriate link.

[0111] 3. Provides monitoring and availability signals for links attached to the network devices. Since these connected network devices cannot respond to LACP packets, BGP sessions established from a proxy on the server can provide indications of link availability.

[0112] When running in a virtual network host, as packets pass through the stack, they can be sent to the VFP, which finds the correct translation, selects it, and offloads the translation to the FPGA, which then performs the translation. However, in the case of FPGAs or other connected devices lacking a traditional network stack, these devices cannot communicate using traditional IP addresses, MAC addresses, etc., because they lack the stack required for communication via traditional protocols. Therefore, such devices cannot respond to ARP, LACP, and other packets. In one example, a device such as a TOR can send BGP packets. A device such as an FPGA will not respond. In the various embodiments disclosed herein, methods and techniques are described to make any programmable device L3 routable, which would otherwise lack a network stack.

[0113] In this embodiment, a virtual IP address may be assigned to the FPGA. Additionally, in order to respond when a packet addresses the virtual IP address, a VFP running on the SDN appliance may act as a proxy for the FPGA and respond on its behalf.

[0114] More generally, virtual MAC and IP addresses can be assigned to connected devices, an agent that maintains or accesses the mapping of these connected devices to (virtual IP, virtual MAC) pairs, and a driver programmed by the agent that is capable of handling network boot requests and sending boot responses through these connected devices.

[0115] The software running on SDN appliances can be used to bootstrap the initial network of these connected devices (e.g., ARP, link discovery). Once the network is bootstrap, traffic can be directed to the links attached to these connected devices, without the devices themselves needing to support network bootstrap protocols.

[0116] The described embodiments can support multiple FPGAs and other connected devices on SDN appliances in multiple different networks and physical topologies.

[0117] For ARP support, since the FPGA does not support the network stack, ARP responses can be generated in the SDN appliance software. For the SDN appliance software to generate an ARP response, the ARP request must reach the NIC, which is a device in the SDN appliance that supports the network stack. The software can generate the ARP response. The ARP response should be sent as if it came from an FPGA link, so that MAC learning can be triggered at the top of the FPGA MAC address.

[0118] refer to Figure 20 In one example, ARP request packets can be routed to the NIC, and response packets can be configured to appear to originate from the FPGA.

[0119] Although FPGAs are used to illustrate the techniques described, it should be understood that these techniques can be applied to other types of connected devices, such as GPUs.

[0120] continue Figure 20 For example, IP addresses can be assigned to each FPGA of SDN appliances 2020 and 2030 as a peer (12 addresses in this example). For ToR 2040 and ToR 2050, there appear to be 12 distinct peers. In another example, FPGA link monitoring can be supported by running LACP to ping links, notify peers of link status, and remove bad links. When implementing LACP, the FPGA may be considered to be in a good link state.

[0121] like Figure 20 As shown in the example, each appliance has 6 FPGAs, and each FPGA has 2 ports. In the example shown, each device uses 6 BGP sessions to advertise appliance data path IP addresses to 2 ToRs, creating a total of 12 BGP sessions via the NIC, with 6 BGP sessions established via the Layer 1 ToR to the second ToR. To create 12 BGP sessions, 12 different IPs can be used for the BGP sessions, which are configured as secondary IPs on the NIC. Furthermore, each BGP session can have a next-hop IP corresponding to one FPGA.

[0122] Packets destined for the appliance IP can be VXLAN encapsulated. The ToR can perform a hash on the internal packets to select the next-hop IP and forward the packets to the appropriate appliance FPGA. ARP requests for the next-hop IP can be responded to by an NM proxy running on the appliance with the FPGA MAC address.

[0123] refer to Figure 21 This shows an example of an abnormal path ARP with ARP request 2110.

[0124] In some embodiments, an enhanced BGP publishing framework can be implemented that provides connectivity to programmable network devices that do not have a native Layer 3 network stack (e.g., an FPGA). Such devices can share a public network interface on a server to bootstrap network connectivity (e.g., ARP). Agents and drivers on SDN appliances can be programmed and communicate with these connected devices to complete the bootstrap of network connectivity.

[0125] Instead of implementing conventional BGP, the speaker designates its own IP address as the next hop for a given Network Layer Reachability Information (NLRI), where the next hop can specify a virtual IP address assigned to the connected device. After network bootstrapping, an agent on the server can provide monitoring signals to the BGP speaker to communicate the health status of the connected devices. Based on these health signals, the BGP server can publish or withdraw BGP sessions associated with the network devices. Since devices connected in this way do not support traditional LACP, BGP control signals from the BGP speaker can serve as a viable alternative for detecting link failures.

[0126] Some networks can have a heterogeneous mix of network switches with different BGP capabilities. To advertise multiple routes with the same NLRI from the same appliance, all network switches need to support BGP ADD_PATH. If a switch does not support ADD_PATH, a second route advertised for the same NLRI from the same peer will result in the implicit withdrawal of the first route. The various embodiments disclosed herein describe a method to ensure homogeneity across all switch types and provide a method to support switches that do not have BGP ADD_PATH capability. Homogeneity is ensured by advertising each route on different sessions, advertising the same NLRI on sessions using different IP addresses assigned to network interfaces on the appliance. This allows upstream network switches to treat these routes as coming from different BGP peers, and network switches can perform ECMP across peers.

[0127] The solution described above can be used to support multiple FPGAs (connected devices) on SDN appliances (servers) in multiple different network and physical topologies. In one example, the SDN appliance can have 6 FPGAs, each with 2 pipelines (per port).

[0128] refer to Figure 22 This illustrates an example SDN appliance network according to some embodiments. Figure 22 Two TORs, 2040 and 2050, are shown, along with two SDN appliances, 2020 and 2030, cross-connected to these two TORs. Each SDN appliance can run a BGP speaker service. Each SDN appliance can create a unique BGP session to the requested TOR for each FPGA. The BGP speaker is a service running on appliances #1 and #2, and it notifies each FPGA of its availability via the NIC. Each FPGA has one BGP session, therefore there are six sessions from appliance #1 to TOR #1 for IPv4, and six sessions from appliance #1 to TOR #2 (due to the cross-connection). Similarly, there are 12 sessions for IPv6 announcements.

[0129] The six sessions from appliance #1 to TOR #2 are multi-hop sessions running across TOR #1 → T1 set → TOR #2. This session announcement connects the FPGA from appliance #1 to TOR #2. The BGP announcement is not a standard next hop, but rather a next hop for a different interface.

[0130] TOR may not support the BGP ADD_PATH attribute, so to support such devices, each session can only publish one next hop (therefore 12 sessions), instead of having all publications happen on a single BGP session and using BGP ADD_PATH to publish multiple next hops.

[0131] BGP speakers can be provided with the special ability to query FPGA availability and selectively announce only active and healthy FPGAs. This supports resilient Layer 3 networks.

[0132] Monitoring FPGA availability can be achieved in several ways:

[0133] Using software register access on the FPGA allows the FPGA to provide the BGP speaker with its health status via software; or

[0134] The data path method is used, which checks the availability of the FPGA by injecting a health signal into the data path.

[0135] Using the above signals, the speaker can selectively announce FPGA availability to the TOR, and the TOR can forward traffic based on FPGA availability, thereby creating a resilient Layer 3 network where changes in FPGA availability keep the data path running across other available links / FPGAs.

[0136] This document describes various aspects of this disclosure with reference to specific examples and embodiments, which are intended to illustrate but not limit the scope of this disclosure. It should be understood that the subject matter presented herein can be implemented as a computer process, a computer control apparatus, a computing system, an article of manufacture such as a computer-readable storage medium, or a component including hardware logic for implementing functionality, such as a field-programmable gate array (FPGA) device, a massively parallel processor array (MPPA) device, a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a multiprocessor system-on-a-chip (MPSoC), and the like.

[0137] Components may also include other ways of performing functions using the device, such as a) implementing at least some tasks in hard ASIC logic, etc.; b) implementing at least some tasks in soft (configurable) FPGA logic, etc.; c) running at least some tasks as software overlaid on an FPGA software processor, etc.; d) running at least some tasks as software on a hard ASIC processor, etc., or any combination thereof. Components may represent a homogeneous collection of hardware acceleration devices, such as FPGA devices. On the other hand, components may represent a heterogeneous collection of different types of hardware acceleration devices, including different types of FPGA devices with different corresponding processing capabilities and architectures, a mixture of FPGA devices and other types of hardware acceleration devices, etc.

[0138] Figure 23A general-purpose computing device 2300 is illustrated. In the illustrated embodiment, the computing device 2300 includes one or more processors 2310a, 2310b, and / or 2310n (which may be referred to as "processor 2310" individually or as "multiple processors 2310" in the case of multiple processors) coupled to system memory 2323 via an input / output (I / O) interface 2330. The computing device 2300 also includes a network interface 2340 coupled to the I / O interface 2330.

[0139] In various embodiments, computing device 2300 may be a single-processor system including one processor 2310 or a multiprocessor system including several processors 2310 (e.g., two, four, eight, or another suitable number). Processor 2310 may be any suitable processor capable of executing instructions. For example, in various embodiments, processor 2310 may be a general-purpose or embedded processor implementing any of a variety of instruction set architectures (ISAs), such as x2323, PowerPC, SPARC, or MIPS ISA, or any other suitable ISA. In a multiprocessor system, each processor of processor 2310 may share, but does not necessarily implement, the same ISA.

[0140] System memory 2323 may be configured to store instructions and data accessible by processor(s) 2310. In various embodiments, system memory 2323 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory. In the illustrated embodiment, program instructions and data (e.g., the methods, techniques, and data described above) that implement one or more desired functions are shown as code 2325 and data 23223 stored within system memory 2320.

[0141] In one embodiment, I / O interface 2330 may be configured to coordinate I / O traffic between processor 2310, system memory 2323, and any peripheral devices (including network interface 2340 or other peripheral interfaces) within the device. In some embodiments, I / O interface 2330 may perform any necessary protocol, timing, or other data conversions to transform data signals from one component (e.g., system memory 2320) into a format suitable for use by another component (e.g., processor 2310). In some embodiments, I / O interface 2330 may include support for devices attached via various types of peripheral buses, such as the Peripheral Component Interconnect (PCI) bus standard or variants of the Universal Serial Bus (USB) standard. In some embodiments, the functionality of I / O interface 2330 may be divided into two or more separate components. Furthermore, in some embodiments, some or all of the functionality of I / O interface 2330 (e.g., the interface to system memory 2320) may be directly integrated into processor 2310.

[0142] Network interface 2340 can be configured to allow data to pass between computing device 2300 and other devices 23230 connected to network(s) 23230 (e.g., such as...). Figures 1 to 5 This can be exchanged between other computer systems or devices (as exemplified). In various embodiments, network interface 2340 can support communication via any suitable wired or wireless general-purpose data network (e.g., Ethernet type). Furthermore, network interface 2340 can support communication via telecommunications / telephone networks (e.g., analog voice networks or digital fiber optic communication networks), via storage area networks (e.g., Fibre Channel SAN), or via any other suitable type of network and / or protocol.

[0143] In some embodiments, as described above, for Figures 1-7In the embodiments described for implementing the corresponding methods and devices, system memory 820 may be one embodiment of a computer-accessible medium configured to store program instructions and data. However, in other embodiments, program instructions and / or data may be received, transmitted, or stored on different types of computer-accessible media. The computer-accessible medium may include non-transitory storage media or memory media, such as magnetic or optical media, for example, a disk or DVD / CD coupled to computing device 800 via I / O interface 830. Non-transitory computer-accessible storage media may also include any volatile or non-volatile media, such as RAM (e.g., SDRAM, DDR SDRAM, RDRAM, SRAM, etc.), ROM, etc., which may be included as system memory 820 or another type of memory in some embodiments of computing device 800. Further, the computer-accessible medium may include transmission media or signals, such as electrical, electromagnetic, or digital signals, transmitted via communication media such as networks and / or wireless links, such as those implemented via network interface 840. Multiple computing devices (e.g., Figure 8 Some or all of those shown can be used to implement the functions described in the various embodiments; for example, software components running on various different devices and servers can cooperate to provide the functions. In some embodiments, in addition to implementation using a general-purpose computer system, or instead of implementation using a general-purpose computer device, a portion of the functions can be implemented using a storage device, a network device, or a dedicated computer system. The term "computing device" as used herein refers to at least all of these types of devices, but is not limited to them.

[0144] Various storage devices and their associated computer-readable media provide non-volatile storage for the computing devices described herein. As discussed herein, computer-readable media can refer to mass storage devices such as solid-state drives, hard disk drives, or CD-ROM drives. However, those skilled in the art will understand that computer-readable media can be any available computer storage medium accessible through the computing device.

[0145] By way of example and not limitation, computer storage media can include volatile and non-volatile, removable and non-removable media implemented with any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. For example, computer media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technologies, CD-ROM, digital versatile disc (“DVD”), HD-DVD, BLU-RAY or other optical storage, cassette tape, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by the computing devices discussed herein. For the purposes of the claims, the phrases “computer storage medium,” “computer-readable storage medium,” and variations thereof do not, by themselves, include wave, signal, and / or other transient and / or intangible communication media.

[0146] Encoding the software modules presented herein can also transform the physical structure of the computer-readable medium presented herein. In different embodiments of this specification, the specific transformation of the physical structure can depend on various factors. Examples of such factors may include, but are not limited to, the technology used to implement the computer-readable medium, whether the computer-readable medium is characterized as primary or secondary storage, etc. For example, if the computer-readable medium is implemented as a semiconductor-based memory, the software disclosed herein can be encoded on the computer-readable medium by transforming the physical state of the semiconductor memory. For example, the software can transform the state of transistors, capacitors, or other discrete circuit elements constituting the semiconductor memory. The software can also transform the physical state of these components to store data thereon.

[0147] As another example, the computer-readable medium disclosed herein can be implemented using magnetic or optical techniques. In such an implementation, when software is coded therein, the software presented herein can transform the physical state of the magnetic or optical medium. These transformations may include altering the magnetic properties of a specific location within a given magnetic medium. These transformations may also include altering the physical characteristics or properties of a specific location within a given optical medium to change the optical properties of those locations. Other transformations of the physical medium are possible without departing from the scope and spirit of this specification; the examples provided above are merely to facilitate the discussion.

[0148] In light of the foregoing, it should be understood that numerous types of physical transformations occur within the disclosed computing device in order to store and execute the software components and / or functions presented herein. It is also conceivable that the disclosed computing device may not include… Figure 8 All of the components shown may include Figure 8 Other components not explicitly shown in the document, or those that can be utilized with Figure 8 The architecture shown is completely different from the one described above.

[0149] Although various configurations have been described in a language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended representation is not necessarily limited to the specific features or actions described. Rather, specific features and actions are disclosed as exemplary forms for implementing the claimed subject matter.

[0150] Conditional language used herein, such as “can,” “may,” “may,” “perhaps,” “for example,” etc., unless otherwise specifically stated or otherwise understood in the context in which they are used, is generally intended to convey that a particular embodiment includes, while other embodiments do not, a particular feature, element, and / or step. Therefore, such conditional language is not generally intended to imply that a feature, element, and / or step is necessary in any way for one or more embodiments, or that one or more embodiments necessarily include logic for determining, with or without author input or prompting, whether such features, elements, and / or steps are included in or will be performed in any particular embodiment. The terms “comprising,” “including,” “having,” etc., are synonyms and are used in an open-ended manner, not excluding other elements, features, actions, operations, etc. Furthermore, the use of the term “or” is inclusive (not exclusive), and therefore, when used to connect a list of elements, the term “or” indicates one, some, or all of the elements in the list.

[0151] While specific exemplary embodiments have been described, these embodiments are presented by way of example only and are not intended to limit the scope of the invention disclosed herein. Therefore, nothing in the foregoing description is intended to imply that any particular feature, characteristic, step, module, or block is necessary or indispensable. In fact, the novel methods and systems described herein can be embodied in a variety of other forms; furthermore, various omissions, substitutions, and changes can be made to the form of the methods and systems described herein without departing from the spirit of the inventions disclosed herein. The appended claims and their equivalents are intended to cover forms or modifications that fall within the scope and spirit of certain inventions disclosed herein.

[0152] It should be understood that any references to items and / or abstract concepts such as “first,” “second,” etc., in the specification are not intended to, and should not be construed as, necessarily corresponding to any references to elements such as “first” and “second” in the claims. In particular, in the content of this invention and / or the following detailed description, items and / or abstract concepts (e.g., the operational state of a single computing device and / or computing cluster) can be distinguished by numerical names without needing to correspond to such names in the claims or even other paragraphs of the content of this invention and / or the detailed description. For example, any designation of a “first operational state” and a “second operational state” for a computing cluster within a paragraph of this disclosure is only used to distinguish two different operational states of the computing cluster within that particular paragraph, and not specifically in any other paragraph of the claims.

[0153] Although various techniques have been described in language specific to structural features and / or methodological behavior, it should be understood that the subject matter defined in the appended representation is not necessarily limited to the specific features or actions described. Rather, specific features and actions are disclosed as exemplary forms for implementing the claimed subject matter.

[0154] Figure 24 This is a flowchart illustrating various aspects of a routine 2400 executed by a data processing system to drive a user in a communication session. Those skilled in the art will understand that the operations of the methods disclosed herein are not necessarily presented in any particular order, and that it is possible and contemplated to perform some or all of the operations in alternative orders(s). For ease of description and illustration, the operations have been presented in the order of demonstration. Operations may be added, omitted, performed together, and / or performed simultaneously without departing from the scope of the appended claims.

[0155] It should also be understood that the method shown can be terminated at any time and need not be fully executed throughout the entire process. Some or all of the operations of the method and / or substantially equivalent operations can be performed by executing computer-readable instructions included on a computer storage medium, as defined herein. The term "computer-readable instructions" and variations thereof, as used in the specification and claims, are used herein to extend to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, etc. Computer-readable instructions can be implemented on a variety of system configurations, including single-processor or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based programmable consumer electronics, combinations thereof, etc.

[0156] It should be understood that the logical operations described herein are implemented as (1) a series of computer-implemented actions or program modules, such as those described herein, running on a computing system and / or (2) as machine logic circuits or circuit modules interconnected within the computing system. The key to implementation depends on the performance and other requirements of the computing system. Therefore, logical operations can be implemented in software, firmware, dedicated digital logic, and any combination thereof. Thus, although routine 2400 is described as running on system 240, it is understood that routine 2400 and the other operations described herein can be executed on a single computing device (e.g., computing device 244) or multiple devices.

[0157] also, Figure 24The operations shown in the other figures can be implemented in association with the UI presented in the examples above. As examples, the various devices and / or modules described herein can generate, send, receive, and / or display content (e.g., live content, broadcast events, recorded content, etc.) and / or present a UI that includes the rendering of one or more participants, such as remote computing devices, avatars, channels, chat sessions, video streams, images, virtual objects, and / or applications associated with the communication session.

[0158] Routine 2400 begins with operation 2402, which illustrates an SDN appliance receiving a first data packet including a network protocol request. In this embodiment, the first data packet includes a virtual address.

[0159] Operation 2404 illustrates a virtual address-based mapping that determines the virtual address of a received data packet to correspond to the MAC address of a programmable network device.

[0160] Operation 2406 illustrates generating a response to a network protocol request based on the configuration of a programmable network device, the response being configured to identify the programmable network device as the source of the response.

[0161] Operation 2408 illustrates the sending of a response on behalf of a programmable network device.

[0162] Operation 2410 illustrates receiving a second data packet addressed to an endpoint in a virtual computing environment via a programmable network device.

[0163] Operation 2412 illustrates mapping one of multiple policies to an endpoint.

[0164] Operation 2414 illustrates the offloading of the application of the mapping strategy for the second data packets by the SDN appliance to a programmable network device. In one embodiment, the programmable network device is configured to operate in response to a network protocol request.

[0165] The disclosures described herein also include the subjects described in the following clauses:

[0166] Clause 1: A system for implementing policies in a software-defined network (SDN) environment of a virtual computing environment, the system comprising: A programmable network device configured to handle data traffic; One or more processors; and A memory communicating with the one or more processors, the memory having computer-readable instructions stored thereon, which, when executed by the one or more processors, cause the system to perform operations including: Receive a first data packet including a network protocol request, the first data packet including a virtual address; Based on the mapping of virtual addresses, the correspondence between the virtual address of the received data packet and the MAC address of the programmable network device is determined; Generate a response to the network protocol request, the response being based on the configuration of the programmable network device and configured to identify the programmable network device as the source of the response; This response is sent on behalf of the programmable network device; Receive a second data packet addressed to an endpoint in the virtual computing environment via the programmable network device; Map one of the multiple strategies to this endpoint; and The application of the mapping policy for the second data packet is offloaded to the programmable network device, wherein the programmable network device is configured to operate in response to the network protocol request.

[0167] Clause 2: In a system pursuant to Clause 1, the virtual address includes both a virtual IP address and a virtual MAC address.

[0168] Clause 3: A system pursuant to any one of Clauses 1-2, wherein the data packet is received by a server having a network interface operable to initiate network connectivity for the network device.

[0169] Clause 4: A system pursuant to any of Clauses 1-3, wherein the network device is an FPGA or an ASIC.

[0170] Clause 5: A system pursuant to any of Clauses 1-4, wherein the identifier is executed by an agent that performs on the system.

[0171] Clause 6: A system pursuant to any of Clauses 1-5, wherein routes from a network interface on a server to the network device are published.

[0172] Clause 7: A method for implementing policies in a software-defined network (SDN) within a virtual computing environment by an SDN appliance configured to decouple the implementation of SDN policies from hosts within the virtual computing environment, the SDN appliance including a programmable network device configured to process data traffic, the programmable network device lacking a Layer 3 network stack, the method comprising: The SDN device receives a first data packet including a network protocol request, the first data packet including a virtual address; Based on the mapping of virtual addresses, the correspondence between the virtual address of the received data packet and the MAC address of the programmable network device is determined; Generate a response to the network protocol request, the response being based on the configuration of the programmable network device and configured to identify the programmable network device as the source of the response; This response is sent on behalf of the programmable network device; Receive a second data packet addressed to an endpoint in the virtual computing environment via the programmable network device; Map one of the multiple strategies to this endpoint; and The SDN device offloads the application of the mapping policy to the second data packet to the programmable network device, wherein the programmable network device is configured to operate in response to the network protocol request.

[0173] Clause 8: The method of Clause 11, wherein the same address prefix with multiple next hops is advertised to the network device.

[0174] Clause 9: The method pursuant to any one of Clauses 11 and 12 further includes: providing monitoring and availability signals for the links attached to the programmable network device, wherein a session can be initiated or withdrawn.

[0175] Clause 10: Any method of Clauses 11-13, wherein the ARP response is generated by an SDN appliance.

[0176] Clause 11: The method of any one of Clauses 11-14, wherein an ARP response is sent if the response is sent from a network device link.

[0177] Clause 12: The method of any one of Clauses 11-15, wherein the virtual address includes a virtual IP and a virtual MAC.

[0178] Clause 13: The method according to any one of Clauses 11-16, wherein the data packet is received by a server having a network interface operable to initiate a network connection for a network device.

[0179] Clause 14: The method of any one of Clauses 11-17, wherein the network device is an FPGA or an ASIC.

[0180] Clause 15: A computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by one or more processors of a computing device, cause the computing device to: Receive a first data packet including a network protocol request, the first data packet including a virtual address of the virtual computing environment; Based on the mapping of virtual addresses, the virtual address of the received data packet is determined to correspond to the MAC address of the programmable network device, which is then configured to process data traffic. Generate a response to the network protocol request, the response being based on the configuration of the programmable network device and configured to identify the programmable network device as the source of the response; This response is sent on behalf of the programmable network device; Receive a second data packet addressed to an endpoint in the virtual computing environment via the programmable network device; Map one of the multiple strategies to this endpoint; and The application of the mapping policy for the second data packet is offloaded to the programmable network device, wherein the programmable network device is configured to operate in response to the network protocol request.

[0181] Clause 16: Computer-readable storage medium pursuant to Clause 15, wherein the virtual address includes a virtual IP and a virtual MAC.

[0182] Clause 17: A computer-readable storage medium pursuant to any of Clauses 15 and 16, wherein the data packet is received by a server having a network interface operable to bootstrap network connectivity for a network device.

[0183] Clause 18: A computer-readable storage medium pursuant to any of Clauses 15-17, wherein the network device is an FPGA or an ASIC.

[0184] Clause 19: A computer-readable storage medium pursuant to any of Clauses 15-18, wherein the identifier is executed by an agent implemented on the computing device.

[0185] Clause 20: A computer-readable storage medium pursuant to any of Clauses 15-19, wherein a route from a network interface on a server to the network device is published.

Claims

1. A system for implementing policies in a software-defined networking (SDN) environment of a virtual computing environment, the system comprising: A programmable network device configured to handle data traffic, wherein the programmable network device lacks a Layer 3 network stack; One or more processors; as well as A memory communicating with the one or more processors, the memory having computer-readable instructions stored thereon, the computer-readable instructions causing the system to perform operations when executed by the one or more processors, the operations including: Receive a first data packet including a network protocol request, the first data packet including a virtual address; Based on the mapping of virtual addresses, it is determined that the virtual address of the received data packet corresponds to the MAC address of the programmable network device; Generate a response to the network protocol request, the response being based on the configuration of the programmable network device and configured to identify the programmable network device as the source of the response; The programmable network device sends the response on behalf of the programmable network device. Receive a second data packet addressed to an endpoint in the virtual computing environment via the programmable network device; Map one of the multiple strategies to the endpoint; and The application of the mapping policy to the second data packet is offloaded to the programmable network device, wherein the programmable network device is configured to operate in accordance with the response to the network protocol request.

2. The system according to claim 1, wherein the virtual address includes a virtual IP and a virtual MAC.

3. The system of claim 1, wherein the data packets are received by a server having a network interface operable to enable network connectivity for the network device.

4. The system of claim 1, wherein the network device is an FPGA or an ASIC.

5. The system of claim 1, wherein the identifier is executed by an agent running on the system.

6. The system of claim 1, wherein routes from the network interface on the server to the network device are published.

7. A method for implementing policies in a software-defined networking (SDN) environment of a virtual computing environment by an SDN appliance, the SDN appliance being configured to decouple the implementation of the SDN policies from hosts in the virtual computing environment, the SDN appliance including a programmable network device configured to process data traffic, the programmable network device lacking a Layer 3 network stack, the method comprising: The SDN device receives a first data packet including a network protocol request, the first data packet including a virtual address; Based on the mapping of virtual addresses, it is determined that the virtual address of the received data packet corresponds to the MAC address of the programmable network device; Generate a response to the network protocol request, the response being based on the configuration of the programmable network device and configured to identify the programmable network device as the source of the response; The programmable network device sends the response on behalf of the programmable network device. Receive a second data packet addressed to an endpoint in the virtual computing environment via the programmable network device; Map one of the multiple strategies to the endpoint; as well as The SDN appliance offloads the application of the mapping policy to the second data packet to the programmable network device, wherein the programmable network device is configured to operate according to the response to the network protocol request.

8. The method of claim 7, wherein the same address prefix having multiple next hops is published to the network device.

9. The method according to claim 7, further comprising: Provides monitoring and availability signals for links attached to the programmable network device, wherein sessions can be initiated or withdrawn.

10. The method of claim 7, wherein the ARP response is generated by the SDN appliance.

11. The method of claim 10, wherein the ARP response is sent as if the response were sent from a network device link.

12. The method of claim 7, wherein the virtual address includes a virtual IP and a virtual MAC.

13. The method of claim 12, wherein the data packets are received by a server having a network interface operable to enable network connectivity for the network device.

14. The method of claim 7, wherein the network device is an FPGA or an ASIC.

15. A non-transitory computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by one or more processors of a computing device, cause the computing device to: Receive a first data packet including a network protocol request, the first data packet including a virtual address of the virtual computing environment; Based on the mapping of virtual addresses, it is determined that the virtual address of the received data packet corresponds to the MAC address of the programmable network device, the programmable network device is configured to process data traffic, and the programmable network device lacks a layer 3 network stack. Generate a response to the network protocol request, the response being based on the configuration of the programmable network device and configured to identify the programmable network device as the source of the response; The programmable network device sends the response on behalf of the programmable network device. Receive a second data packet addressed to an endpoint in the virtual computing environment via the programmable network device; Map one of the multiple strategies to the endpoint; as well as The application of the mapping policy to the second data packet is offloaded to the programmable network device, wherein the programmable network device is configured to operate in accordance with the response to the network protocol request.

16. The non-transitory computer-readable storage medium of claim 15, wherein the virtual address includes a virtual IP and a virtual MAC.

17. The non-transitory computer-readable storage medium of claim 15, wherein the data packets are received by a server having a network interface operable to initiate network connectivity for the network device.

18. The non-transitory computer-readable storage medium of claim 17, wherein the network device is an FPGA or an ASIC.

19. The non-transitory computer-readable storage medium of claim 15, wherein the identifier is executed by an agent executing on the computing device.

20. The non-transitory computer-readable storage medium of claim 15, wherein a route from a network interface on a server to the network device is published.

Images