Virtual network interface for managed layer 2 connectivity in computing service extension locations
The virtual network interface system with CVNIs and LVNIs addresses the challenge of seamless Layer 2 connectivity across cloud and on-premises environments, simplifying management and ensuring secure, efficient communication without intermediary devices.
Patent Information
- Application Number
- JP2024529745
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-11-29
- Filing Date
- 2022-11-15
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-11-15
AI Technical Summary
Existing technologies face challenges in providing seamless Layer 2 connectivity between compute instances in virtualized computing services across both cloud and on-premises environments without the need for intermediary networking devices, leading to complex configuration and management.
The implementation of a virtual network interface system that includes Cloud Access Virtual Network Interfaces (CVNIs) and Local Premises Access Virtual Network Interfaces (LVNIs) programmatically attached to compute instances, enabling Layer 2 connectivity through a control plane server, which manages IP and MAC addresses independently of physical network cards, allowing seamless communication between cloud and on-premises resources.
This solution simplifies network connectivity management by eliminating the need for intermediary devices, ensuring full connectivity and application isolation while maintaining security and performance, enhancing user experience for on-premises network administrators.
Smart Images

Figure 2024540761000001_ABST
Abstract
Description
[Background technology]
[0001] Many businesses and other organizations operate computer networks that interconnect numerous computing systems that support their operations, such as with computing systems that are co-located (e.g., as part of a local network) or located in multiple separate geographic locations (e.g., connected via one or more private or public intermediate networks). Data centers that house significant numbers of interconnected computing systems have become commonplace, including private data centers operated by and on behalf of a single organization, and public data centers operated as businesses by entities to provide computing resources to customers. Some public data center operators provide network access, power, and secure installation facilities for various customer-owned hardware, while others provide "full-service" facilities that also include hardware resources made available for use by customers.
[0002] The advent of virtualization technologies for commodity hardware has brought benefits in terms of managing large-scale computing resources for many customers with diverse needs, allowing various computing resources to be shared efficiently and securely among multiple customers. For example, virtualization technologies may allow a single physical virtualization host to be shared among multiple users by providing each user with one or more "guest" virtual machines hosted by the single virtualization host. Each such virtual machine may represent a software simulation that functions as a separate logical computing system that gives the user the illusion that he or she is the sole operator of a given hardware computing resource, and also provides application isolation and security among the various virtual machines. Instantiating several different virtual machines on the same host may also help increase the overall hardware utilization level in a data center, resulting in a higher return on investment. In some cases, the benefits of virtualization management technologies and associated resource manageability may be extended to premises other than the cloud provider's data center. [Brief description of the drawings]
[0003] [Figure 1] 1 illustrates an example system environment in which a local premises access virtual network interface used for Layer 2 communication in a managed open systems interconnection model, according to at least some embodiments, can be programmatically attached to a compute instance launched on an extended server of a virtualized computing service. [Diagram 2] 1 illustrates an example of the use of mapping services and encapsulation protocols in a virtualized computing service where a logical network is configured on a substrate network, according to at least some embodiments. [Diagram 3]FIG. 1 illustrates an example use of an enhanced traffic mediation device, according to at least some embodiments, to enable control plane traffic to flow securely from a data center of a provider network to an external premises where compute instances managed by a virtualized computing service are launched. [Figure 4] 1 illustrates an example network configuration for compute instances on an expanded server, according to at least some embodiments. [Diagram 5] 1 illustrates an example configuration in which compute instances configured within separate isolated virtual networks of a virtualized computing service in an extended server, according to at least some embodiments, may share a network cable to communicate with devices in a VCS extended premises. [Figure 6] 1 illustrates an example configuration in which several local premises access virtual network interfaces, each with its own security and performance-related properties, may be configured in an extension server of a virtualized computing service, according to at least some embodiments. [Figure 7] 1 illustrates example programmatic interactions associated with configuring and using a local premises access virtual network interface, according to at least some embodiments. [Figure 8] 1 is a flow diagram illustrating aspects of operations that may be performed to configure and utilize a local premises access virtual network interface according to at least some embodiments. [Figure 9] 1 illustrates example components of an enhanced server, according to at least some embodiments. [Figure 10] FIG. 1 is a block diagram illustrating an example of a computing device that may be used in at least some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0004] Although the embodiments are described herein by way of example in several embodiments and illustrative drawings, those skilled in the art will recognize that the embodiments are not limited to the described embodiments or drawings. The drawings and detailed description thereof are not intended to limit the embodiments to the particular disclosed forms; on the contrary, the intent is to cover all modifications, equivalents, and alternatives that are within the spirit and scope defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or claims. As used throughout this application, the term "may" is used in an permissive sense (i.e., meaning having the potential to), rather than a mandatory sense (i.e., meaning necessary). Similarly, the terms "include," "including," and "includes" mean to include, but not be limited to. When used in the claims, the term "or" is used as an inclusive or, not an exclusive or. For example, the phrase "at least one of x, y, or z" means any one of x, y, and z, and any combination thereof. Unless otherwise indicated, articles such as "a" or "an" should generally be construed to include one or more of the listed items throughout this application. Thus, phrases such as "a device configured to" are intended to include one or more of the listed devices. Such one or more listed devices may be collectively configured to perform the listed enumeration. For example, "a processor configured to perform enumerations A, B, and C" may include a first processor configured to perform enumeration A working in conjunction with a second processor configured to perform enumerations B and C.
[0005] The present disclosure relates to a method and apparatus for setting up premises outside a data center of a provider network and local devices at the external premises to enable managed Open System Interconnection Model Layer 2 (OSI L2) communication between compute instances of a Virtualized Computing Service (VCS) of a provider network or cloud computing environment. Such external premises may include customer data centers, manufacturing facilities, restaurants, etc., and owners of the external premises wish to run applications within VCS compute instances on local servers and also wish to implement applications and protocols that require OSI L2 connectivity between non-VCS devices (devices not managed or owned by VCS) located at the premises and the compute instances. The external premises are referred to as VCS extended premises because various types of features and functions supported by VCS in the data center of the provider network are extended to the external premises. The local servers on which the VCS compute instances run at the external premises are referred to as VCS extended servers or "outpost" servers.
[0006] Compute instances may be set up using commands sent over secure network paths established between the VCS control plane server (located in the provider network's data center) and the extension servers. Multiple types of virtual network interfaces, or VNIs (logical devices managed by the VCS and mapped to physical network interfaces by the VCS), including Cloud Access VNIs (CVNIs) and Local Premises Access VNIs (LVNIs), may be used to enable network connectivity for VCS compute instances. VNIs (which may also be referred to as "elastic network interfaces" or ENIs) allow various network-related attributes, such as IP (Internet Protocol) addresses and / or security settings governing inbound and outbound messages, to be easily transferred between compute instances without necessarily reconfiguring physical network cards. Such attribute transfer may be accomplished by programmatically detaching a VNI from one compute instance and programmatically attaching it to another compute instance, independent of the specific hardware network interface card (NIC) of the host on which the compute instance runs. For example, when a VNI is created or modified at the request of a VCS client or customer, metadata indicating the VNI's properties (such as IP address, Media Access Control (MAC) address, security rules, etc.) is stored or updated by a management component of the VCS.
[0007] By programmatically attaching a Cloud Access Virtual Network Interface (CVNI) to a compute instance of an extended server, the compute instance may be configured with an IP (Internet Protocol) address in an isolated virtual network (IVN) established in VCS on behalf of the client. As a result of the use of the CVNI, the compute instance running in the extended server has connectivity to other provider network resources (e.g., other compute instances implemented in virtualized servers in the provider network datacenter, resources in other provider network services such as storage services, database services, etc.) equivalent to the connectivity provided if the compute instance was launched in the provider network datacenter. A second type of virtual network interface, called a local premises access virtual network interface (LVNI), may also be programmatically attached to the compute instance of the extended server to provide OSI L2 connectivity managed using the VCS control plane. A MAC address is assigned to the LVNI by the VCS control plane, and this address may be utilized for bidirectional OSI L2 communication (including broadcasts, if necessary) between the compute instance and devices in the on-premise network set up by the customer, without the need for an intermediary networking device such as a router or gateway. By using a combination of LVNI and CVNI, compute instances set up on a VCS extended server can therefore access local on-premise resources (via OSI L2) and cloud-based resources (via IP) with equal ease, without administrators of the on-premise resources having to go to the trouble of configuring or managing intermediary networking devices for local traffic.
[0008] As one skilled in the art will appreciate in light of this disclosure, an embodiment may be capable of achieving various advantages including some or all of: (a) enabling programs running on VCS compute instances on extended servers to participate in applications and protocols running at OSI Layer 2 in the extended premises without having to configure intermediary devices, such as gateways or routers, in the extended premises while still maintaining full connectivity to data center resources in the provider network; and (b) improving the user experience of local network administrators of the extended premises, for example, by enabling configuration and management of both cloud access virtual network interfaces and local premises access virtual network interfaces using a common toolset.
[0009] According to some embodiments, the system may include one or more control plane servers (CPS) of a virtualized computing service (VCS) of a provider network and a networking manager running on an extension server (ES) of the VCS. The CPS may be located within one or more data centers of the provider network, and the ES may be located on a premises outside the data centers of the provider network, referred to as a VCS extension premises. The CPS, in various embodiments, may programmatically attach at least one pair of virtual network interfaces (VNIs) to compute instances (e.g., virtual machines) of the VCS running on the ES. The programmatic attachment process, in at least some embodiments, may involve sending a command or request from the CPS to a local management component of the ES. One of the VNIs, referred to as a cloud access VNI or CVNI, may be assigned an IP address from an IP address range of an isolated virtual network (IVN) configured in the VCS on behalf of VCS clients whose requests the extension servers are configured to serve, and may be primarily utilized to communicate with resources of the provider network's data centers, such as compute instances, storage servers of a storage service, database servers of a database service, etc. An IVN (also known as a virtual private cloud) may include a collection of networked resources (e.g., including compute instances) assigned to a given VCS client that are logically isolated from (and by default inaccessible to) resources assigned to other clients in other isolated virtual networks. The client on whose behalf the IVN is established may be granted significant flexibility regarding the network configuration for the IVN's resources.For example, a private IP address for a compute instance may be selected by the client without having to consider that other resources in other IVNs may be assigned the same IP address, that a subnet of the client's choice may be established in the IVN, that security rules may be set up by the client for inbound and outbound traffic on the IVN, etc. In at least some embodiments, the CVNI may be assigned an address from within a private IP address range selected by the client. In contrast to the CVNI, a second VNI programmatically attached to the compute instance, called a local premises access VNI (LVNI) or simply a local VNI, may not be assigned an IP address in the IVN. The CPS may assign a media access control (MAC) address to the LVNI, which in various embodiments may be used as a source or destination address for OSI data link layer (Layer 2) communications with non-VCS devices in the extended premises.
[0010] The networking manager of the extended server (which may be implemented, for example, as part of the virtualization management software and / or in a virtualization management offload hardware card of the extended server) may be responsible in various embodiments for determining how to respond to a data link layer frame or message received at the extended server. In some embodiments, the extended server may be assigned an IP address from within the VCS substrate network or underlying physical network, similar to how a VCS virtualization server in a provider network datacenter is assigned a substrate address. In response to determining that a data link layer frame received at the extended server includes a first IP packet having a substrate address as a destination address, the networking manager may utilize an encapsulation protocol of the VCS to extract a second IP packet from the first IP packet. If the destination address of the extracted IP packet matches an IP address assigned to a CVNI of a compute instance launched on the extended server (note that there may be multiple such compute instances, more than one of which may have a CVNI attached), the extracted IP packet may be delivered to that compute instance by the networking manager in various embodiments.
[0011] In contrast, if the received data link layer frame does not include an IP packet with a substrate destination address, the networking manager may attempt to determine whether the destination MAC address of the received frame matches a MAC address assigned to an LVNI of any of the compute instances running on the expansion server, and an encapsulation protocol may not be used. If an LVNI with a matching MAC address is found, the networking manager, in various embodiments, may deliver at least a portion of the contents of the received frame to the compute instance to which the LVNI is attached. The received frame may also be a broadcast frame (with a special destination MAC address), in which case, in at least some embodiments, the networking manager may deliver the contents of the received frame to each of zero or more compute instances (a) running on the expansion server, (b) to which an LVNI is attached, and (c) that are configured to accept L2 broadcast frames (note that in some embodiments, a configuration setting may be used to prevent delivery of broadcast frames to one or more compute instances or their attached LVNIs).
[0012] In some embodiments, LVNIs may be attached to compute instances of an expansion server as part of a workflow executed in response to a launch request for the compute instance. In other embodiments, a separate programmatic request may be submitted by a VCS client to create the LVNIs and / or programmatically attach the LVNIs to the specified compute instances (e.g., prior to launching the compute instances to which they are ultimately programmatically attached). Similarly, in some embodiments, CVNIs may be automatically attached to compute instances as part of the launch workflow, but in at least some embodiments, programmatic requests may also be submitted to create CVNIs and / or attach them.
[0013] In at least one embodiment, the VCS maintains a pool of MAC addresses from which individual MAC addresses are assigned to LVNIs, and to which the MAC addresses of decommissioned LVNIs are returned for reuse. In some embodiments, by default, an LVNI may be deleted or decommissioned when the compute instance to which it was attached ceases execution, and an indication that the MAC address of the LVNI is available for reuse may be stored in the VCS control plane when a request to terminate the compute instance is received (or when the compute instance is terminated for other reasons, such as an error or failure). Such a MAC address may later be reassigned to another LVNI by the VCS control plane, for example, prior to programmatically attaching that LVNI to another compute instance. In one embodiment, a VCS client may request that a MAC address that satisfies specified properties (e.g., matches a specified substring indicated by the client) be used for the LVNI to be attached to the client's compute instance. If the VCS can identify a MAC address that satisfies the client's request from among the set of MAC addresses that the VCS is permitted to use, that MAC address may be used. In this way, a VCS client may, at least in some cases, be able to influence the MAC address utilized for the client's LVNI.
[0014] According to at least some embodiments, the VCS may collect a respective set of metrics for each VNI attached to a given compute instance and provide LVNI metrics and CVNI metrics separately (if desired) via a programmatic interface to the client on whose behalf the VNI is being used. Such metrics may include, in different embodiments, for example, the total number of messages received / sent using each VNI, the total number of bytes received / sent, the distribution of inbound / outbound traffic among different IP and / or MAC addresses, etc. In at least some embodiments, a client may obtain the MAC addresses assigned to the client's LVNI using a programmatic request.
[0015] In various embodiments, data link layer traffic associated with an LVNI may be transmitted over a physical cable (e.g., an Ethernet cable) linking the extended server to an on-premise network switch (a part of the physical network set up in the extended premises and managed by the client, not the VCS). Other devices configured in the on-premise network may communicate with the compute instance to which the LVNI is attached using OSI L2 frames, and may include, for example, in different embodiments, Internet of Things (IoT) devices, industrial automation devices (e.g., components implementing a pipeline of automated tasks in a manufacturing facility), servers owned / managed by the VCS client on whose behalf the extended server is configured, etc. The VCS client, in various embodiments, may specify security settings for traffic associated with the LVNI (e.g., restrictions on sources from which frames may be received or destinations to which frames may be sent, which may be expressed using VLAN (Virtual Local Area Network) identifiers or tags to enable VLAN-based segmentation of local premises OSI Layer 2 traffic), and the networking manager may apply such security settings when determining the disposition of data link layer frames received at or transmitted from the extended server.
[0016] An LVNI (and / or CVNI) may, in various embodiments, be programmatically detached from a compute instance and reattached to a different compute instance while retaining properties such as a MAC address and / or IP address. The compute instance Cl2 to which an LVNI is attached after detachment from the compute instance Cl1 to which it was previously attached may run on the same expansion server or a different expansion server. In at least some embodiments, Cl2 may be configured (e.g., via its CVNI) as part of a different IVN than Cl1. In some embodiments, multiple LVNIs (and / or CVNIs) may be attached to a given compute instance. For example, multiple LVNIs may be attached to a given compute instance to facilitate segmentation of local premises OSI Layer 2 traffic by a network manager of the expansion server on which the compute instance is running, e.g., messages associated with different VLAN identifiers / tags may be handled using the respective LVNIs. After an LVNI with a given MAC address is detached from a compute instance, the networking manager, in various embodiments, may no longer be able to deliver frames with that MAC address designated as a destination to that compute instance. OSI L2 messages sent / received using the MAC address of the LVNI may, in some embodiments, include, among others, messages of Address Resolution Protocol (ARP), Domain Host Configuration Protocol (DHCP), or custom data link layer applications implemented within the extended premises' local network.
[0017] In at least some embodiments, the VCS may be implemented as one of a set of services in a provider network or cloud computing environment. A cloud provider network (sometimes simply referred to as a "cloud") refers to a pool of network-accessible computing resources (such as compute, storage, and networking resources, applications, and services), which may be virtualized or bare metal. A cloud may provide convenient on-demand network access to a shared pool of configurable computing resources that can be programmatically provisioned and released in response to customer commands. These resources can be dynamically provisioned and reconfigured to adjust to changing loads. Thus, cloud computing may be viewed as both applications delivered as a service over a publicly accessible network (e.g., the Internet or a cellular communications network), as well as the hardware and software in the cloud provider datacenters that provide those services.
[0018] A cloud provider network may be formed as several regions, where a region is a distinct geographical area in which a cloud provider clusters its data centers. Such regions may also be referred to as provider network-defined regions, since their boundaries may not necessarily coincide with the boundaries of a country, state, etc. Each region may include two or more availability zones interconnected via a private high-speed network, e.g., fiber communications connections. An availability zone (also called an availability domain, or simply a "zone") refers to an isolated failure domain that includes one or more data center facilities that have a separate power source, separate network, and separate cooling from another availability zone. A data center refers to a physical building or enclosure that houses and provides power and cooling to the servers of the cloud provider network. The availability zones within a region are preferably located far enough away from each other so that the same natural disaster does not take more than one availability zone offline at the same time. Customers may connect to the availability zones of the cloud provider network over publicly accessible networks (e.g., the Internet, cellular communications networks) via transit centers (TCs). TCs can be considered as the main backbone locations linking customers to the cloud provider network, and may be located in other network provider facilities (e.g., Internet service providers, telecommunications providers) and securely connected to availability zones (e.g., via VPN or direct connections). Each region may operate two or more TCs for redundancy. Regions are connected to a global network that connects each region to at least one other region. The cloud provider network may deliver content from points of presence that are outside of, but networked with, these regions via edge locations and regional edge cache servers (points of presence, or PoPs).This partitioning and geographic distribution of computing hardware enables cloud provider networks to offer customers global, low-latency resource access with a high degree of fault tolerance and stability.
[0019] A cloud provider network may implement various computing resources or services, which may include virtualized compute services (VCS), data processing service(s) (e.g., map-reduce, dataflow, and / or other large scale data processing techniques), data storage services (e.g., object storage services, block-based storage services, or data warehouse storage services), network function virtualization services or packet processing services, and / or any other type of network-based services (which may include various other types of storage, processing, analytics, communication, event handling, visualization, and security services). Resources (e.g., computational and storage resources) required to support the operation of such services may be provisioned in an account associated with the cloud provider, as opposed to resources that may be requested by a user of the cloud provider network and provisioned in a user account.
[0020] Various network-accessible services may be implemented in one or more data centers of a provider network in different embodiments. The network-accessible computing services may include elastic compute cloud services (referred to in various implementations as elastic compute services, virtual machine services, computing cloud services, compute engines, or cloud compute services). The services may provide compute instances (also referred to as guest virtual machines, or simply "instances") having varying computational and / or memory resources, which are managed by a compute virtualization service (referred to in various implementations as elastic compute services, virtual machine services, computing cloud services, compute engines, or cloud compute services). In one embodiment, each of the virtual compute instances may correspond to one of several instance types or families. An instance type may be characterized by its hardware type, computational resources (e.g., the number, type, and configuration of virtualized central processing units (VCPUs or VCPU cores)), memory resources (e.g., the capacity, type, and configuration of local memory), storage resources (e.g., the capacity, type, and configuration of locally accessible storage), network resources (e.g., the characteristics of its network interfaces and / or network capabilities), hardware accelerator resources, and / or other suitable descriptive characteristics (such as a "burstable" instance type that has a baseline performance guarantee and the ability to periodically burst above that baseline, or a non-burstable or dedicated instance type that is allocated and guaranteed a fixed amount of resources). Each instance type may have a particular ratio of processing, local storage, memory, and network resources, and different instance families may have different types of these resources as well. Within a given instance type, multiple sizes of these resource configurations may be available.The instance type selection function may be used to select an instance type for a customer based (at least in part) on input from the customer, for example. For example, a customer may select an instance type from a set of predefined instance types. As another example, a customer may specify desired resources for the instance type and / or requirements of the workload on which the instance will run, and the instance type selection function may select an instance type based on such specification. A suitable host for the requested instance type may be selected based at least in part on factors such as collected network performance metrics, resource utilization levels on different available hosts, etc. In some embodiments, instances of several different instance types may be launched (e.g., with LVNIs attached) in the extended premises in response to a programmatic request from a client.
[0021] The provider network's computing services may also include container orchestration and management services (referred to in various embodiments as container services, cloud container services, container engines, or container cloud services). A container represents a logical packaging of a software application that abstracts the application from the computing environment in which the application runs. For example, a containerized version of a software application includes the software code and dependencies used by that code, such that the application may be consistently executed on any infrastructure that hosts a suitable container engine (e.g., Docker® or Kubernetes® container engines). Compared to a virtual machine (VM), which emulates an entire computer system, a container virtualizes at the operating system level and thus typically represents a more lightweight package for running an application on a host computing system. An existing software application may be "containerized" by packaging the software application in an appropriate manner and generating other artifacts (e.g., container images, container files, or other configurations) that are used to make the application executable on a container engine. In some embodiments, the container engine may be executed on a virtual machine instance, which is selected based at least in part on the described network performance metrics. In some embodiments, other types of network-accessible services, such as packet processing services, database services, wide area networking (WAN) services, etc., may also be implemented in the cloud provider network.
[0022] Cloud provider network traffic and operations, in various embodiments, can be roughly subdivided into two categories: control plane operations, which occur on a logical control plane, and data plane operations, which occur on a logical data plane. The data plane represents the movement of user data through a distributed computing system, and the control plane represents the movement of control signals through a distributed computing system. The control plane generally includes one or more control plane components distributed across and implemented by one or more control servers. Control plane traffic generally includes management operations, such as system configuration and management (e.g., resource deployment, hardware capacity management, diagnostic monitoring, or system state information). The data plane includes customer resources (e.g., computing instances, containers, block storage volumes, databases, or file storage) implemented on the cloud provider network. Data plane traffic generally includes non-management operations, such as the transfer of customer data to and from customer resources. While some control plane components (e.g., tier 1 control plane components such as a control plane for virtualized computing services) are typically implemented on a set of servers separate from the data plane servers, other control plane components (e.g., tier 2 control plane components such as analytics services) may share virtualization servers with the data plane, and control plane traffic and data plane traffic may be transmitted over separate / distinct networks.
[0023] FIG. 1 illustrates an example of a system environment in which a local premises access virtual network interface used for Layer 2 communication in a managed open systems interconnection model may be programmatically attached to compute instances launched on an extension server of a virtualized computing service, according to at least some embodiments. As illustrated, the system 100 includes resources and artifacts of a virtualized computing service (VCS) 110 of a provider network 101. The VCS 110 may implement one or more programmatic interfaces 188 that may be used by clients to submit programmatic requests from client devices 185 (such as laptops, desktops, mobile computing devices, etc.), including, for example, requests to launch, terminate, or reconfigure compute instances 130. The programmatic interfaces 188 may include, for example, one or more web-based consoles, command line tools, graphical user interfaces, APIs, etc. Requests submitted by VCS clients are directed to a set of control plane servers 111 of the VCS, which may cause corresponding internal requests or commands to be implemented in various other components of the VCS.
[0024] In the embodiment shown in FIG. 1, various hardware servers may be used to set up compute instances 130 on behalf of VCS clients. An operator of VCS 110 may set up racks containing various categories of standardized servers in one or more data centers of provider network 101, including, for example, virtualization servers (VS) 125A, 125B, and 125C. Various categories of compute instances (CIs), such as CIs 130A, 130B, 130C, 130D, 130E, or 130C, may be launched on servers in the data center in response to client requests submitted to the control plane. Just as the capabilities of the underlying virtualization servers may differ from one another, the compute instances launched on different sets of servers may also differ in capabilities. For example, CI 130A may be optimized for computationally intensive applications, while CIs 130B and 130C may be optimized for storage-intensive tasks.
[0025] In addition to allowing clients to obtain and use compute instances at provider network datacenters, VCS 110 may also enable clients to request compute instances at premises outside the provider network in various embodiments. For example, in the illustrated embodiment, a logical extension of the VCS may be set up at client premises 121 in response to a request from a client (C2) that owns / manages premises 121. Thus, the client premises may be referred to as a VCS-extended premises in some embodiments. A group of VCS-managed resources, referred to as a VCS-extended resource group (ERG) 122, may be configured at client premises 121. ERG 122 may include one or more VCS extension servers (ES), such as ES 127 in the illustrated embodiment. A given ES may include zero or more compute instances (such as CI 130K), as well as various management components, such as networking manager 155 and ES configuration manager 154, which receives and coordinates responses to commands from the VCS control plane, in the illustrated embodiment.
[0026] Just as compute instances may be launched on VS 125 in response to client requests received on the VCS control plane, compute instances may be launched on the VCS expansion servers in the illustrated embodiment via a secure communication channel established between the control plane and the expansion servers. Additionally, the VCS control plane server may attach zero or more virtual network interfaces (VNIs) to a compute instance, such as CI 130K. A VNI may be represented, in various embodiments, as a set of metadata entries stored in the VCS control plane that indicate a set of network-related properties (e.g., IP addresses, MAC addresses, security rules for inbound / outbound traffic, etc.) that may be associated / disassociated with a compute instance without modifying the configuration of the physical networking devices (e.g., network interface cards, or NICs) of the server on which the compute instance is running.
[0027] VCS 110 may include a physical network 115, also referred to as a substrate network, to which, in the illustrated embodiment, virtualization servers may be connected. Links 111A, 111B, and 111C show physical connections between the VSs and substrate network 115. In the illustrated embodiment, tunneling-based techniques may be used to associate substrate network addresses with extension servers, such as ES 127, which association is represented by line 117. Thus, the VCS extension servers may be effectively integrated into substrate network 115.
[0028] In the embodiment shown in FIG. 1, a set of logical networks, such as logical network 116A or 116B, may be configured using substrate network 115 as the physical infrastructure. Examples of logical networks may include, at least in some embodiments, isolated virtual networks (IVNs) established on behalf of VCS clients. For example, in the scenario shown in FIG. 1, logical network 116A may include the IVN of client C1, while logical network 116B may include the IVN of client C2. In the illustrated embodiment, compute instance 130 may be provided with a network address in a logical network by a VCS control plane server by programmatically attaching a cloud access VNI (CVNI) to which the network address is assigned. Thus, for example, CI 130A of VS 125A may be configured in logical network 116A via CVNI 160A, as shown by element 112A. CI 130B of VS 125B may be configured in logical network 116B using CVNI 160B, as shown by line 112D. 1. CI 130C of VS 125B may be configured into logical network 116A using CVNI 160C, as shown by line 112C. CI 130D of VS 125C may be configured into logical network 116B using CVNI 160D, as shown by line 112E. CI 130E may be configured into logical network 116A using CVNI 160E, as shown by line 112B. In addition, the VCS control plane server may also configure CI 130K running on expansion server 127 as part of logical network 116B using CVNI 160F (which is assigned an IP address in logical network 116B), as shown by line 112F in the scenario shown in FIG. 1.To manage and route network traffic directed to and from logical network entities (such as compute instances 130) using the underlying substrate network to which the virtualization servers and extension servers are connected, mapping services and / or encapsulation protocols may be employed in VCS 110, as described in further detail below with respect to FIG. 2.
[0029] The client premises 121, in the illustrated embodiment, may include a local network 192 having a switch 144 or similar networking intermediary device. Various types of client-owned or client-managed devices, such as IoT devices including sensors or smart appliances, servers running client management applications, industrial automation devices, etc., may be configured as part of the client premises local network 192. Some such client premises devices 177 may be physically linked to the switch 144 via cables such as 118B.
[0030] In at least some embodiments, a client C2, in response to whose request an expansion server 127 is configured by the VCS, may wish to establish communications at OSI Layer 2 between a number of client premises devices 177 and the expansion server's CI 130K. To facilitate management of such communications, a local premises access virtual network interface (LVNI) 162 may be programmatically attached to the CI 130K by the VCS control plane in the illustrated embodiment. A MAC address may be assigned to the LVNI 162 by the VCS control plane server, but an IP address in logical network 116B may not be assigned to the LVNI. The MAC address may be used by networking manager 155 to direct inbound traffic received from local network 192, in the illustrated embodiment via physical cable 118A, to CI 130K. A logical network address assigned to CVNI 160F, in contrast, may be used in at least some embodiments to communicate with resources in a provider network data center. Note that ERG 122 may include one or more other compute instances (not shown in FIG. 1 ) running on ES 127 or another ES server. Network communications between such extended premises compute instances may be transmitted, in some cases, using OSI Layer 2 (e.g., utilizing a hardware loopback interface of a networking card attached to the ES) or using IP and other higher level protocols, depending on factors such as the application for which the communication is generated and whether the source and destination compute instances are running on the same ES. In some cases, a VCS encapsulation protocol may be employed for IP packets transmitted between at least some pairs of compute instances of an ERG.
[0031] When a data link layer frame is received at ES 127, networking manager 155 may be responsible for its disposition in the illustrated embodiment. The networking manager may inspect the frame's contents to determine whether the frame includes an IP packet with the substrate address of ES 127 as its destination address. If the frame includes such an IP packet, the networking manager may use the VCS encapsulation protocol to extract a second IP packet from the first IP packet. If the destination address of the extracted IP packet matches the logical network address (CVNI address) of any CI (such as CI 130K) running in the ES, the extracted IP packet may be delivered by the networking manager to that CI (otherwise, the extracted IP packet may be discarded in at least some embodiments). If the frame did not include an IP packet with a substrate destination IP address, the networking manager may inspect the frame's destination MAC address and deliver it to a CI, such as 130K, whose attached LVNI has the same MAC address in the illustrated embodiment. Broadcast data frames (which are special addresses whose destination MAC address does not match the MAC address of an LVNI) may be delivered by the networking manager to one or more CIs of an ES in the illustrated embodiment. Frames that are not broadcast frames, that do not have a matching MAC address, and that do not contain an IP packet with a substrate destination IP address may be dropped in the illustrated embodiment.
[0032] FIG. 2 illustrates an example of the use of mapping services and encapsulation protocols in a virtualized computing service where logical networks are configured on a substrate network, according to at least some embodiments. The illustrated scenario is presented to illustrate the types of networking-related operations that may need to be performed to enable traffic flow between compute instances set up in such logical networks (either in the VCS's data center or external premises). In FIG. 1, IP (Internet Protocol) version 4 addresses are used as examples, but in at least some embodiments, addresses formatted according to other protocols, such as IPv6, may be used. It should be noted that the specific IP addresses shown in FIG. 2 are chosen as examples, and the techniques described herein are not limited to any particular addresses or address ranges.
[0033] In the illustrated embodiment, two virtualization servers (VS) and an expansion server (ES) are shown in the VCS data center 210. VS 225A and 225B may each be physically connected to the VCS substrate network via one or more Ethernet or similar cables linked to a top-of-rack switch configured in the substrate network. ES 226 may be connected to the VCS data center via at least some network links not managed / owned by the provider network (e.g., links of the public Internet and / or private links owned by other network service providers), but may still be assigned a network address in the substrate network using techniques described in more detail below. Substrate address 192.168.0.3 is assigned to VS 225A, substrate address 192.168.1.3 is assigned to VS 225C, and substrate address 192.168.1.4 is assigned to ES 226.
[0034] In the illustrated embodiment, compute instances launched on the virtualization server may be assigned network addresses within the isolated virtual network. For example, CIs 230A (in VS 225A), 230D (in VS 225B), and 230G (in ES 226) may all be configured within the same IVN 233A and assigned IVN private addresses 10.0.0.2, 10.0.0.4, and 10.0.0.3, respectively. Similarly, CIs 230B and 230C may be assigned IVN private addresses 10.0.0.2 and 10.0.0.3 within IVN 233B. Note that, as previously indicated, address ranges used within an IVN for private addresses assigned to CIs may overlap with each other. Thus, CIs 230A and 230B have the same private address 10.0.0.2 within separate IVNs (233A and 233B). In some embodiments, intra-IVN addresses may be considered private in that they are not published or made accessible outside the IVN, at least by default. In at least some embodiments, as described above, private addresses may be assigned to each cloud access virtual network interface (CVNI), which may be programmatically attached or associated with a compute instance. In at least some embodiments, in contrast, at least some of the substrate addresses may be assigned to a physical network interface card, i.e., a NIC (or NIC emulator), e.g., in a virtualization server.
[0035] To send a network packet originating on one CI to another CI, three types of network information may need to be considered in the illustrated embodiment: the source and destination IVN private addresses, the IVN to which the source and destination belong, and the substrate address of the underlying virtualization server. For example, a packet originating on CI 230A and destined for CI 230G may show its source (private) address as 10.0.0.2 and destination address as 10.0.0.3. However, the packet may actually have to be forwarded from substrate network address 192.168.0.3 to substrate network address 192.168.1.4 to reach its intended destination. An encapsulation protocol 244 (used to envelope or encapsulate packets associated with logical network sources / destinations within larger "extended" packets associated with substrate network sources / destinations) and the associated mapping service 245 of the VCS may be used to accomplish this type of forwarding in the illustrated embodiment. The networking virtualization management components of the VCS (including the networking manager running on the expansion servers and the networking manager running on the virtualization management hardware / software stack of the VS225) perform protocol encapsulation and decapsulation operations and may utilize the mapping service 245 to determine the specific substrate addresses to which packets involved in such transfers should be sent.
[0036] In the above example where a packet is sent from CI 230A to CI 230G, mapping service 245 may indicate to a networking manager associated with VS 225A that for IVN 233A, destination private address 10.0.0.3 corresponds to substrate address 192.168.1.4. The networking manager associated with VS 225A may generate an encapsulation packet containing the original packet inside, having substrate source address 192.168.0.3, substrate destination address 192.168.1.4, and identifying IVN 233A as the IVN within which the packet is being forwarded. On the receiving side of external premises 220, the networking manager running in ES 226 may extract (de-encapsulate) the original packet from the encapsulation packet and provide it to destination CI 230G. In some embodiments, to ensure that a packet is from a trusted / valid source, the networking manager may query a mapping service to perform a reverse mapping (e.g., identify the source of the packet) before extracting the original packet. Thus, the mapping service 245 may provide security by preventing the opening of packets that have not been validated. For packets sent in the reverse direction, in at least some embodiments, the networking manager may query a mapping service to obtain the correct substrate address of the destination and perform the necessary encapsulation operations.
[0037] 3 illustrates an example of the use of an enhanced traffic mediation device to enable control plane traffic to flow securely from a provider network data center to external premises where compute instances managed by the virtualized computing service are launched, according to at least some embodiments. Data plane resources 345 of a VCS 310 located in a provider network data center 301 may be organized into a set of isolated virtual networks (IVNs) similar to those described above. For example, IVN 315A may be set up on behalf of client C1, IVN 315B may be established on behalf of client C2, etc., with clients being given flexibility regarding network configuration settings within each IVN. IVN 315A may include compute instances running on virtualization servers 317A and 317B, and IVN 315B may include compute instances (CIs) (e.g., CI 325A) launched on virtualization servers 317J and 317K. The VCS control plane resources 341 may include a set of control plane servers (CPS) 302, such as 302A and 302B in the illustrated embodiment, that are responsible for management tasks such as receiving and responding to requests to set up VCS extended resource groups (ERGs), instance configuration requests in data center virtualization servers or extended servers 360, provisioning additional virtualization servers, and monitoring the health of data plane resources.
[0038] In the embodiment shown in FIG. 3, each extension server 360 may be set up as part of an extension resource group that extends VCS data plane functionality at some premises other than data center 301. For example, extension server (ES) 360A may be set up as part of ERG 335A at customer premises 332C of client C2, ES 360B may be set up as part of ERG 335B at customer premises 332A on behalf of client C1, and ES 360C may be set up in ERG 335C at a second customer premises 332B of client C2. Each ES may include a respective configuration manager (ECM) (such as ECM 361 of ES 360A) that is responsible for receiving management commands from control plane resources 341 and executing them at the ES. In addition to ES 360, at least some of external premises 332A, 332B, and 332C may also include non-VCS devices (e.g., servers not utilized to host VCS compute instances). For example, in the illustrated embodiment, premises 332A may include non-VCS device 323B, premises 332B may include non-VCS device 323C, and premises 332C may include non-VCS device 323A. The non-VCS devices may communicate with compute instances (such as CI 325P, 325Q, or 325R) launched on the ES using local premises access VNIs (LVNIs) and OSI Layer 2 messages sent over links in local premises networks 368A, 368B, or 368C. Thus, an extension server may be configured in multiple networks, including a local on-premise network and one or more VCS networks (e.g., using addresses assigned to the extension server from the range used for the VCS substrate network). It should be noted that the various capabilities of ES 360 may differ from one another depending on the needs of the clients on whose behalf ES 360 is configured by the VCS control plane.For example, ES360A may have a different or more CPU than ES360B, ES360C may have more memory or a different CPU architecture than ES360A, etc. Also, note that while only a single ES is shown in each external premises in Figure 3, in at least some embodiments, multiple such ESs may be deployed and used to host VCS compute instances within a single external premises.
[0039] From the perspective of a client of the VCS, in various embodiments, an ERG may represent a local extension of the VCS capabilities and may be set up in any desired physical location with Internet access and accommodate extension servers that are acceptable to the VCS and its clients. From the perspective of the VCS itself, an ERG may be considered to be physically located in a customer-selected premises, while being virtually located in the same provider network data center as the core VCS infrastructure. In various embodiments, once an ERG is set up in a customer-selected location, the ERG's resources may be managed by a control plane component of the VCS located in the provider network's data center. Thus, in at least some embodiments, setting up and using an ERG at a given premises may not require replicating the VCS's control plane capabilities locally. Instead, a secure network connection may be set up to send control plane commands from the provider network's data center to the ERG's PVMD, and the ERG's resources may be devoted primarily to data plane operations.
[0040] In some embodiments, one or more of the compute instances (CIs) launched on the ESs of the ERG may be configured in the IVN 315 (e.g., the CIs may be programmatically attached to a cloud access virtual network interface (CVNI) having an address in the address range of the IVN, the CIs may be included in the database of compute instances of the IVN, information about the membership of the CIs in the IVN may be provided to a mapping service of the VCS, etc.). In some cases, the CIs running in the ERG may be included in an IVN that also includes CIs in the provider network datacenter, and such an ERG may be said to extend the IVN. For example, ERG 335A extends IVN 315B for client C2 of FIG. 3, and ERG 335B extends IVN 315A configured for client C1. Some ERGs may include CIs configured in an IVN that does not include any compute instances in the VCS datacenter, or may not even be configured in an IVN.
[0041] Because external premises 332 do not have direct access to the VCS substrate network, additional work may be required from the VCS to configure compute instances at the external premises than is required for compute instances in data center 301. In the embodiment shown in FIG. 3, ES configuration manager (ECM) 361 may perform several functions. For example, the ECM may establish a connection with one or more enhanced traffic intermediations (ETIs) 377 of the VCS over one or more secure channels (such as virtual private network (VPN) tunnels) 366 (e.g., channels 366A, 366B, or 366C). Secure channel 366 may be required because traffic between the ECM and the VCS control plane (and data plane) must flow over physical network paths that are not controlled or managed by the VCS, and therefore such physical network paths may not be trusted from the VCS perspective. The ETI 377 may be implemented in a computing device in the provider network data center 301 (e.g., using one or more compute instances and / or IVNs set up on behalf of the VCS control plane), and the ETI may be configured to perform one or more security operations regarding traffic between the external premises and the provider network data center. Control plane commands may be allowed to flow only in one direction, from the provider network data center to the external premises, in the illustrated embodiment, using the secure channel established with the ETI. In contrast, data plane traffic may flow in either direction between the external premises and the provider network in various embodiments. Additional details regarding how control plane and data plane traffic flows between the provider network data center and the external premises in various embodiments are provided below.
[0042] The ECM of the ES 360 may also, in at least some embodiments, assign the ES a network address in the VCS substrate network (i.e., a network address selected from the range of addresses assigned to virtualization servers in the VCS substrate network). The substrate address assigned to the ES may also, in at least some embodiments, be assigned to one of the ETIs 377, allowing the ETI to act as a proxy or representative of the ES within the provider network data center.
[0043] Additionally, the ECM may, in at least some embodiments, act as an intermediary for configuration commands related to CIs initiated at the ES 360 in the ERG 335. For example, a client of a VCS may submit a request for a configuration operation of a CI in an ERG to the VCS control plane (using a path other than the secure channel set up with the ETI), and an indication of the request may be provided from the ETI to the ECM of that ERG. In response to the indication, one or more configuration operations of the compute instances may be executed in the CSS. In at least some embodiments, the ECM may be instantiated in response to detection of one or more trigger conditions in the ES (e.g., detection of power and / or Internet connectivity). The ECM may then automatically initiate (or at least participate in) the automatic establishment of a secure network connection with one or more VCS components (e.g., ETI 377) of one or more provider network data centers, for example, without requiring additional configuration guidance from a VCS client on whose behalf the ES is configured. After the connection is established, in various embodiments, the client may issue commands to instantiate the compute instance in the ES (and / or perform other operations related to the compute instance, such as attaching an LVNI) in a manner similar to how such commands are issued for a compute instance that uses only provider network resources. From the perspective of the VCS client, the functionality of the VCS may now be seamlessly utilized using local resources (and resources located in the provider network datacenter, if desired). Compute instances set up in the ERG, in various embodiments, may communicate with non-VCS devices 323 and other CIs set up in the provider network datacenter or external premises, if desired.At least some of the CIs set up in an ERG, and associated upstream services that use such CIs as building blocks, may, in some embodiments, continue to function even during periods when connectivity to the provider network datacenter is temporarily interrupted. In particular, for VCS customers who wish to access and process large amounts of application data stored in the customer's datacenter (e.g., for regulatory compliance, security, or other reasons) with low latency, the ability to set up VCS CIs that are co-located with the application data may be highly beneficial in various embodiments.
[0044] In the provider network data center 301 in which the VCS 310 is implemented, the control plane resources 341 may include, in the illustrated embodiment, one or more extended connection managers (EXCMs) 378 (implemented using some combination of hardware and software in one or more servers of the VCS). The EXCM may determine that a connection over a secure channel should be established with an ECM, such as ECM 361, for example, in response to a programmatic request received in the control plane. In some embodiments, the ECM may send a request for a secure connection to the VCS control plane (e.g., over a public Internet path) once the ECM comes online and is connected to the Internet via the external premises' local premises network. To enable the secure connection, a set of extended traffic intermediaries (ETIs) 377 may be set up by the EXCM. The ETIs 377 may include, for example, extended server proxies (ESPs) and tunnel endpoints (referred to as provider network-side tunnel endpoints) in various embodiments. In some embodiments, the ESPs and tunnel endpoints may include respective compute instances of the VCS. In at least one embodiment, the ESP may be set up within one IVN established on behalf of the VCS control plane (different from the IVN set up on behalf of the VCS client), but the tunnel endpoint may be set up within a different IVN established for the VCS control plane.
[0045] In some embodiments, a virtual network interface (VNI) may be programmatically attached to the ESP by the EXCM. In at least some embodiments, a VNI attached to an ESP may be assigned the same substrate network address assigned to the ES 360 for which the ESP is configured. In at least some embodiments, the ECM may receive an indication of the substrate network address from the EXCM and assign that address to the ES 360. The ECM may act as the external endpoint of a secure (e.g., encrypted) network tunnel established with the provider network-side tunnel endpoint. Any of a variety of tunneling protocols (e.g., standard VPN protocols, customized protocols utilized within the VCS, etc.) may be used in various embodiments to transmit packets between the provider network and the external premises over a potentially untrusted network.
[0046] Because the ESP and ES are assigned the same substrate network address, the VCS control plane, in the illustrated embodiment, may use the ESP as the destination for control plane commands (e.g., configuration commands generated in response to client-issued requests) that are actually intended for the ES. Thus, from the perspective of the VCS control plane, the ES may appear as just another virtualization server that may be accessed via the VCS's substrate network. When a control plane command is generated, it may be received at the ESP, and a version of the command may be sent over channel 366. In some embodiments, the ESP and / or the provider network-side tunnel endpoint may apply one or more security-related transformation operations to the command before sending the command to the ECM. For example, the version of the command obtained at the proxy from the VCS control plane may, in some embodiments, include one or more security tokens (e.g., a token that can be used to verify the identity of a requester of an action to be performed as a result of the command, and that the requester has the necessary authorization to request that action), and the ESP may strip or exclude the tokens from the version of the command forwarded to the ECM in the ERG. In at least some embodiments, the original security token may be transformed, and the transformed version of the security token may be included in the forwarded version of the command. In at least one embodiment, a respective message authentication code (e.g., including a hash-based message authentication code, or HMAC) may be generated for each outbound control plane command sent from the ESP to the ES. In various embodiments, the ESP may log all outbound communication messages sent to a given ECM, and the logged messages may be inspected by the client for which the ES is set up, if necessary.In some embodiments, at least two virtual network interfaces may be associated with a given ESP, one used to receive commands from the VCS control plane and one used to communicate with the tunnel endpoints.
[0047] In some embodiments, data plane traffic may also flow between the VCS and the external premises, e.g., through a secure channel or tunnel separate from the channel used for control plane traffic. Data plane traffic may flow in either direction through such a tunnel and may not involve the use of a proxy. In one embodiment, a VNI with the same substrate address as the address assigned to the ES may also be set up in the VCS for data plane traffic to / from the ES.
[0048] 4 illustrates example network configurations for compute instances in an expansion server, according to at least some embodiments. Depending on the particular application that a VCS client wants to run on a given compute instance, in the illustrated embodiment, any of at least three network configurations can be set up for a given CI by the VCS control plane.
[0049] CI 410 represents a first network configuration example in VCS extension server 401, where a set of applications running on the CI communicate with resources located in a provider network data center and also communicate with resources of other devices in the local premises network. CI 410 has an attached CVNI 415A and an attached LVNI 417A. OSI layer 2 messages 423A to / from devices in the local premises network use MAC addresses assigned to LVNI 417A as source / destination MAC addresses. Data plane messages 422A to / from the provider network data center use IP addresses assigned to attached CVNI 415A as source / destination IP addresses. Management commands 421A from the VCS control plane (including, for example, commands to start / stop the CI or to change various configuration settings of the CI) can also be delivered to CI 410 using an intermediary, such as a secure channel and an ECM of the type described above.
[0050] In a second type of network configuration, represented by CI 412, a CI running in an ES may not have an attached LVNI, but may have an attached CVNI 415B. Management commands 421B from the VCS control plane may still be received in CI 412, and data plane messages 422D using the IP address of the CVNI may be exchanged with resources in the provider network data center, but OSI Layer 2 messages using the MAC address of the CVNI may not be exchanged. In some embodiments, data plane messages between a CI having the configuration of CI 412 and a CI having the configuration of CI 410 may be forwarded through the provider network data center or through a local networking intermediary device set up in the extended premises. In at least one embodiment, as an optimization, data plane messages between CI 412 and CI 410 may be sent using a hardware loopback mechanism implemented by the networking manager of the ES, as indicated by arrow 477.
[0051] CI 414 of FIG. 4 is an example of a third type of network configuration. CI 415B has CVNI 415B and LVNI 417B attached to it, but security parameters of CVNI 415B (and / or the IVN with an address assigned to CVNI 415B) are set to prohibit data plane traffic to / from the provider network data center. OSI Layer 2 messages 423B may be exchanged with devices in the local premises network using a MAC address assigned to LVNI 417B, and management commands 421C from the VCS control plane may be received at CI 414. The type of configuration shown for CI 414 may be utilized in embodiments where, for example, (a) a CVNI must be attached to a CI based on VCS configuration rules, e.g., to receive control plane commands, (b) a client wishes to use the CI for an application that requires OSI L2 connectivity to a non-VCS device in the premises, and (c) a client wishes to isolate the CI from VCS data plane traffic. In one embodiment, if a client wishes to enable communication at OSI Layer 2 between CI 414 and CI 410 (both CIs have LVNIs attached), a hardware loopback mechanism implemented by the networking manager of the ES may be used, as indicated by arrow 478. In some embodiments, other types of network configurations than those shown in Figure 4 may be employed for the CIs of the expansion servers.
[0052] In some embodiments, CIs belonging to separate IVNs may be launched in a given VCS extended server. Figure 5 illustrates an example configuration in which compute instances configured in separate isolated virtual networks of a virtualized computing service in an extended server may share a network cable to communicate with devices in a VCS extended premises, according to at least some embodiments. VCS extended server 501 includes CIs 510A and 510B, and a networking manager 555. CI 510A is used by a VCS client to run application set 561A, and CI 510B is used to run a different application set 561B. A VCS client for which extended server 501 is set up may want to isolate the two application sets while still allowing OSI L2 communication between both application sets and a (potentially separate) set of on-premises devices.
[0053] Thus, in the illustrated embodiment, CI 510A may be configured into IVN 544A by assigning IP address 525A of IVN 544A to CVNI 515A attached to CI 510A, and CI 510B may be configured into a different IVN 544B by assigning IP address 525B of IVN 544B to CVNI 515B attached to CI 510B. To enable OSI L2 connectivity, a shared network cable 570 (e.g., an Ethernet cable) may be used to link the physical network interface of VCS expansion server 501 to the on-premise local network switch 544, and each LVNI 517 (e.g., 517A or 517B) may be attached to two CIs. OSI L2 frames 507A may be exchanged between CI 510A and local network devices via network cable 570, and OSI L2 frames 507B may also be exchanged between CI 510B and local network devices using the same network cable 570, with the networking manager acting as an intermediary (logically similar to a switch) for both sets of frames.
[0054] In the illustrated embodiment, a variety of non-VCS devices may be attached to the local network switch 544 via respective network cables such as 573A, 573B, and 573C, some or all of which may communicate at OSI Layer 2 with one or more CIs executed in the ES 501. Such devices may include, among others, an on-premise DHCP server 575, a non-VCS general purpose server 571, and / or one or more IoT devices 572. Note that in some embodiments, multiple network cables may be attached between the extended servers and devices of the local network (such as a network switch). For example, one such cable may be used to communicate with provider network datacenter resources and the VCS control plane via tunneling through the public Internet, and another cable may be used for OSI Layer 2 traffic with the local network.
[0055] In some embodiments, the VCS client may decide to apply custom traffic management policies to local premises traffic of LVNIs attached to different compute instances in a given expansion server. FIG. 6 illustrates an example configuration in which several local premises access virtual network interfaces with respective security and performance related properties may be configured in an expansion server of a virtualized computing service, according to at least some embodiments. In the example scenario illustrated in FIG. 6, a VCS expansion server 601 includes a pair of compute instances CI 610A and CI 610B. CI 610A is used for application set 661A and has CVNI 615A and LVNI 617A attached. CI 610B is used for application set 661B and has CVNI 615B and two LVNIs 617B and 617C attached. An IP address 625A within the IVN of the VCS is assigned to CVNI 615A, and an IP address 625B within the IVN (which may be the same IVN as the address assigned to CVNI 615A or a different IVN) is also assigned to CVNI 615B.
[0056] The VCS clients on which the CIs 610 and LVNIs 617 are configured are configured to apply different sets of security and performance related rules to the respective OSI Layer 2 traffic using the MAC addresses of the LVNIs in the illustrated embodiment. The respective security settings 644A, 644B, and 644C may be specified by the clients via VCS control plane programmatic interfaces for the LVNIs 617A, 617B, and 617C, respectively. In addition, the respective traffic rate limits 645A, 645B, and 645C may be specified by the clients via VCS control plane programmatic interfaces for the LVNIs 617A, 617B, and 617C, respectively. The security settings may specify properties such as, for example, whether broadcast frames received by the networking manager should be sent to the compute instance to which the LVNI is attached, whether frames with a specified VLAN tag or identifier should be delivered to the compute instance, etc. In the illustrated embodiment, traffic rate limits may be used to throttle outbound OSI Layer 2 traffic from a given CI (or a given LVNI of a given CI). Recall that in some embodiments, a shared network cable (e.g., to a switch as shown in FIG. 5) may be used to connect the expansion server with the rest of the local premises network, and such rate limits may be useful to ensure that the available bandwidth on such a cable is divided according to the VCS client's intent. Security settings and rate limits obtained through a programmatic interface from a client at the VCS control plane server may be sent to an expansion server's expansion configuration manager (ECM) and / or the expansion server's networking manager, and in at least some embodiments, the settings may be applied or enforced by the networking manager. Other LVNI parameters other than those shown in FIG. 6 may also or instead be customized by the VCS client in some embodiments.
[0057] FIG. 7 illustrates example programmatic interactions associated with configuration and use of a local premises access virtual network interface, according to at least some embodiments. The VCS 712, which is similar in features and functionality to the VCS 110 of FIG. 1, may implement a set of programmatic interfaces 777, such as one or more web-based consoles, command line tools, graphical user interfaces, application programming interfaces (APIs), etc., in the embodiment illustrated in FIG. 7. A client 710 of the VCS may submit a ConfigureExtensionServer request 714 to the VCS control plane via the programmatic interfaces to request the establishment of at least one extension server of the type previously described at a specified premises outside the provider network data center. After the requested server is provisioned and online, in the illustrated embodiment, an ExtensionServerOnline response message 716 may be sent to the client. Note that in some cases, it may take some time (e.g., days or weeks) to ship or transmit the necessary server components from the provider network to the external premises, then configure the servers (e.g., in a standard server rack), attach the servers to the Internet, and power on the servers, and the ExtensionServerOnline message may be sent only after all these tasks are completed, at least in some embodiments. As described above, the extension server configuration manager at the external premises may, at least in some embodiments, establish a connection over a secure channel with an extension connection manager in the VCS control plane, located in the provider network's data center, as part of the initial configuration of the extension server.
[0058] In the illustrated embodiment, a client 710 may request the creation of one or more LVNIs via a CreateLVNIs request 718 and one or more CVNIs via a CreateCVNIs request 723. In some embodiments, such LVNIs and / or CVNIs may be created prior to launch of the compute instance to which they are programmatically attached. In response, metadata indicating properties of the requested LVNI / CVNI (e.g., name, description, IP address if specified / requested by the client, MAC address if specified / requested by the client, security rules, traffic rate limits if applicable, etc.) may be stored in the VCS control plane. An LVNI-IDs message 720 may be sent to the client in some embodiments indicating successful creation of the requested LVNI and providing an identifier of the created LVNI that may be used in subsequent interactions as needed. A CVNI-IDs message 725 may be sent to the client after the requested CVNI is created to indicate that the requested CVNI has been created and to provide an identifier of the created LVNI that may be used in subsequent interactions as needed.
[0059] In the illustrated embodiment, a compute instance may be launched on an extension server in response to a LaunchCIAtExtensionServer request 727. Parameters of the launch request may govern how VNIs (e.g., one or more CVNIs and / or LVNIs) are programmatically attached to the CI. In some cases, the client may request that the VCS control plane create and attach at least one CVNI within a specified IVN at instance launch time and / or that the VCS create and attach at least one LVNI with a VCS-selected MAC address at instance launch time. In other cases, the client may postpone attaching one or more VNIs until the instance is launched and a CILaunched message 728 is received at the client.
[0060] In the illustrated embodiment, a client may request programmatic attachment of one or more VNIs (e.g., LVNIs or CVNIs or both) to a CI in an extension server by submitting an AttachVNIToExtensionServerCI request 730 to the VCS. Optionally, the client may specify an identifier for the VNI (e.g., CVNI or LVNI) to be attached or may request that the VCS control plane create and attach a new VNI. In the illustrated embodiment, after metadata indicating the association of the VNI with the CI is stored in the VCS control plane and propagated to the extension server, a VNIAttached message 732 may be sent to the client.
[0061] In the illustrated embodiment, an LVNI or CVNI may be programmatically detached from a CI by the VCS control plane in response to a DetachVNIFromExtensionServerCI request 734. Such detachment may include removing an indication of the association of the VNI with the CI to which it is attached, while retaining at least some of the other properties of the VNI, such as a MAC address and / or IP address. A VNIDetached message 736 may be sent to the client to indicate that the VNI has been disassociated from the CI.
[0062] In the illustrated embodiment, a DescribeVNIs request 741 may be submitted by a client 710 to see the properties of a specified set of VNIs (e.g., LVNIs, CVNs, or both types of VNIs) and / or all VNIs attached to a CI. The properties of the VNI(s) may be presented to the client via one or more VNInfo messages 743. For example, the client may use the DescribeVNI request to obtain the MAC address of the LVNI (before or after attaching the LVNI to a CI) and may use that information to make configuration changes in the on-premises network such that OSI L2 traffic from the on-premises network is exchanged with the CI to which that LVNI is attached.
[0063] As discussed above, for example in the context of Figure 6, a client may wish to customize various properties, such as security settings or traffic rate limits, of the LVNI (and / or CVNI). In some embodiments, a ChangeVNISettings request 747 indicating the desired settings may be submitted by the client using a programmatic interface 777. Metadata stored for the VNI may be modified accordingly, and a VNISettingsChanged response message 749 may be sent to the client.
[0064] A VCS, in different embodiments, may collect various types of metrics associated with particular CVNIs and LVNIs, such as the rate at which messages are received / sent using addresses assigned to the VNIs, the distribution of message sizes, trends in traffic rates, etc. A client may request metrics for LVNIs and / or CVNIs, e.g., per VNI type or per VNI, in the illustrated embodiment, by submitting a GetVNIMetrics request 751. The requested metrics may be provided in one or more MetricsSet response messages 753.
[0065] In some embodiments, an LVNI may be treated (by the VCS control plane) as a special case of a more general VNI construct supported by VCS. In such an embodiment, if a client 710 requests the creation or attachment of a VNI without specifying that the VNI is to be used as an LVNI, the VCS may assume by default that the VNI is to be used as a CVNI (e.g., that the VNI is assigned an IVN address, etc.). In an embodiment in which such a general VNI construct is supported, the VCS client 710 may use an IVN subnet configuration setting to indicate that an IP address (i.e., a CVNI IP address) is to be configured for a CI that is within the IP address range of that subnet. The VCS client may submit a SetSubnetAttributesForLVNIs request 755 to indicate that for a compute instance configured within a specified subnet of a specified IVN, any VNI at a specified integer interface index in an array or list of VNIs attached to that CI is to be used or treated as an LVNI. In response, the VCS may store the LVNI index as an attribute of the subnet and treat any subsequently attached VNIs at that index (e.g., in response to an attach request that specifies the index) as an LVNI. A SubnetAttributesSet message 757 may be sent from the VCS to the client. The subnet attribute method for configuring LVNIs may simplify management of LVNIs for VCS clients, since clients may simply use the same types of programmatic requests for LVNIs as are used for other VNIs without using LVNI-specific parameters (as long as the correct interface index is used for the LVNI). Note that programmatic interactions related to the use of LVNIs, CVNIs, and extended servers other than those shown in FIG. 7 may be supported by the VCS in at least some embodiments. Additionally, FIG. 7 is not intended to show the necessary sequence in which programmatic interactions must occur.For example, although the DescribeVNI request is shown below the LaunchCIAtExtensionServer request in FIG. 7, the DescribeVNI request may be submitted before the LaunchCIAtExtensionServer request.
[0066] FIG. 8 is a flow diagram illustrating aspects of operations that may be performed to configure and utilize a local premises access virtual network interface, according to at least some embodiments. As shown in element 801, an extension server (ES) of a VCS, similar in features and functionality to the VCS 110 of the provider network 101 of FIG. 1, may be configured on a premises outside the data center of the provider network, in the illustrated embodiment. Configuring the ES may include establishing a connection over a secure channel from a control plane server (CPS) of the VCS to an ES configuration manager (ECM) running on the ES. The ECM may be responsible, in various embodiments, for launching compute instances (CIs) on the ES in response to requests / commands from the VCS control plane, among other tasks. The ES may also include a networking manager, in various embodiments, responsible for managing the flow of network traffic between CIs launched on the ES and communication endpoints external to the ES, and between CIs of the ES.
[0067] In response to a request from a VCS client where an ES is being set up, the VCS control plane may bring up a CI, CI1, in the ES. In the illustrated embodiment, a local premises access virtual network interface (LVNI) may be programmatically attached to CI1 by the VCS control plane, and a cloud access virtual network interface (CVNI) may also be programmatically attached to the CI (element 804). In at least some embodiments, a control plane server in the VCS may receive a request to attach an LVNI or CVNI to a CI from a client via a programmatic interface, while the control plane server may send an internal request / command to a configuration management component or networking manager in the ES, which completes the attach operation. The CVNI may be assigned an IP address IPAddr1 from the address range of an isolated virtual network (IVN) configured in the VCS. The LVNI may be assigned a MAC address M1 by the VCS control plane, but not from within that IVN or from any range of IP addresses managed by the VCS. In at least some embodiments, the networking manager may be provided with configuration information (including, for example, IP addresses and MAC addresses) associated with the attached CVNIs and attached LVNIs from the VCS control plane.
[0068] In the ES, the networking manager may utilize the VCS's encapsulation protocol to translate outbound IP packets from CI1 that have a destination address within the VCS (or within other services in the provider network) (element 807). Outbound data link layer frames that have a destination MAC address within the premises' local network (or have a destination address used for broadcast) may be directed by the networking manager to a network switch or other similar networking intermediary device in the premises via a cable connecting the ES to the intermediary device.
[0069] An inbound data link layer frame DLLF may be obtained (element 810) at a physical networking interface (e.g., a network interface card or equivalent) of the ES and processed by the networking manager. In some implementations, the DLLF may have associated metadata (e.g., an indication of the source queue in which it is placed by the lower driver software or hardware networking components) that indicates whether the source from which the DLLF was received was a provider network resource or whether it was received from a device local to the premises where the ES is located. In such implementations, such metadata may first be checked by the networking manager to determine the type of source from which the DLLF was sent, and then the information about the source may be used along with the contents of the DLLF to determine how the DLLF should be processed. For example, the VCS encapsulation protocol may be used, as described below, only if the DLLF was received from a provider network resource. If the DLLF includes an IP packet IPP1 having a destination address SDA that is a VCS substrate network address previously assigned to the ES (e.g., during initial configuration of the ES), as determined in an operation corresponding to element 813, the networking manager may extract a second IP packet IPP2 encapsulated within IPP1 using the encapsulation protocol of the VCS (element 819). If the destination IP address of IPP2 matches an IP address assigned to the CVNI of any CI running in the ES, such as IP address IPAddr1 assigned to the CVNI of CI1, the networking manager may deliver the contents of the second IP packet IPP2 to that CI. Should there be no IP address matching the CVNI of a CI in the ES, in the illustrated embodiment, IPP2 may be dropped. In some implementations, metadata of the type described above may not be used.
[0070] In at least some embodiments, if the DLLF does not include an IP packet with SDA as the destination address, the networking manager may attempt to use the DLLF's destination MAC address to identify one or more CIs to which the DLLF content should be delivered. If the DLLF destination MAC address matches a MAC address assigned to any CI running on the ES (such as MAC address M1 assigned to the LVNI of CI1) (as determined in operations corresponding to element 816), the networking manager, in the illustrated embodiment, may deliver at least a subset of the DLLF content to that CI (element 859).
[0071] If an exact match with the LVNI MAC address is not found, the networking manager may determine whether the DLLF is a broadcast frame (element 822). If the DLLF is a broadcast frame, its contents may be distributed to all CIs running on ESs with LVNIs configured to accept broadcast frames (e.g., based on the LVNI security settings or other parameters) in the illustrated embodiment (element 825). If not, the networking manager could not find a suitable destination CI for the DLLF using a CVNI IVN IP address match or an LVNI MAC address match, and since the DLLF was not a broadcast frame, the DLLF may be dropped in various embodiments (element 828). Operations corresponding to elements 810 et seq. may be repeated for at least some inbound data link layer frames by the network manager in the illustrated embodiment.
[0072] It should be noted that in various embodiments, some of the operations shown in the flow diagram of Figure 8 may be performed in an order different from that shown in the figure, or may be performed in parallel rather than sequentially. For example, in some embodiments, a MAC address-based check (elements 816 and 859) may be performed before an IP address-based check (elements 813 and 819) to identify a destination CI for an inbound message. Additionally, some of the operations shown in Figure 8 may not be required in one or more implementations.
[0073] In some embodiments, a networking manager responsible for directing incoming messages to the compute instances of the expansion server may be implemented at least in part in an offload card of the expansion server. FIG. 9 illustrates example components of an expansion server, according to at least some embodiments. As illustrated, a VCS expansion server (ES) 902 may include a primary physical processor set 904, a main memory (e.g., one or more modules of random access memory, or RAM) 908, a partially offloaded virtualization manager (PVM) 970, and a CI 950, such as CIs 950A, 950B, and 950C, to which LVNIs and / or CVNIs of the type described above may be programmatically attached. In some embodiments, each CI of CI 950 may include a respective virtual machine, referred to as a guest virtual machine (GVM). ES 902 may also include several other components, such as various persistent storage devices, which are not illustrated in FIG. 9 to avoid confusion. A primary physical processor set 904 may include several physical CPUs (pCPUs), including pCPUs 905A-905C in the illustrated embodiment. A virtualized version of a pCPU, called a vCPU or virtual CPU, may be assigned to each CI by PVM 970 during the CI's lifetime. Each CI 950 may include a respective instance of an operating system (e.g., operating systems 952A-952B) and a set of applications (e.g., 954A-954C) running on behalf of clients of the virtualized computing service. One or more of the applications 954 and / or associated operating systems 952 may exchange data link layer messages with devices in the local premises network using the LVNI as described above.
[0074] The PVM 970, in the illustrated embodiment, may include an opportunistic stripped-down hypervisor 920 (running on a pCPU) and one or more offloaded virtualization manager components (OVMCs) 972 that do not use the pCPU. The OVMC 972 may include, for example, a virtualization controller 915 and at least a portion of a networking manager 916 in the offload card 910. The networking manager may inspect data link layer frames received at the ES via a cable connected to the network cable port 991 of the offload card 910 and determine the CI to which the frame's contents are delivered using the methodology described above (e.g., whether the CVNI's IVN address is used to select the destination CI or whether the LVNI's MAC address is used). The other end of the cable may be connected to a local network switch as described in the context of FIG. 4. In some embodiments, the individual OVMCs of the OVMCs may be implemented using respective system-on-chip designs. While in the illustrated embodiment, both OVMCs 972 are shown as being incorporated within a single offload card 910 (e.g., a Peripheral Component Interconnect Express, or PCIe, card), other approaches to OVMC placement and organization may be employed in different embodiments. For example, in one embodiment, a single system-on-chip implementation may be used to perform the functions of the virtualization controller and the networking manager, thereby eliminating the need for two different OVMCs. In another embodiment, respective offload cards may be used for the virtualization controller 915 and the networking manager 916. The virtualization controller, as suggested by its name, may be responsible for orchestrating or coordinating many of the virtualization management activities performed in the ES 902 in the illustrated embodiment. For example, it may be the first of the PVM's components to boot and may trigger the start-up of other components of the PVM, communicate with the VCS control plane, make memory allocation decisions regarding the CIs, etc.In the illustrated embodiment, the networking manager 916 may implement one or more networking protocols (including, for example, the encapsulation protocol used within VCS) and may serve as an intermediary between the CI and the external networking endpoints of the ES.
[0075] In the illustrated embodiment, the hypervisor 920 may be described as stripped down because much of the work performed by at least some conventional hypervisors may instead be handled by the OVMC 972, thereby reducing the complexity and size of the hypervisor 920. Additionally, the hypervisor 920 may be shown as opportunistic because in most situations, it may wait until the CI voluntarily relinquishes control of the pCPU 905 before the hypervisor uses CPU cycles. Thus, for example, if a particular CI 950 issues an I / O request (where I / O is expected to take approximately time T1 to complete) and relinquishes the pCPU until a response to the I / O request is received, the hypervisor may take advantage of this opportunity to perform one or more virtualization management tasks (which may typically take time T2, where T2 << T1) using the pCPU while the CI does not expect to use the pCPU. In this way, in the illustrated embodiment, the impact on the performance of the application 954 of the hypervisor 920 may be minimal.
[0076] The hypervisor 920 may itself include several subcomponents, including, in the illustrated embodiment, a set of operating system kernel level components 922, a hypervisor coordinator 925, one or more GVM managers 928, an isolation / security component 929, and / or a messaging manager 931. The hypervisor coordinator 925, each of the GVM managers 928, the isolation / security component 929, and / or the messaging manager 931 may be implemented as respective user mode processes in at least some embodiments. In various embodiments, at least some of these components may be implemented as instances of respective statically linked programs that communicate with each other via pipes using a simple specialized protocol. The hypervisor subcomponents may remain passive or dormant by default in the illustrated embodiment, reacting and activating only in response to events (messages from other subcomponents, context switches initiated by the CI, etc.). In some implementations, for example, some of the hypervisor subcomponents may typically remain blocked in a polling system call (such as epoll() or equivalent) most of the time.
[0077] Kernel level components 922 may provide support for various lower level operations, such as the initial response to a VM exit instruction issued by a CI (e.g., when a CI relinquishes a pCPU). Hypervisor coordinator 925, as its name suggests, may be responsible for coordinating the operations of the other subcomponents. Hypervisor coordinator 925 may implement an API that may be used to communicate between OVMC 972 and the hypervisor, for example, initiate the startup and shutdown of CIs (e.g., at the request of OVMC receiving a request from the VCS control plane to start or shut down a CI), expose metrics collected by the GVM manager, provide debugging capabilities, etc.
[0078] Each GVM manager 928 may be responsible for launching or instantiating each CI based on specifications provided by the coordinator 925, monitoring GVM metrics and logs, etc. In some embodiments, the GVM manager 928 may also assist with I / O operations requested by a CI to a device, for example, by trapping the CI's I / O requests and converting them into memory-mapped I / O operations that are completed with the help of the OVMC. In at least some embodiments, in accordance with the security-related principle of least privilege, the GVM manager 928 may drop many of its own privileges as early as possible during the instantiation of a CI. For example, after one or more vPCU (virtual CPU) threads are created for a CI and the CI's memory is mapped, the GVM manager may revoke some of its privileges to reduce the opportunity for security to be breached. In some embodiments, there may be a one-to-one mapping between GVM managers and CIs, while in other embodiments, a single GVM manager may be responsible for multiple GVMs.
[0079] The messaging manager 931 may act as an intermediary between the virtualization controller 915 and the hypervisor, for example, by translating commands issued by the virtualization controller using a queue-based protocol into pipe messages within the hypervisor. The security and isolation component 929 may be responsible for scrubbing or cleaning up the memory of a CI when the CI is terminated, for example, so that unintended sharing of data between CIs may be avoided. In some embodiments, at least a portion of the networking manager functionality may be implemented in the hypervisor. It should be noted that the PVM may include additional components (not shown in FIG. 9) in at least some embodiments, and that in at least one embodiment, one or more of the PVM components shown in FIG. 9 may not be required.
[0080] In at least some embodiments, a server (e.g., including a VCS control plane function and / or an external server function) implementing the types of techniques described herein may include a general-purpose computer system including or configured to access one or more computer-accessible media. FIG. 10 illustrates such a general-purpose computing device 9000. In the illustrated embodiment, the computing device 9000 includes one or more processors 9010 coupled to a system memory 9020 (which may include both non-volatile and volatile memory modules) via an input / output (I / O) interface 9030. The computing device 9000 further includes a network interface 9040 coupled to the I / O interface 9030.
[0081] In various embodiments, the computing device 9000 may be a uniprocessor system including one processor 9010, or a multiprocessor system including several processors 9010 (e.g., two, four, eight, or another suitable number). The processor 9010 may be any suitable processor capable of executing instructions. For example, in various embodiments, the processor 9010 may be a general-purpose or embedded processor implementing any of a variety of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, ARM, or MIPS ISA, or any other suitable ISA. In a multiprocessor system, each processor 9010 may typically, but not necessarily, implement the same ISA. In some implementations, a graphics processing unit (GPU) and / or a field programmable gate array (FPGA) may be used in place of or in addition to a conventional processor.
[0082] The system memory 9020 may be configured to store instructions and data accessible by the processor(s) 9010. In at least some embodiments, the system memory 9020 may include both volatile and non-volatile portions, while in other embodiments, only volatile memory may be used. In various embodiments, the volatile portion of the system memory 9020 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM, or any other type of memory. For the non-volatile portion of the system memory (which may include, for example, one or more NVDIMMs), in some embodiments, flash-based memory devices, including NAND flash devices, may be used. In at least some embodiments, the non-volatile portion of the system memory may include a power source, such as a supercapacitor or other power storage device (e.g., a battery). In various embodiments, any of memristor-based resistive random access memory (ReRAM), 3D NAND technology, ferroelectric RAM, magnetoresistive RAM (MRAM), or various types of phase change memory (PCM) may be used for at least the non-volatile portion of the system memory. In the illustrated embodiment, program instructions and data implementing one or more desired functions, such as those methods, techniques, and data described above, are shown stored in system memory 9020 as code 9025 and data 9026.
[0083] In one embodiment, the I / O interface 9030 may be configured to coordinate I / O traffic between the processor 9010, the system memory 9020, and any peripheral devices within the device, including the network interface 9040, or other peripheral interfaces such as various types of persistent and / or volatile storage devices. In some embodiments, the I / O interface 9030 may perform any necessary protocol, timing, or other data conversions to convert data signals from one component (e.g., the system memory 9020) into a format suitable for use by another component (e.g., the processor 9010). In some embodiments, the I / O interface 9030 may include support for devices attached through various types of peripheral buses (including various types of hardware accelerators and / or offloaders), such as, for example, variants of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard. In some embodiments, the functionality of the I / O interface 9030 may be split into two or more separate components, such as, for example, a northbridge and a southbridge. Also, in some embodiments, some or all of the functionality of the I / O interface 9030, such as an interface to the system memory 9020, may be incorporated directly into the processor 9010.
[0084] The network interface 9040 may be configured to allow data to be exchanged between the computing device 9000 and other devices 9060 attached to one or more networks 9050, such as, for example, other computer systems or devices as illustrated in FIGS. 1-9. In various embodiments, the network interface 9040 may support communication over any suitable wired or wireless general data network, such as, for example, a type of Ethernet network. Additionally, the network interface 9040 may support communication over a telecommunications / telephony network, such as an analog voice network or a digital fiber communications network, over a storage area network, such as a Fibre Channel SAN, or over any other suitable type of network and / or protocol.
[0085] In some embodiments, the system memory 9020 may represent one embodiment of a computer-accessible medium configured to store at least a subset of the program instructions and data used to implement the methods and apparatus discussed in the context of FIG. 1 through FIG. 9. However, in other embodiments, the program instructions and / or data may be received, transmitted, or stored on different types of computer-accessible media. Generally speaking, the computer-accessible medium may include non-transitory storage or memory media, such as magnetic or optical media, e.g., disks or DVDs / CDs, coupled to the computing device 9000 via the I / O interface 9030. The non-transitory computer-accessible storage media may also include any volatile or non-volatile media, such as RAM (e.g., SDRAM, DDR SDRAM, RDRAM, SRAM, etc.), ROM, etc., that may be included in some embodiments of the computing device 9000 as the system memory 9020 or another type of memory. In some embodiments, multiple non-transitory computer-readable storage media may collectively store program instructions that, when executed on or across one or more processors, implement at least a subset of the methods and techniques described above. Computer-accessible media may further include transmission media transmitted over a communication medium such as a network and / or wireless link, as may be implemented via network interface 9040, or signals such as electrical, electromagnetic, or digital signals. Some or all of the computing devices as shown in FIG. 10 may be used to implement the described functionality in various embodiments, e.g., software components executing on various different devices and servers may cooperate to provide functionality. In some embodiments, some of the described functionality may be implemented using storage devices, network devices, or special purpose computer systems in addition to or instead of being implemented using a general purpose computer system.The term "computing device" as used herein refers to at least all of these types of devices, but is not limited to these types of devices.
[0086] Embodiments of the present disclosure may be described in light of the following clauses.
[0087] Article 1. 1. A system comprising: A control plane server of a virtualized computing service of a provider network, the control plane server being located in a data center of the provider network; a networking manager of an extension server of the virtualization computing service, the extension server being located on a premises outside the provider network; The control plane server, attaching a cloud access virtual network interface to a compute instance of the virtualized computing service, the compute instance running on the expansion server, and the cloud access virtual network interface being assigned a first Internet Protocol (IP) address from a range of IP addresses of an isolated virtual network configured in the virtualized computing service; attaching a local premises access virtual network interface to the compute instance, the local premises access virtual network interface being assigned a specific Media Access Control (MAC) address by the control plane server, and configured such that the local premises access virtual network interface is not assigned an IP address from the network address range of the isolated virtual network; The networking manager, in response to at least determining that a first data link layer frame received at the enhanced server includes a first IP packet having a destination address within a set of virtual host network addresses of a substrate network of the virtualized computing service; Extracting a second IP packet from the first IP packet using an encapsulation protocol of the virtualized computing service; In response to determining that the destination address of the second IP packet matches the first IP address, deliver the second IP packet to the compute instance; in response to determining that a destination MAC address of a second data link layer frame received at the enhanced server matches the particular MAC address, the second data link layer frame does not include an IP packet having the destination address of the first IP packet and delivers at least a portion of the contents of the second data link layer frame to the compute instance without utilizing the encapsulation protocol.
[0088] Article 2. The control plane server, 13. The system of claim 1, further configured to receive a request to launch the compute instance via a programmatic interface, wherein the local premises access virtual network interface is created and programmatically attached to the compute instance in response to the request to launch the compute instance.
[0089] Article 3. The control plane server, In response to receiving a request via a programmatic interface to terminate the compute instance, store an indication that the particular MAC address is available for reuse; assigning the particular MAC address to a separate local premises access virtual network interface; 13. The system of claim 1, further configured to programmatically attach the other local premises access virtual network interface to another compute instance.
[0090] Article 4. The control plane server, 13. The system of claim 1, further configured to determine at least a portion of the particular MAC address based on input received via a programmatic interface from a client of the virtualized computing service on whose behalf the compute instance is launched.
[0091] Article 5. The control plane server, The system of clause 1, further configured to provide, via a programmatic interface, (a) a first set of metrics related to network traffic associated with the cloud access virtual network interface, and (b) a second set of metrics related to network traffic associated with the local premises access virtual network interface.
[0092] Article 6. 1. A computer-implemented method comprising: Attaching a first local premises access virtual network interface to a first compute instance of the virtualized computing service by a control plane server of the virtualized computing service, the control plane server being located in a data center of a provider network, the first compute instance being executed on a specific server located in a premises outside the data center, and the first local premises access virtual network interface being assigned a first media access control (MAC) address by the control plane server; in response to determining, by a networking manager of the particular server, that a first data link layer frame received at the particular server includes a first IP packet having a particular destination address assigned to the particular server; extracting a second IP packet from the first IP packet using an encapsulation protocol of the virtualized computing service; delivering the second IP packet to the first compute instance; and in response to the networking manager determining that a destination MAC address of a second data link layer frame received at the particular server matches the first MAC address, delivering at least a portion of the contents of the second data link layer frame to the first compute instance.
[0093] Article 7. 7. The computer-implemented method of claim 6, further comprising creating, by the control plane server, the first local premises access virtual network interface prior to launching the first compute instance on the particular server in response to a programmatic request.
[0094] Article 8. The computer-implemented method of any one of clauses 6-7, wherein the second data link layer frame is received at the particular server from one of: (a) an Internet of Things (IoT) device at the premises; or (b) an industrial automation device at the premises.
[0095] Article 9. The computer-implemented method of any one of clauses 6 to 8, wherein the second data link layer frame is received at the particular server via a cable connected to a network switch in a local network of the premises.
[0096] Article 10. detaching the first local premises access virtual network interface from the first compute instance and attaching the first local premises access virtual network interface to a second compute instance in response to one or more requests received via a programmatic interface; 10. The computer-implemented method of claim 6, further comprising: after the first local premises access virtual network interface is programmatically attached to the second compute instance, in response to determining that a destination MAC address of a third data link layer frame received at a server on which the second compute instance executes matches the first MAC address, delivering at least a portion of the contents of the third data link layer frame to the second compute instance.
[0097] Article 11. 11. The computer-implemented method of claim 10, wherein the first compute instance is configured within a first isolated virtual network of the virtualized computing service and the second compute instance is configured within a second isolated network of the virtualized computing service.
[0098] Article 12. attaching, by the control plane server, a second local premises access virtual network interface to a second compute instance launched on the particular server of the virtualized computing service, wherein the second local premises access virtual network interface is assigned a second MAC address and the second local premises access virtual network interface is not assigned an Internet Protocol (IP) address by the control plane server; The computer-implemented method of any one of clauses 6-9, further comprising: in response to the networking manager determining that a destination MAC address of a third data link layer frame received at the particular server matches the second MAC address, delivering at least a portion of the contents of the third data link layer frame to the second compute instance.
[0099] Article 13. The computer-implemented method of any one of clauses 6-9, further comprising providing the first MAC address in response to a request received via a programmatic interface.
[0100] Article 14. attaching, by the control plane server, a second local premises access virtual network interface to the first compute instance, wherein the second local premises access virtual network interface is assigned a second MAC address and the second local premises access virtual network interface is not assigned an Internet Protocol (IP) address by the control plane server; The computer-implemented method of any one of clauses 6-9, further comprising: in response to the networking manager determining that a third data link layer frame having the second MAC address as a destination has been received at the particular server, delivering at least a portion of the contents of the third data link layer frame to the first compute instance.
[0101] Article 15. obtaining, via a programmatic interface at the control plane server, an indication of security settings to be applied with respect to network traffic associated with the first local premises access virtual network interface, the security settings including restrictions on sources from which data link layer frames are accepted; 10. The computer-implemented method of any one of clauses 6-9, further comprising: enforcing, by the networking manager, the security settings, the enforcing including verifying that a source from which the second data link layer frame is received does not violate the restrictions before delivering a portion of the content of the second data link layer frame to the first compute instance.
[0102] Article 16. A non-transitory computer-accessible storage medium that, when executed on a processor, Detecting, by a networking manager in an expansion server of a virtualized computing service, that a first data link layer frame is acquired at the expansion server; the non-transitory computer-accessible storage medium storing program instructions for, in response to the networking manager determining that a destination media access control (MAC) address of the first data link layer frame matches a particular MAC address of a first local premises access virtual network interface attached to the first compute instance, delivering at least a portion of the contents of the first data link layer frame to a first compute instance running on the expansion server, wherein the first local premises access virtual network interface is not assigned an Internet Protocol (IP) address from a range of IP addresses managed by the virtualization computing service.
[0103] Article 17. When executed on the processor, 17. The non-transitory computer-accessible storage medium of clause 16, further storing program instructions for throttling a rate at which data link layer frames are transmitted from the first compute instance based at least in part on a rate limit indicated via a programmatic interface.
[0104] Article 18. When executed on the processor, 18. The non-transitory computer-accessible storage medium of any one of clauses 16-17, further storing program instructions for, in response to detecting that a broadcast data link layer frame has been received at the extended server, distributing, by the networking manager, at least a portion of the contents of the broadcast data link layer frame to a plurality of compute instances of the extended server, including the first compute instance and a second compute instance to which a second local premises access virtual network interface is programmatically attached.
[0105] Article 19. When executed on the processor, 19. The non-transitory computer-accessible storage medium of any one of clauses 16 to 18, further storing program instructions for determining, by the networking manager, following detaching the first local premises access virtual network interface from the first compute instance, that contents of a second data link layer frame received at the expansion server should not be delivered to the first compute instance, wherein a destination MAC address of the second data link layer frame matches the particular MAC address.
[0106] Article 20. 20. The non-transitory computer-accessible storage medium of any one of clauses 16-19, wherein the first data link layer frame includes a message formatted in accordance with one of Address Resolution Protocol (ARP), Domain Host Configuration Protocol (DHCP), or a custom data link layer application implemented within a local network at the premises.
[0107] conclusion Various embodiments may further include receiving, transmitting, or storing instructions and / or data implemented in accordance with the foregoing description of a computer-accessible medium. Generally speaking, a computer-accessible medium may include storage or memory media, such as magnetic or optical media, e.g., disks or DVD / CD-ROMs, volatile or non-volatile media, such as RAM (e.g., SDRAM, DDR, RDRAM, SRAM, etc.), ROM, and transmission media or signals, such as electrical, electromagnetic, or digital signals transmitted over a communication medium, such as a network and / or a wireless link.
[0108] The various methods shown in the figures and described herein represent exemplary embodiments of the methods. The methods may be implemented in software, hardware, or a combination thereof. The order of the methods may be changed, and various elements may be added, reordered, combined, omitted, modified, etc.
[0109] Various modifications and changes may be made, as would be apparent to a person skilled in the art having the benefit of this disclosure. All such modifications and changes are intended to be included, and therefore the above description should be considered in an illustrative rather than a limiting sense.
Claims
1. 1. A system comprising: A control plane server of a virtualized computing service of a provider network, the control plane server being located in a data center of the provider network; a networking manager of an extension server of the virtualization computing service, the extension server being located on a premises outside the provider network; The control plane server, attaching a cloud access virtual network interface to a compute instance of the virtualized computing service, the compute instance running on the expansion server, and the cloud access virtual network interface being assigned a first Internet Protocol (IP) address from a range of IP addresses of an isolated virtual network configured in the virtualized computing service; attaching a local premises access virtual network interface to the compute instance, the local premises access virtual network interface being assigned a specific media access control (MAC) address by the control plane server, and configured such that the local premises access virtual network interface is not assigned an IP address from the network address range of the isolated virtual network; The networking manager, in response to at least determining that a first data link layer frame received at the enhanced server includes a first IP packet having a destination address within a set of virtual host network addresses of a substrate network of the virtualized computing service; Extracting a second IP packet from the first IP packet using an encapsulation protocol of the virtualized computing service; In response to determining that the destination address of the second IP packet matches the first IP address, deliver the second IP packet to the compute instance; the system is configured to: in response to determining that a destination MAC address of a second data link layer frame received at the enhanced server matches the particular MAC address, the second data link layer frame does not include an IP packet having the destination address of the first IP packet, and deliver at least a portion of the contents of the second data link layer frame to the compute instance without utilizing the encapsulation protocol.
2. The control plane server, 2. The system of claim 1, further configured to receive a request to launch the compute instance via a programmatic interface, wherein the local premises access virtual network interface is created and programmatically attached to the compute instance in response to the request to launch the compute instance.
3. The control plane server, In response to receiving a request via a programmatic interface to terminate the compute instance, store an indication that the particular MAC address is available for reuse; assigning the particular MAC address to a separate local premises access virtual network interface; The system of claim 1 , further configured to programmatically attach the other local premises access virtual network interface to another compute instance.
4. The control plane server, 2. The system of claim 1, further configured to determine at least a portion of the particular MAC address based on input received via a programmatic interface from a client of the virtualized computing service on whose behalf the compute instance is launched.
5. The control plane server, 11. The system of claim 1, further configured to cause, via a programmatic interface, to provide: (a) a first set of metrics related to network traffic associated with the cloud access virtual network interface; and (b) a second set of metrics related to network traffic associated with the local premises access virtual network interface.
6. 1. A computer-implemented method comprising: Attaching a first local premises access virtual network interface to a first compute instance of the virtualized computing service by a control plane server of the virtualized computing service, the control plane server being located in a data center of a provider network, the first compute instance being executed on a specific server located in a premises outside the data center, and the first local premises access virtual network interface being assigned a first media access control (MAC) address by the control plane server; in response to determining, by a networking manager of the particular server, that a first data link layer frame received at the particular server includes a first IP packet having a particular destination address assigned to the particular server; extracting a second IP packet from the first IP packet using an encapsulation protocol of the virtualized computing service; delivering the second IP packet to the first compute instance; and in response to the networking manager determining that a destination MAC address of a second data link layer frame received at the particular server matches the first MAC address, delivering at least a portion of the contents of the second data link layer frame to the first compute instance.
7. 7. The computer-implemented method of claim 6, further comprising creating, by the control plane server, the first local premises access virtual network interface prior to launching the first compute instance on the particular server in response to a programmatic request.
8. obtaining, via a programmatic interface at the control plane server, an indication of security settings to be applied with respect to network traffic associated with the first local premises access virtual network interface, the security settings including restrictions on sources from which data link layer frames are accepted; 8. The computer-implemented method of claim 6, further comprising: enforcing, by the networking manager, the security settings, the enforcing including verifying that a source from which the second data link layer frame is received does not violate the restrictions before delivering a portion of the content of the second data link layer frame to the first compute instance.
9. detaching the first local premises access virtual network interface from the first compute instance and attaching the first local premises access virtual network interface to a second compute instance in response to one or more requests received via a programmatic interface; 8. The computer-implemented method of claim 6, further comprising: after the first local premises access virtual network interface is programmatically attached to the second compute instance, in response to determining that a destination MAC address of a third data link layer frame received at a server on which the second compute instance executes matches the first MAC address, delivering at least a portion of the contents of the third data link layer frame to the second compute instance.
10. attaching, by the control plane server, a second local premises access virtual network interface to the first compute instance, wherein the second local premises access virtual network interface is assigned a second MAC address and the second local premises access virtual network interface is not assigned an Internet Protocol (IP) address by the control plane server; 9. The computer-implemented method of claim 6, further comprising: in response to determining, by the networking manager, that a third data link layer frame having the second MAC address as a destination has been received at the particular server, delivering at least a portion of the contents of the third data link layer frame to the first compute instance.
11. A non-transitory computer-accessible storage medium that, when executed on a processor, Detecting, by a networking manager in an expansion server of a virtualization computing service, that a first data link layer frame is acquired at the expansion server; the non-transitory computer-accessible storage medium storing program instructions for delivering at least a portion of the contents of the first data link layer frame to a first compute instance running on the expansion server in response to the networking manager determining that a destination media access control (MAC) address of the first data link layer frame matches a particular MAC address of a first local premises access virtual network interface attached to the first compute instance, wherein the first local premises access virtual network interface is not assigned an Internet Protocol (IP) address from a range of IP addresses managed by the virtualization computing service.
12. When executed on the processor, 12. The non-transitory computer-accessible storage medium of claim 11 further storing program instructions for throttling a rate at which data link layer frames are transmitted from the first compute instance based at least in part on a rate limit indicated via a programmatic interface.
13. When executed on the processor, 13. The non-transitory computer-accessible storage medium of claim 11, further storing program instructions for, in response to detecting that a broadcast data link layer frame has been received at the extended server, distributing, by the networking manager, at least a portion of the contents of the broadcast data link layer frame to a plurality of compute instances of the extended server, including the first compute instance and a second compute instance to which a second local premises access virtual network interface is programmatically attached.
14. When executed on the processor, 14. The non-transitory computer-accessible storage medium of claim 11, further storing program instructions for determining, following detaching the first local premises access virtual network interface from the first compute instance, by the networking manager that content of a second data link layer frame received at the expansion server should not be delivered to the first compute instance, wherein a destination MAC address of the second data link layer frame matches the particular MAC address.
15. 15. The non-transitory computer-accessible storage medium of claim 11, wherein the first data link layer frame comprises a message formatted according to one of the Address Resolution Protocol (ARP), the Domain Host Configuration Protocol (DHCP), or a custom data link layer application implemented within a local network at the premises where the external server is located.
Citation Information
Patent Citations
Method and system for enforcing security policies on network traffic
US20100333189A1
User-Configured On-Demand Virtual Layer-2 Network for Infrastructure-As-A-Service (IAAS) on a Hybrid Cloud Network
US20150365281A1
Provider network service extensions
US20200159555A1
Cloud service bandwidth management and configuration methods and related device
WO2021052382A1
