Method and system for multi-operator access to provider underlay extension
By deploying the provider's underlying expansion within the communication service provider network, the problem of high delay between the end user device and the computing resources is solved, low-latency access is achieved, and the performance and reliability of the application is improved. , supports multiple network connections.
Patent Information
- Application Number
- CN202080092752.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-11-29
- Filing Date
- 2020-11-05
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2040-11-05
AI Technical Summary
The physical constraints between the end user device and the computing resources of existing cloud computing platforms lead to high delays, making it difficult to achieve low latency access, especially when deploying computing resources within the communication service provider network, limiting applications such as games. Performance for streaming, video conferencing and real-time rendering.
By deploying provider underlying extensions (PSEs) within the communication service provider network, the computing resources of the cloud provider network are extended to a location closer to the end user, and the use of heterogeneous servers and local network managers can achieve low latency access, supporting Multi-operator access and traffic separation ensure resource availability and security.
It significantly reduces access delay between end user devices and cloud computing resources, improves application responsiveness and reliability, supports a variety of network connection methods, and meets the needs of low latency.
Smart Images

Figure CN114946222B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to cloud computing. Background Art
[0002] Cloud computing platforms typically provide on-demand, managed computing resources to customers. Such computing resources (e.g., computing and storage capacity) are typically provided by large pools of capacity installed in data centers. Customers can request computing resources from the "cloud," and the cloud can provide computing resources to those customers. Technologies such as virtual machines and containers are often used to allow customers to securely share the capacity of a computer system. BRIEF DESCRIPTION OF THE DRAWINGS
[0003] Various embodiments according to the present disclosure will be described with reference to the following drawings.
[0004] Figure 1 An exemplary system including a cloud provider network and also including various provider underlay extensions is shown according to some embodiments.
[0005] Figure 2 An exemplary system is shown in which a cloud provider network underlay extension is deployed within a communications service provider network according to some embodiments.
[0006] Figure 3 Exemplary components of a provider underlay extension and cloud provider network and connectivity therebetween are shown in greater detail according to some embodiments.
[0007] Figure 4 An exemplary cloud provider network including provider underlay extensions (or “edge locations”) is shown according to some embodiments.
[0008] Figure 5 is a diagram showing an environment in which a user establishes a connection between a computing instance and a communications service provider network according to some embodiments.
[0009] Figure 6 is a diagram illustrating another implementation for connectivity between computing instances and a communications service provider network according to some embodiments.
[0010] Figure 7 An exemplary communications service provider network and provider underlay extensions to be deployed in the communications service provider network are shown in accordance with some embodiments.
[0011] Figure 8 An exemplary deployment of a provider underlay extension within a communications service provider network is shown in accordance with some embodiments.
[0012] Fig. 9An exemplary deployment of provider underlay extensions and data plane functionality within a transport network of a communications service provider network is shown in accordance with some embodiments.
[0013] Fig.10 An exemplary deployment of a provider underlay extension and traffic disruption system within a transport network of a communications service provider network is shown in accordance with some embodiments.
[0014] Fig.11 Pooled capacity provided by deploying a provider underlay extension within a communications service provider network is illustrated according to some embodiments.
[0015] Fig.12 is a flow chart illustrating the operation of a method for providing multi-operator access to a provider underlay extension according to some embodiments.
[0016] Fig.13 An exemplary provider network environment is shown according to some embodiments.
[0017] Fig.14 is a block diagram of an exemplary provider network that provides storage services and hardware virtualization services to customers according to some embodiments.
[0018] Fig.15 is a block diagram illustrating an exemplary computer system that may be used in some embodiments. DETAILED DESCRIPTION
[0019] The present disclosure relates to methods, devices, systems and non-transitory computer-readable storage media for implementing multi-operator access to the underlying extension of the provider. A cloud provider network (or "cloud") refers to a large network-accessible pool of computing resources (such as computing, storage and networking resources, applications and services). A cloud can provide convenient, on-demand network access to a shared pool of configurable computing resources, which can be programmatically provisioned and released in response to customer commands. Therefore, cloud computing can be viewed as an application delivered as a service over a public accessible network (e.g., the Internet, a cellular communication network) and the hardware and software in the cloud provider data center that provides those services. Some customers may wish to use the resources and services of such cloud provider networks, but for various reasons (e.g., communication latency with customer devices, legal compliance, security or other reasons) prefer to provision these resources and services within their own network (e.g., at the customer's local location), at a separate network managed by a cloud provider, within a communications service provider's network, or within another independent network.
[0020] In some embodiments, a fragment of a cloud provider network (referred to herein as a provider underlay extension ("PSE")) can be provisioned within a network independent of the cloud provider network. For example, a cloud provider network typically includes a physical network (e.g., sheet metal boxes, cables, rack hardware) referred to as an underlay. The underlay can be considered a network structure containing the physical hardware that runs the services of the provider network. In some embodiments, a provider underlay "extension" can be an extension of the cloud provider network underlay formed by one or more servers located locally at a customer or partner facility, a separate cloud provider-managed facility, a communications service provider facility, or other facilities including servers, wherein such servers communicate with nearby availability zones or regions of the cloud provider network via a network (e.g., a publicly accessible network, such as the Internet). Customers can access the provider underlay extension via the cloud provider underlay or another network, and can use the same application programming interface (API) to create and manage resources in the provider underlay extension as they would for creating and managing resources in a region of the cloud provider network.
[0021] As indicated above, one exemplary type of provider underlay extension is one formed by servers located locally at a customer or partner facility. This type of underlay extension located outside of a cloud provider network data center may be referred to as an "outpost" of the cloud provider network. Another exemplary type of provider underlay extension is one formed by servers located in a facility managed by the cloud provider but includes data plane capacity that is at least partially controlled by a remote / different control plane of the cloud provider network.
[0022] In some embodiments, another example of provider bottom layer extension is a network deployed at a communication service provider. Communication service providers typically include companies that have deployed networks, and end users obtain network connectivity through the network. For example, communication service providers may include mobile or cellular network providers (e.g., operating 3G, 4G and / or 5G networks), wired Internet service providers (e.g., cables, digital subscriber lines, optical fibers, etc.), and WiFi providers (e.g., at locations such as hotels, coffee shops, airports, etc.). Although the traditional deployment of computing resources in data centers provides various benefits due to centralization, physical constraints (such as network distance and the number of network hops) between end-user devices and those computing resources may prevent very low latency from being achieved. By installing or deploying capacity in the communication service provider network, cloud provider network operators can provide computing resources with significantly reduced access latency to end-user devices, providing single-digit millisecond latency in some cases. This low-latency access to computing resources is an important driving force for providing improved responsiveness for existing cloud-based applications and realizing next-generation applications such as game streaming, video conferencing, Internet-based real-time communications, virtual reality, real-time rendering, industrial automation, collaborative platforms, autonomous vehicles, etc.
[0023] As used herein, computing resources of a cloud provider network installed within a communications service provider network are sometimes referred to as "cloud provider network edge locations" or simply "edge locations" because they are closer to the "edge" where end users connect to the network than computing resources in centralized data centers. Such edge locations may include one or more networked computer systems that provide computing resources to customers of the cloud provider network to provide services to end users at lower latency than would be achievable if those computing instances were hosted in a data center site. Provider underlay extensions deployed in a communications service provider network may also be referred to as "wavelength zones."
[0024] Figure 1 An exemplary system including an underlying extension of a provider network according to some embodiments is shown, where customers of the provider network can deploy computing resources. A cloud provider network 100 (sometimes referred to simply as a "cloud") refers to a network-accessible pool of computing resources (such as computing, storage and networking resources, applications and services), which can be virtualized or bare metal. A cloud can provide convenient, on-demand network access to a shared pool of configurable computing resources that can be programmatically provisioned and released in response to customer commands. These resources can be dynamically provisioned and reconfigured to adjust to variable loads. Therefore, cloud computing can be viewed as applications delivered as services over a publicly accessible network (e.g., the Internet, a cellular communication network) and the hardware and software in a cloud provider data center that provides those services.
[0025] The cloud provider network 100 can provide users with an on-demand, scalable computing platform through a network, for example, allowing users to have scalable "virtual computing devices" for their use through their use of computing servers (the computing servers provide computing instances through one or both of a central processing unit (CPU) and a graphics processing unit (GPU), optionally used together with a local storage device) and block storage servers (the block storage servers provide virtualized persistent block storage for a specified computing instance). These virtual computing devices have the attributes of a personal computing device, including hardware (various types of processors, local memory, random access memory (RAM), hard disk and / or solid state drive (SSD) storage devices), operating system options, networking capabilities, and pre-loaded application software. Each virtual computing device can also virtualize its console input and output (e.g., keyboard, display, and mouse). This virtualization allows users to connect to their virtual computing devices using computer applications such as browsers, application programming interfaces (APIs), software development kits (SDKs), etc., so as to configure and use their virtual computing devices like personal computing devices. Unlike personal computing devices that have a fixed amount of hardware resources available to a user, the hardware associated with a virtual computing device can be scaled up or down depending on the resources required by the user.
[0026] As indicated above, a user (e.g., user 138) can connect to virtualized computing devices and other cloud provider network 100 resources and services using various interfaces 104 (e.g., APIs) via the intermediate network 136. An API refers to an interface and / or communication protocol between a client (e.g., electronic device 134) and a server, such that if a client issues a request in a predefined format, the client should receive a response in a specific format or cause a defined action to be initiated. In the context of a cloud provider network, an API provides a gateway for clients to access cloud infrastructure by allowing them to obtain data from a cloud provider network or cause an action within a cloud provider network, thereby enabling the development of applications that interact with resources and services hosted in a cloud provider network. The API can also enable different services of a cloud provider network to exchange data with each other. Users may choose to deploy their virtual computing systems to provide network-based services for their own use and / or for use by their customers or clients.
[0027] Cloud provider network 100 may include a physical network (e.g., sheet metal boxes, cables, rack hardware) referred to as an underlay. The underlay may be considered the network fabric that contains the physical hardware that runs the provider network's services. The underlay may be isolated from the rest of cloud provider network 100, e.g., it may not be possible to route from an underlay network address to an address in a production network running a cloud provider service, or to a customer network hosting customer resources.
[0028] The cloud provider network 100 may also include an overlay network of virtualized computing resources running on the bottom layer. In at least some embodiments, the hypervisor or other device or process on the network bottom layer may use encapsulation protocol technology to encapsulate and route network packets (e.g., client IP packets) between client resource instances on different hosts within the provider network through the network bottom layer. Encapsulation protocol technology may be used on the network bottom layer to route encapsulated packets (also referred to as network bottom layer packets) between endpoints on the network bottom layer via an overlay network path or route. Encapsulation protocol technology may be viewed as providing a virtual network topology overlaid on the network bottom layer. Thus, network packets may be routed along the bottom layer network according to the construction in the overlay network (e.g., a virtual network that may be referred to as a virtual private cloud (VPC), a port / protocol firewall configuration that may be referred to as a security group). A mapping service (not shown) may coordinate the routing of these network packets. A mapping service may be a regional distributed lookup service that maps a combination of an overlay Internet protocol (IP) and a network identifier to a bottom layer IP so that a distributed bottom layer computing device may find where to send the packet.
[0029] For illustration, each physical host device (e.g., computing server 106, block storage server 108, object storage server 110, control server 112) may have an IP address in the underlying network. Hardware virtualization technology may enable multiple operating systems to run simultaneously on a host computer, for example as virtual machines (VMs) on computing server 106. A hypervisor or virtual machine monitor (VMM) on the host allocates the hardware resources of the host between various VMs on the host and monitors the execution of the VMs. Each VM may be provided with one or more IP addresses in the overlay network, and the VMM on the host may know the IP address of the VM on the host. The VMM (and / or other devices or processes on the network bottom layer) may use encapsulation protocol technology to encapsulate network packets (e.g., client IP packets) and route the network packets between virtualized resources on different hosts within the cloud provider network 100 through the network bottom layer. Encapsulation protocol technology may be used on the network bottom layer to route encapsulated packets between endpoints on the network bottom layer via an overlay network path or route. Encapsulation protocol technology may be viewed as providing a virtual network topology overlaid on the network bottom layer. The encapsulation protocol technology may include a mapping service that maintains a mapping directory that maps IP overlay addresses (e.g., IP addresses visible to the customer) to underlying IP addresses (IP addresses not visible to the customer) that can be accessed by various processes on the cloud provider network for routing packets between endpoints.
[0030] As shown, in various embodiments, the traffic and operations of the cloud provider network bottom layer can be broadly subdivided into two categories: control plane traffic carried on the logical control plane 114A and data plane operations carried on the logical data plane 116A. Although the data plane 116A represents the movement of user data through the distributed computing system, the control plane 114A represents the movement of control signals through the distributed computing system. The control plane 114A generally includes one or more control plane components or services distributed on one or more control servers 112 and implemented by them. The control plane traffic generally includes management operations, such as establishing isolated virtual networks for various customers, monitoring resource utilization and health status, identifying specific hosts or servers for computing instances to be started for requests, and provisioning additional hardware as needed. The data plane 116A includes customer resources (e.g., computing instances, containers, block storage volumes, databases, file storage) implemented on the cloud provider network. The data plane traffic generally includes non-management operations, such as transmitting data to customer resources and transmitting data from customer resources.
[0031] The control plane components are typically implemented on a different set of servers than the data plane servers, and control plane traffic and data plane traffic can be sent over separate / different networks. In some embodiments, control plane traffic and data plane traffic can be supported by different protocols. In some embodiments, messages (e.g., packets) sent through the cloud provider network 100 include a flag to indicate whether the traffic is control plane traffic or data plane traffic. In some embodiments, the payload of the traffic can be inspected to determine its type (e.g., control plane or data plane). Other techniques for distinguishing traffic types are possible.
[0032] As shown, the data plane 116A may include one or more computing servers 106, which may be bare metal (e.g., a single tenant) or may be virtualized by a hypervisor to run multiple VMs (sometimes referred to as "instances") or microVMs for one or more customers. These computing servers 106 may support virtualized computing services (or "hardware virtualization services") for cloud provider networks. The virtualized computing service may be part of the control plane 114A, thereby allowing customers to issue commands via an interface 104 (e.g., an API) to start and manage computing instances (e.g., VMs, containers) of their applications. The virtualized computing service may provide virtual computing instances with different computing and / or memory resources. In one embodiment, each of the virtual computing instances may correspond to one of several instance types. The characteristics of an instance type may be its hardware type, computing resources (e.g., the number, type, and configuration of CPUs or CPU cores), memory resources (e.g., the capacity, type, and configuration of local memory), storage resources (e.g., the capacity, type, and configuration of locally accessible storage devices), network resources (e.g., the characteristics of its network interface and / or network capabilities), and / or other suitable descriptive characteristics. Using the instance type selection functionality, an instance type can be selected for a customer based, for example, at least in part, on input from the customer. For example, the customer can select an instance type from a set of predefined instance types. As another example, the customer can specify the desired resources for the instance type and / or the requirements of the workload that the instance will run, and the instance type selection functionality can select the instance type based on such specifications.
[0033] The data plane 116A may also include one or more block storage servers 108, which may include persistent storage devices for storing customer data volumes and software for managing these volumes. These block storage servers 108 may support managed block storage services for the cloud provider network. The managed block storage service may be part of the control plane 114A, allowing customers to issue commands via the interface 104 (e.g., API) to create and manage volumes for applications running on their computing instances. The block storage server 108 includes one or more servers on which data is stored as blocks. A block is a sequence of bytes or bits, typically containing a certain integer number of records, with a maximum length of the block size. Blocked data is typically stored in a data buffer and the block data is read or written to the entire block at a time. Typically, a volume may correspond to a logical collection of data, such as a set of data maintained on behalf of a user. A user volume (which may be viewed as, for example, an individual hard drive ranging in size from 1GB to 1 terabyte (TB) or more) consists of one or more blocks stored on a block storage server. Although viewed as an individual hard drive, it should be understood that a volume may be stored as one or more virtualized devices implemented on one or more underlying physical host devices. A volume can be partitioned several times (e.g., up to 16 times), with each partition hosted by a different host. The data of a volume can be replicated between multiple devices within a cloud provider network to provide multiple copies of the volume (where such copies can collectively represent a volume on a computing system). Volume copies in a distributed computing system can, for example, be beneficially provided with automatic failover and recovery by allowing a user to access a primary copy of the volume or a secondary copy of the volume that is synchronized with the primary copy at a block level, so that the failure of the primary copy or the secondary copy does not prevent access to volume information. The role of the primary copy can be to facilitate reading and writing (sometimes referred to as "input and output operations" or simply "I / O operations") on the volume, and propagate any writes to the secondary copy (preferably synchronously propagated in the I / O path, but asynchronous replication can also be used). The secondary copy can be updated synchronously with the primary copy and provide a seamless transition during a failover operation, whereby the secondary copy assumes the role of the primary copy, and the previous primary copy is designated as a secondary copy or a new replacement secondary copy is provisioned. Although some examples in this article discuss primary and secondary copies, it should be understood that a logical volume can include multiple secondary copies. The computing instance can virtualize its I / O to a volume through a client. The client represents instructions that enable a computing instance to connect to a remote data volume (e.g., a data volume stored on a physically separate computing device accessed over a network) and perform I / O operations at the remote data volume. The client can be implemented on an offload card of a server that includes a processing unit (e.g., a CPU or a GPU) of the computing instance.
[0034] The data plane 116A may also include one or more object storage servers 110, which represent another type of storage device within the cloud provider network. The object storage server 110 includes one or more servers on which data is stored as objects within a resource called a bucket and can be used to support the managed object storage service of the cloud provider network. Each object typically includes the stored data, a variable metadata that enables the object storage server to analyze the various capabilities of the stored object, and a globally unique identifier or key that can be used to retrieve the object. Each bucket is associated with a given user account. Users can store as many objects as desired in their buckets, can write, read and delete objects in their buckets, and can control access to their buckets and objects contained in the buckets. In addition, in an embodiment with multiple different object storage servers distributed in different regions in the above-mentioned regions, users can select the region (or multiple regions) of the storage bucket, for example, to optimize latency. Customers can use buckets to store various types of objects, including machine images that can be used to start VMs, and snapshots representing point-in-time views of volume data.
[0035] Provider underlay extensions 102 ("PSEs") provide the resources and services of cloud provider network 100 within a separate network, thereby extending the functionality of cloud provider network 100 to new locations (e.g., for reasons related to latency of communications with customer devices, legal compliance, security, etc.). As indicated, such provider underlay extensions 102 may include cloud provider network-managed provider underlay extensions 140 (e.g., formed of servers located in cloud provider-managed facilities separate from those associated with cloud provider network 100), communications service provider underlay extensions 142 (e.g., formed of servers associated with communications service provider facilities), customer-managed provider underlay extensions 144 (e.g., formed of servers located locally at customer or partner facilities), as well as other possible types of underlay extensions.
[0036] As shown in the exemplary provider underlying extension 140, the provider underlying extension 102 can similarly include a logical separation between a control plane 118B and a data plane 120B, which respectively extend the control plane 114A and the data plane 116A of the cloud provider network 100. The provider underlying extension 102 can be pre-configured, for example, by a cloud provider network operator with an appropriate combination of hardware and software and / or firmware elements to support various types of computing-related resources, and to do so in a manner that reflects the experience of using the cloud provider network. For example, one or more provider underlying extension location servers can be provisioned by a cloud provider to be deployed within the provider underlying extension 102. As described above, the cloud provider network 100 can provide a set of predefined instance types, each with a different type and number of underlying hardware resources. Each instance type can also be provided in various sizes. In order to enable customers to continue to use the same instance types and sizes as they use in the region in the provider underlying extension 102, the servers can be heterogeneous servers. Heterogeneous servers can support multiple instance sizes of the same type at the same time, and can also be reconfigured to host any instance type supported by its underlying hardware resources. The reconfiguration of heterogeneous servers can occur instantly using the available capacity of the servers, i.e., while other VMs are still running and consuming other capacity of the provider underlying extension location servers. This can improve the utilization of computing resources within the edge location by allowing better packaging of running instances on the servers, and also provide a seamless experience regarding the use of instances on the cloud provider network 100 and the cloud provider network provider underlying extensions.
[0037] As shown, the provider underlying extension server can host one or more computing instances 122. The computing instance 122 can be a VM, or a container that packages the code and all its dependencies, so that the application can run quickly and reliably across the computing environment (e.g., including VMs). In addition, if the customer needs, the server can host one or more data volumes 124. In the region of the cloud provider network 100, such volumes can be hosted on dedicated block storage servers. However, due to the possibility of having significantly smaller capacity at the provider underlying extension 102 than in the region, if the provider underlying extension includes such a dedicated block storage server, it may not provide the best utilization experience. Therefore, the block storage service can be virtualized in the provider underlying extension 102, so that one of the VMs runs the block storage software and stores the data of the volume 124. Similar to the operation of the block storage service in the region of the cloud provider network 100, the volume 124 within the provider underlying extension 102 can be replicated to achieve persistence and availability. The volume can be provisioned in its own isolated virtual network within the provider underlying extension 102. The compute instances 122 and any volumes 124 together comprise a data plane extension 120B of the provider network data plane 116A within the provider underlay extension 102 .
[0038] In some embodiments, servers within the provider underlay extension 102 may host certain local control plane components 126, for example, components that enable the provider underlay extension 102 to continue to operate in the event of a loss of connectivity back to the cloud provider network 100. Examples of these components include a migration manager that can move compute instances 122 between provider underlay extension servers if necessary to maintain availability, and a key value data store that indicates where volume replicas are located. However, the control plane 118B functionality typically used for the provider underlay extension will remain in the cloud provider network 100 to allow customers to use as much of the resource capacity of the provider underlay extension as possible.
[0039] The server software running on the provider underlay extension 102 may be designed by the cloud provider to run on the cloud provider underlay network, and the software may be able to run unmodified in the provider underlay extension 102 to create a private copy of the underlay network (“shadow underlay”) within the edge location by using a local network manager 128. The local network manager 128 may run on the provider underlay extension 102 server and bridge the shadow underlay with the provider underlay extension 102 network, for example, by acting as a virtual private network (VPN) endpoint or endpoint between the provider underlay extension 102 and the proxies 130, 132 in the cloud provider network 100 and by implementing a mapping service (for traffic encapsulation and decapsulation) to correlate data plane traffic (from the data plane proxies) and control plane traffic (from the control plane proxies) with the appropriate servers. By implementing a local version of the provider network's underlay-to-overlay mapping service, the local network manager 128 allows resources in the provider underlay extension 102 to communicate seamlessly with resources in the cloud provider network 100. In some embodiments, a single local network manager may perform these actions for all servers hosting compute instances 122 in the provider underlay extension 102. In other embodiments, each of the servers hosting the compute instances 122 can have a dedicated local network manager.In a multi-rack edge location, inter-rack communications can go through the local network managers, where the local network managers maintain open tunnels between each other.
[0040] The provider underlying extension location can utilize secure networking tunnels through the provider underlying extension 102 network to the cloud provider network 100, for example, to maintain the security of customer data when traversing the provider underlying extension 102 network and any other intermediate networks (possibly including the public Internet). Within the cloud provider network 100, these tunnels consist of virtual infrastructure components including isolated virtual networks (e.g., in an overlay network), control plane agents 130, data plane agents 132, and underlying network interfaces. Such agents can be implemented as containers running on computing instances. In some embodiments, each server in the provider underlying extension 102 location hosting a computing instance can utilize at least two tunnels: one tunnel for control plane traffic (e.g., Constrained Application Protocol (CoAP) traffic) and one tunnel for encapsulated data plane traffic. A connectivity manager (not shown) within the cloud provider network manages their cloud provider network-side lifecycles, for example, by automatically provisioning these tunnels and their components when needed and maintaining them in a healthy operating state. In some embodiments, a direct connection between the provider underlying extension 102 location and the cloud provider network 100 can be used to control the data plane and conduct data plane communications. Compared to VPNs that go through other networks, direct connections can provide constant bandwidth and more consistent network performance because its network path is relatively fixed and stable.
[0041] A control plane (CP) agent 130 may be provisioned in the cloud provider network 100 to represent a specific host in an edge location. The CP agent is an intermediary between the control plane 114A in the cloud provider network 100 and the control plane target in the control plane 118B of the provider bottom layer extension 102. That is, the CP agent 130 provides an infrastructure for tunneling management API traffic destined for the provider bottom layer extension server from the regional bottom layer to the provider bottom layer extension 102. For example, the virtualized computing service of the cloud provider network 100 may issue a command to the VMM of the server of the provider bottom layer extension 102 to start the computing instance 122. The CP agent maintains a tunnel (e.g., VPN) to the local network manager 128 of the provider bottom layer extension. The software implemented in the CP agent ensures that only well-formed API traffic leaves and returns to the bottom layer. The CP agent provides a mechanism to expose remote servers on the cloud provider bottom layer while still protecting the bottom layer security materials (e.g., encryption keys, security tokens) from leaving the cloud provider network 100. The unidirectional control plane traffic tunnel imposed by the CP agent also prevents any (potentially compromised) device from calling back to the bottom layer. The CP agent may be instantiated one-to-one with a server at the provider underlay extension 102, or may be able to manage control plane traffic for multiple servers in the same provider underlay extension.
[0042] Data plane (DP) agent 132 can also be provisioned in cloud provider network 100 to represent a specific server in provider bottom layer extension 102. DP agent 132 acts as a shadow or anchor point for the server and can be used by services within cloud provider network 100 to monitor the health of the host (including its availability, used / free computing and capacity, used / free storage and capacity, and network bandwidth utilization / availability). DP agent 132 also allows isolated virtual networks to span provider bottom layer extension 102 and cloud provider network 100 by acting as an agent for servers in cloud provider network 100. Each DP agent 132 can be implemented as a packet forwarding computing instance or container. As shown, each DP agent 132 can maintain a VPN tunnel with a local network manager 128, which manages traffic to the server represented by DP agent 132. The tunnel can be used to send data plane traffic between provider bottom layer extension servers and cloud provider network 100. Data plane traffic flowing between provider bottom layer extension 102 and cloud provider network 100 can pass through the DP agent 132 associated with the provider bottom layer extension. For data plane traffic flowing from the provider underlay extension 102 to the cloud provider network 100, the DP agent 132 can receive the encapsulated data plane traffic, verify its correctness, and allow it to enter the cloud provider network 100. The DP agent 132 can forward the encapsulated traffic from the cloud provider network 100 directly to the provider underlay extension 102.
[0043] The local network manager 128 may provide secure network connectivity with the proxies 130, 132 established in the cloud provider network 100. After the connection between the local network manager 128 and the proxies has been established, the customer may issue commands via the interface 104 to instantiate a compute instance (and / or perform other operations using the compute instance) using the provider underlay extension resources in a manner similar to the manner in which such commands would be issued for a compute instance hosted within the cloud provider network 100. From the customer's perspective, the customer may now seamlessly use local resources within the provider underlay extension (as well as resources located in the cloud provider network 100, if desired). The compute instances set up on the server at the provider underlay extension 102 may communicate with electronic devices located in the same network, as well as other resources set up in the cloud provider network 100 as needed. The local gateway 146 may be implemented to provide network connectivity between the provider underlay extension 102 and a network associated with the extension (e.g., a communications service provider network in the example of the provider underlay extension 142).
[0044] There may be situations where data needs to be transferred between the object storage service and the provider underlying extension 102. For example, the object storage service may store machine images for booting VMs, as well as snapshots representing point-in-time backups of volumes. The object gateway may be provided on a PSE server or dedicated storage device, and provides customers with a configurable per-bucket cache of the contents of their object storage buckets in the PSE to minimize the impact of PSE regional latency on customer workloads. The object gateway may also temporarily store snapshot data from snapshots of volumes in the PSE, and then synchronize with the object server in the region when possible. The object gateway may also store machine images that customers specify for use within the PSE or at the customer's site. In some embodiments, data within the PSE may be encrypted with a unique key, and for security reasons, the cloud provider may restrict key sharing from the region to the PSE. Therefore, data exchanged between the object storage server and the object gateway may utilize encryption, decryption, and / or re-encryption in order to preserve security boundaries regarding encryption keys or other sensitive data. The transformation agent may perform these operations, and may create a PSE bucket (on the object storage server) using the PSE encryption key to store snapshots and machine image data.
[0045] In the above manner, PSE forms an edge location because it provides resources and services of the cloud provider network outside the traditional cloud provider data center and closer to the customer device. The edge location as referred to herein can be structured in a variety of ways. In some embodiments, the edge location can be an extension of the cloud provider network bottom layer, including a limited amount of capacity provided outside the available zone (for example, in a small data center of a cloud provider or located near the customer workload and possibly away from other facilities in any available zone). Such edge locations can be referred to as "far zone" (due to being away from other available zones) or "near zone" (due to being close to the customer workload). Far zone can be connected to a public accessible network such as the Internet in various ways (for example, directly, via another network or via a dedicated connection with a region). Although generally far zone has a more limited capacity than a certain region, in some cases, far zone may have a considerable capacity, such as thousands or more racks.
[0046] In some embodiments, an edge location may be an extension of the underlying cloud provider network formed by one or more servers located locally at a customer or partner facility, where such servers communicate with nearby availability zones or regions of the cloud provider network over a network (e.g., a publicly accessible network such as the Internet). This type of underlying extension located outside of a cloud provider network data center may be referred to as an "outpost" of the cloud provider network. Some outposts may be integrated into a communications network, for example, as multi-access edge computing (MEC) sites, the physical infrastructure of which is distributed across telecommunication data centers, telecommunication aggregation sites, and / or telecommunication base stations within a telecommunication network. In a local example, the limited capacity of an outpost may be available only to the customer who owns the location (and any other accounts permitted by the customer). In a telecommunication example, the limited capacity of an outpost may be shared between multiple applications (e.g., games, virtual reality applications, healthcare applications) that send data to users of the telecommunication network.
[0047] Edge locations may include data plane capacity that is at least partially controlled by a control plane of a nearby availability zone of a provider network. Thus, an availability zone group may include a "parent" availability zone and any "child" edge locations that are subordinate to the parent availability zone (e.g., at least partially controlled by its control plane). Certain limited control plane functionality (e.g., features that require low-latency communication with customer resources, and / or features that enable the edge location to continue to operate when disconnected from the parent availability zone) may also exist in some edge locations. Thus, in the above examples, an edge location refers to an extension of at least data plane capacity located at the edge of the cloud provider network, close to customer devices and / or workloads.
[0048] Figure 2 An exemplary system in which a cloud provider network edge location is deployed within a communication service provider network according to some embodiments is shown. A communication service provider (CSP) network 200 typically includes a downstream interface to an end-user electronic device and an upstream interface to other networks (e.g., the Internet). In this example, the CSP network 200 is a wireless "cellular" CSP network that includes a radio access network (RAN) 202, 204, an aggregation site (AS) 206, 208, and a core network (CN) 210. RAN 202, 204 includes base stations (e.g., NodeB, eNodeB, gNodeB) that provide wireless connectivity to an electronic device 212. The core network 210 typically includes functionality related to the management of the CSP network (e.g., billing, mobility management, etc.) and transmission functionality that relays traffic between the CSP network and other networks. Aggregation sites 206, 208 can be used to integrate traffic from many different radio access networks into the core network and direct traffic originating from the core network to various radio access networks.
[0049] The end user electronic device 212 is Figure 2 200 wirelessly connected to the radio access network 202 from left to right. Such electronic devices 212 are sometimes referred to as user equipment (UE) or customer premises equipment (CPE). Data traffic is typically routed to the core network 210 through a fiber optic transport network consisting of multiple hops of layer 3 routers (e.g., at aggregation sites). The core network 210 is typically located in one or more CSP sites, which may be data centers, SAP sites or SAP edge sites, LG extension sites, etc. For data traffic to locations outside the CSP network 200, the network components 222 to 226 typically include a firewall through which traffic can enter or leave the CSP network 200 to reach an external network, such as the Internet or a cloud provider network 100. Note that in some embodiments, the CSP network 200 may include facilities that allow traffic to enter or leave from sites further downstream of the core network 210 (e.g., at an aggregation site or RAN).
[0050] Provider underlay extensions 216 to 220 include computing resources that are managed as part of the cloud provider network but are installed or located within various points of the CSP network (e.g., locally in space owned or leased by the CSP). Computing resources typically provide a certain amount of computing and memory capacity that the cloud provider can allocate to its customers for use. Computing resources may also include storage and accelerator capacity (e.g., solid-state drives, graphics accelerators, etc.). Here, provider underlay extensions 216, 218, and 220 communicate with the cloud provider network 100.
[0051] Typically, the farther the provider underlay extension is from the cloud provider network 100 (or closer to the electronic device 212), for example, in terms of network hops and / or distance, the lower the network latency between the computing resources within the provider underlay extension and the electronic device 212. However, physical site constraints typically limit the amount of provider underlay extension location computing capacity that can be installed at various points within the CSP, or determine whether computing capacity can be installed at all at various points. For example, a provider underlay extension located within the core network 210 can typically have a much larger footprint (in terms of physical space, power requirements, cooling requirements, etc.) than a provider underlay extension located within the RAN 202, 204.
[0052] The installation or positioning of the provider underlay extension within the CSP network may vary depending on the specific network topology or architecture of the CSP network. Figure 2Indicates that the provider underlay extension can generally be connected to any location where the CSP network can interrupt packet-based traffic (e.g., IP-based traffic). In addition, communications between a given provider underlay extension and the cloud provider network 100 generally securely transit at least a portion of the CSP network 200 (e.g., via a secure tunnel, a virtual private network, a direct connection, etc.). In the example shown, network component 222 facilitates data traffic routing to and from provider underlay extension 216 integrated with RAN 202, network component 224 facilitates data traffic routing to and from provider underlay extension 218 integrated with AS 206, and network component 226 facilitates data traffic routing to and from provider underlay extension 220 integrated with CN 210. Network components 222 to 226 may include routers, gateways, or firewalls. To facilitate routing, the CSP may assign one or more IP addresses from the CSP network address space to each of the edge locations.
[0053] In the 5G wireless network development work, edge locations can be considered as a possible implementation of multi-access edge computing (MEC). Such edge locations can be connected to various points in the CSP 5G network that provide interruptions for data traffic as part of the user plane function (UPF). Older wireless networks can also contain edge locations. For example, in 3G wireless networks, edge locations can be connected to the packet-switched network portion of the CSP network, such as to the serving general packet radio service support node (SGSN) or to the gateway general packet radio service support node (GGSN). In 4G wireless networks, edge locations can be connected to a serving gateway (SGW) or a packet data network gateway (PGW) as part of the core network or evolved packet core (EPC).
[0054] In some embodiments, traffic between the provider underlay extension 228 and the cloud provider network 100 may be terminated from the CSP network 200 without being routed through the core network 210. For example, the network component 230 of the RAN 204 may be configured to route traffic between the provider underlay extension 216 of the RAN 204 and the cloud provider network 100 without traversing the aggregation site or the core network 210. As another example, the network component 231 of the aggregation site 208 may be configured to route traffic between the provider underlay extension 232 of the aggregation site 208 and the cloud provider network 100 without traversing the core network 210. The network components 230, 231 may include gateways or routers having routing data to direct traffic from edge locations destined for the cloud provider network 100 to the cloud provider network 100 (e.g., through a direct connection or an intermediate network 234) and to direct traffic from the cloud provider network 100 destined for the provider underlay extension to the provider underlay extension.
[0055] In some embodiments, a provider underlay extension may be connected to more than one CSP network. For example, when two CSPs share or route traffic through a common point, a provider underlay extension may be connected to two CSP networks. For example, each CSP may allocate a portion of its network address space to the provider underlay extension, and the provider underlay extension may include a router or gateway that may differentiate traffic exchanged with each of the CSP networks. For example, traffic from one CSP network destined for a provider underlay extension may have a different destination IP address, source IP address, and / or virtual local area network (VLAN) tag than traffic received from another CSP network. Traffic originating from the provider underlay extension to a destination on one of the CSP networks may be similarly encapsulated with the appropriate VLAN tag, source IP address (e.g., from a pool assigned to the provider underlay extension from the destination CSP network address space), and destination IP address.
[0056] It should be noted that although Figure 2 An exemplary CSP network architecture includes a radio access network, an aggregation site, and a core network, but the naming and structure of the architecture of the CSP network may vary between generations of wireless technologies, between different CSPs, and between wireless CSP networks and fixed-line CSP networks. Figure 2 Several locations are shown where edge locations may be located within the CSP network, but other locations are possible (eg, at a base station).
[0057] Figure 3Exemplary components of a provider underlay extension and a cloud provider network associated with a communications service provider and connectivity therebetween are shown in greater detail according to some embodiments. The provider underlay extension 300 provides resources and services of the cloud provider network within a CSP network 302, thereby extending the functionality of the cloud provider network 100 closer to an end user device 304 connected to the CSP network.
[0058] Provider underlay extension 300 similarly includes a logical separation between control plane 306B and data plane 308B, which respectively extend control plane 114A and data plane 116A of cloud provider network 100. Provider underlay extension 300 may be pre-configured, for example, by a cloud provider network operator with an appropriate combination of hardware and software and / or firmware elements to support various types of computing-related resources, and to do so in a manner that reflects the experience of using a cloud provider network. For example, one or more provider underlay extension location servers 310 may be provisioned by a cloud provider for deployment within CSP network 302.
[0059] In some embodiments, the servers 310 within the provider underlay extension 300 may host certain local control plane components 314, such as components that enable the provider underlay extension 300 to continue to operate in the event of a loss of connectivity back to the cloud provider network 100. In addition, certain controller functions may typically be implemented locally on data plane servers, or even in cloud provider data centers, such as functions for collecting and sending metrics to monitoring services for monitoring instance health, and functions for coordinating the transfer of instance state data during live migration. However, the control plane 306B functionality typically used for the provider underlay extension 300 will remain in the cloud provider network 100 to allow customers to use as much of the resource capacity of the provider underlay extension as possible.
[0060] As shown, the provider underlying extension server 310 can host computing instances 312. The computing instance can be a VM, microVM, or a container that packages the code and all its dependencies, so that the application can run quickly and reliably across computing environments (e.g., including VMs). Therefore, the container is an abstraction of the application layer (meaning that each container simulates a different software application process). Although each container runs an isolated process, multiple containers can share a common operating system, for example, by starting in the same virtual machine. In contrast, the virtual machine is an abstraction of the hardware layer (meaning that each virtual machine simulates a physical machine that can run software). Virtual machine technology can use a physical server to run the equivalent of multiple servers (each of which is called a virtual machine). Although multiple virtual machines can run on a physical machine, each virtual machine typically has its own copy of the operating system, as well as applications and their related files, libraries, and dependencies. Virtual machines are generally referred to as computing instances or simply "instances." Some containers can run on instances running container agents, while some containers can run on bare metal servers.
[0061] In some embodiments, the execution of edge-optimized computing instances is supported by a lightweight virtual machine manager (VMM) running on server 310, on which edge-optimized computing instances are started based on application profiles. These VMMs enable lightweight micro virtual machines (microVMs) to be started in a fraction of a second. These VMMs can also enable container runtimes and container orchestrators to manage containers as microVMs. Nevertheless, these microVMs also utilize the security and workload isolation provided by traditional VMs, such as running them as isolated processes through VMMs, and the resource efficiency brought by containers. As used herein, microVMs refer to VMs that are initialized using a limited device model and / or using a minimal OS kernel supported by a lightweight VMM, and each microVM can have a low memory overhead of <5 MiB, so that thousands of microVMs can be packaged on a single host. For example, a microVM can have a streamlined version of an OS kernel (e.g., only having required OS components and their dependencies) to minimize startup time and memory footprint. In one embodiment, each process of a lightweight VMM encapsulates one and only one microVM. The process can run the following threads: API, VMM, and vCPU. The API thread is responsible for the API server and the associated control plane. The VMM thread exposes the machine model, the minimal legacy device model, the microVM metadata service (MMDS), and the VirtIO device simulation network and block devices. In addition, there are one or more vCPU threads (one vCPU thread per guest CPU core).
[0062] Additionally, if desired by the customer, the server 310 can host one or more data volumes 324. A volume can be provisioned in its own isolated virtual network within the provider underlay extension 300. The compute instances 312 and any volumes 324 together constitute a data plane extension 308B of the provider network data plane 116A within the provider underlay extension 300.
[0063] A local gateway 316 may be implemented to provide network connectivity between the provider underlay extension 300 and the CSP network 302. The cloud provider may configure the local gateway 316 with an IP address on the CSP network 302 and exchange routing data (e.g., via Border Gateway Protocol (BGP)) with the CSP network component 320. The local gateway 316 may include one or more routing tables that control the routing of inbound traffic to the provider underlay extension 300 and outbound traffic leaving the provider underlay extension 300. The local gateway 316 may also support multiple VLANs in situations where the CSP network 302 uses separate VLANs for different portions of the CSP network 302 (e.g., one VLAN tag for wireless networks and another VLAN tag for fixed networks).
[0064] In some embodiments of the provider underlay extension 300, the extension includes one or more switches, sometimes referred to as top-of-rack (TOR) switches (e.g., in rack-based embodiments). The TOR switches are connected to CSP network routers (e.g., CSP network components 320), such as provider edge (PE) or software-defined wide area network (SD-WAN) routers. Each TOR switch may include an uplink link aggregation (LAG) interface to the CSP network router, each LAG supporting multiple physical links (e.g., 1G / 10G / 40G / 100G). The link may run a link aggregation control protocol (LACP) and be configured as an IEEE802.1q trunk to enable multiple VLANs on the same interface. This LACP-LAG configuration allows the edge location management entity of the control plane of the cloud provider network 200 to add more peer links to the edge location without adjusting the routing. Each of the TOR switches may establish an eBGP session with the operator PE or SD-WAN router. The CSP may provide a dedicated autonomous system number (ASN) for the edge location and the ASN of the CSP network 302 to facilitate the exchange of routing data.
[0065] Data plane traffic originating from provider underlay extension 300 may have multiple different destinations. For example, traffic addressed to a destination in data plane 116A of cloud provider network 100 may be routed via a data plane connection between provider underlay extension 300 and cloud provider network 100. Local network manager 318 may receive a packet from compute instance 312 addressed to, for example, another compute instance in cloud provider network 100 and encapsulate the packet with a destination as the underlay IP address of a server hosting the other compute instance and then send it to cloud provider network 100 (e.g., via a direct connection or a tunnel). For traffic from compute instance 312 addressed to another compute instance hosted in another provider underlay extension 322, local network manager 318 may encapsulate the packet with a destination as the IP address assigned to the other provider underlay extension 322, thereby allowing CSP network component 320 to handle routing of the packet. Alternatively, if the CSP network component 320 does not support inter-edge location traffic, the local network manager 318 can address the packet to a repeater in the cloud provider network 200, which can send the packet to another provider underlay extension 322 via its data plane connection (not shown) to the cloud provider network 100. Similarly, for traffic from a compute instance 312 address to a location outside the CSP network 302 or cloud provider network 100 (e.g., on the Internet), if the CSP network component 320 allows routing to the Internet, the local network manager 318 can encapsulate the packet with a source IP address corresponding to an IP address in the operator address space assigned to the compute instance 312. Otherwise, the local network manager 318 can send the packet to an Internet gateway in the cloud provider network 100, which can provide Internet connectivity for the compute instance 312. For traffic from a compute instance 312 addressed to an electronic device 304, the local gateway 316 can use network address translation (NAT) to change the source IP address of the packet from an address in the address space of the cloud provider network to an address in the operator network space.
[0066] The local gateway 316, local network manager 318, and other local control plane components 314 may run on the same server 310 that hosts the compute instances 312, may run on a dedicated processor integrated with the edge location server 310 (e.g., on an offload card), or may be executed by servers separate from those hosting customer resources.
[0067] Figure 4An exemplary cloud provider network including a provider bottom extension location (or edge location) according to some embodiments is shown. As shown, a cloud provider network 400 can be formed into multiple regions 402, wherein a region is a separate geographic region in which a cloud provider has one or more data centers 404. Each region 402 can include two or more availability zones (AZs) connected to each other via a dedicated high-speed network such as, for example, a fiber optic communication connection. An availability zone refers to an isolated fault domain including one or more data center facilities, which have separate power supplies, separate networking, and separate cooling relative to other availability zones. A cloud provider may strive to locate the availability zones within a region, making them far enough apart from each other so that natural disasters, large-scale power outages, or other unexpected events will not simultaneously take more than one availability zone offline. Customers can connect to resources within the availability zone of a cloud provider network via a publicly accessible network (e.g., the Internet, a cellular communication network, a CSP network). A switching center (TC) is the main backbone location that links customers to a cloud provider network, and can be co-located in other network provider facilities (e.g., an Internet service provider, a telecommunications provider). Two or more TCs can be operated in each region to achieve redundancy.
[0068] Compared to the number of regional data centers or availability zones, the number of edge locations 406 can be much higher. This widespread deployment of edge locations 406 can provide low-latency connectivity to the cloud for a much larger group of end-user devices (compared to those end-user device groups that happen to be very close to regional data centers). In some embodiments, each edge location 406 can be equivalent to a certain part of the cloud provider network 400 (e.g., a parent availability zone or regional data center). This peering allows various components operating in the cloud provider network 400 to manage the computing resources of the edge location. In some cases, multiple edge locations can be located or installed in the same facility (e.g., a separate rack of a computer system) and managed by different districts or data centers to provide additional redundancy. It should be noted that although edge locations are generally described herein as being within a CSP network, in some cases, such as when a cloud provider network facility is relatively close to a communication service provider facility, the edge location can be retained in the physical premises of the cloud provider network while being connected to the communication service provider network via optical fiber or other network links.
[0069] Edge location 406 can be structured in a variety of ways. In some embodiments, edge location 406 can be an extension of the underlying cloud provider network, including a limited amount of capacity provided outside the availability zone (e.g., in a small data center of a cloud provider or other facilities located near customer workloads and possibly far from any availability zone). Such edge locations can be referred to as far zones (due to being far from other availability zones) or near zones (due to being close to customer workloads). Far zones can be connected to public accessible networks such as the Internet in various ways (e.g., directly, via another network, or via a dedicated connection to a region). Although far zones usually have more limited capacity than a certain region, in some cases, far zones may have considerable capacity, such as thousands or more racks. As indicated herein, a cloud provider network can be formed into multiple regions, where a region can be a geographical region in which a cloud provider gathers data centers. Each region can include multiple (e.g., two or more) availability zones (AZs) connected to each other via dedicated high-speed networks such as fiber optic communication connections. AZs can provide isolated fault domains including one or more data center facilities, which have separate power supplies, separate networking, and separate cooling relative to those data center facilities in another AZ. Preferably, AZs within a region are located far enough apart from each other so that the same natural disaster (or other failure-inducing event) does not affect more than one AZ at the same time or take them offline at the same time. Customers can connect to the AZs of the cloud provider's network via a publicly accessible network (e.g., the Internet, a cellular communication network).
[0070] The parenting of a given edge location as an AZ or region of a cloud provider network can be based on many factors. One such parenting factor is data sovereignty. For example, in order to keep data originating from a CSP network in a certain country in the country, an edge location deployed in the CSP network can be a parent of an AZ or region in the country. Another factor is the availability of services. For example, some edge locations may have different hardware configurations, such as the presence or absence of components, such as local non-volatile storage devices (e.g., solid-state drives), graphics accelerators, etc. for customer data. Some AZs or regions may lack services that utilize these additional resources, so edge locations can be parents of AZs or regions that support the use of those resources. Another factor is the latency between an AZ or region and an edge location. Although there are latency benefits to deploying edge locations in a CSP network, those benefits may be offset by using edge locations as parents of distant AZs or regions, which can introduce significant latency to edge location to regional traffic. Therefore, edge locations are typically parents of nearby (in terms of network latency) AZs or regions.
[0071] Figure 5is a diagram showing an environment in which a user establishes a connection between a computing instance and a communications service provider network according to some embodiments. Since an edge location can be connected to multiple CSPs, or a given CSP can have multiple networks or network slices (e.g., a 4G network and a 5G network), and each CSP or CSP network can have a different configuration, the cloud provider network provides a new type of gateway that does not consider the underlying configuration details of a specific CSP network from the customer, yet allows the customer to easily establish a connection with an available CSP network. The CSP gateway can be implemented as a new gateway type to enable a customer isolated virtual network ("IVN") to establish a connection with a CSP network. The CSP gateway can be implemented with other types of gateways, such as an Internet gateway (e.g., to facilitate routing from a computing instance of an IVN to the Internet) and a VPN gateway (e.g., to facilitate routing between an IVN and another customer network such as a local network).
[0072] As shown, a customer (e.g., user 138) has launched a compute instance 514 hosted by a server 510 of a PSE 508 (or edge location), which extends a cloud provider network 500 within a CSP network 501 (also referred to as "network A"). The compute instance 514 is connected to an isolated virtual network ("IVN") 512, which is referred to herein as "network B" and can be implemented as an overlay network on the bottom layer of the cloud provider network 500 (referred to herein as "network C"). For example, IVN 512 can use a private IP address block (e.g., 192.168.0.0 / 24), which can be used to address devices on IVN 512 (such as compute instance 514), and a local network manager (not shown) performs encapsulation and decapsulation of packets sent from and to the devices on the bottom layer of the cloud provider network 500, respectively, within packets addressed to the devices (e.g., server 510) on the bottom layer of the cloud provider network 500. Here, computing instance 514 has been assigned IP address 192.168.0.2.
[0073] As part of deploying PSE 508 within CSP network 501, the CSP may allocate some pool of network addresses from CSP network 501 for use by PSE 508. In this example, the CSP network address range 10.0.2.0 / 24 is allocated to PSE 508 (some larger ranges may be allocated to all PSEs deployed within the CSP network and distributed to the various PSEs by the cloud provider network). PSE 508 may advertise those addresses to CSP network 501. For example, a local gateway (not shown) may advertise the allocated address block to CSP network 501 via BGP.
[0074] In view of the above environment, circles numbered “ 1 ” to “ 3 ” illustrate an exemplary process including a user requesting a gateway to provide connectivity between a computing instance 514 and a CSP network 501 .
[0075] At circle "1", user 138 generates a request 594 to create a CSP gateway using electronic device 134. As described above, communications between electronic device 134 and provider network 100 (such as a request to create a CSP gateway) can be routed through interface 104 (such as by using an application programming interface (API) call, via a console implemented as a website or application, etc.). In addition to serving as a front end for control plane services, interface 104 can also perform operations such as verifying the identity and authority of the user initiating the request, evaluating the request and routing it to the appropriate control plane service. Request 594 can include an identifier of the IVN in which a CSP gateway such as IVN 512 is to be created. Upon receiving the request, CSP gateway service 590 can create CSP gateway 516 as part of the specified IVN, assign an IP address within the range of IVN 512 to CSP gateway 516 (e.g., 192.168.0.1), and return an identifier such as an IP address to electronic device 134.
[0076] Note that the customer IVN 512 may include a router and associated routing table. The routing table may include a default entry for IP addresses within the private range of the customer IVN (e.g., locally route packets destined for 192.168.0.0 / 24), and the user 138 may add another entry for packets that do not satisfy the default route to the CSP gateway 516 (e.g., route packets destined for 0.0.0.0 / 0 to the CSP gateway).
[0077] Also as shown in circle “1”, user 138 generates a request 596 to obtain a CSP network address from a CSP network address pool. Request 596 may include an identifier of a specific CSP network address pool from which an address is to be allocated. For example, the request may be for an IP address of operator A's 4G network, operator A's 5G network, operator B's network, etc. Using the identifier, the CSP gateway service 590 allocates one of the available addresses in the associated pool to the customer and records the allocation. The CSP gateway service 590 may track the status of the address in the pool in a database provided by the database service 592 or other data storage area (e.g., whether it has been allocated), as shown in circle “2”. The CSP gateway service 590 may return the allocated IP address (or its identifier) to the electronic device 134. In this example, the CSP gateway service 590 allocates address 10.0.2.55 from the 10.0.2.0 / 24 pool.
[0078] As also shown in circle "1", user 138 generates a request 598 to associate the assigned IP address with a specific computing instance. Request 598 may include an instance identifier and an IP address or IP address identifier. In this example, the request identifies the computing instance 514 and the assigned 10.0.2.55 address. Using the identifier, CSP gateway service 590 configures CSP gateway 516 to establish connectivity between computing instance 514 and CSP network 501. This connectivity can allow devices such as device 522 and device 524 to connect to computing instance 514 through CSP network 501, thereby taking advantage of the low latency provided by deploying PSE 508 within CSP network 501.
[0079] Since various CSP networks or network slices may have a variety of network configuration parameters, the CSP gateway service 590 obtains the configuration data for a particular CSP network from a database provided by the database service 592 or other data storage area, as indicated by circle "2". For each CSP network, the database may include information such as the IP address of the network component through which traffic entering and leaving a given PSE is routed, whether the CSP network has a specific vlan tag, whether the CSP network is reached through a specific interface of the PSE (e.g., the first network adapter or the second network adapter), etc. In this example, the configuration data indicates that the PSE 508 is connected to a CSP network component 520 (e.g., a gateway or router) having an IP address of 10.0.10.1.
[0080] exist Figure 5At circle "3" in FIG. 1 , the CSP gateway service 590 sends one or more commands to configure the CSP gateway 516. Such commands can be sent as control plane traffic via an agent (e.g., agent 130). Here, connectivity is established by configuring the routing table 517 and network address translation (NAT) table 518 of the CSP gateway 516. Specifically, the CSP gateway service 590 creates or updates the routing table 517 to have a route for packets with a range of 10.0.0.0 / 16 to the CSP network component 520 at 10.0.10.1, thereby allowing messages sent from the computing instance 514 to, for example, device 522 or device 524 to be routed via the CSP gateway 516 and the CSP network 501. The routing table 517 also includes local routes to indicate that packets addressed to destinations within the range of the IVN 512 are routed within the IVN 512. In addition, CSP gateway service 590 creates or updates NAT table 518 to associate the assigned IP address (10.0.2.55) with compute instance 514 (192.168.0.2), thereby allowing messages sent from the CSP network (e.g., from device 522 or device 524) to the 10.0.2.55 address to reach compute instance 514. Note that CSP network component 520 includes an entry in its routing table 519 that routes messages destined for the 10.0.2.0 / 24 range to the CSP gateway (10.0.2.1). Such an entry can be based on an exchange of routing information (e.g., via BGP).
[0081] although Figure 5 IP version 4 addressing is shown in , but IP version 6 addressing may also be used. In addition, in some embodiments, the CSP gateway bridges between IP version 4 and IP version 6. For example, the IVN of the compute instance may use IPv4 addressing, while the CSP network may use IPv6 addressing, or vice versa.
[0082] Note that the above description assumes that the customer computing instance 514 is already running before establishing a connection to the CSP network 501 via the CSP gateway 516. In other embodiments, the customer may request to launch a computing instance (e.g., via a hardware virtualization service) with one or more configuration parameters (e.g., as described with reference to circle "1") to automatically connect the launched instance to a specified CSP network via the CSP gateway. In addition, one or more of the requests 594, 596, and / or 598 may be combined. For example, a single request may be used to obtain an address of a particular CSP network address and assign it to an identified computing instance.
[0083] In some embodiments, the customer can configure the CSP gateway to allow or deny certain types of traffic. For example, the customer can specify that the CSP gateway only allows outbound traffic from the compute instance to the CSP network (e.g., compute instance 514 connecting to device 522). As another example, the customer can specify that the CSP gateway only allows inbound traffic (e.g., device 522 connecting to compute instance 514).
[0084] In the case of implementing a separate CSP gateway for each customer IVN, the CSP gateway may have multiple network addresses, for example, one network address on the customer IVN and one or more network addresses for each connected CSP network. Due to the limited number of addresses on customer IVNs and CSP networks, the CSP gateways for multiple (or all) customer IVNs hosted by the PSE can be implemented by common components. Figure 6 One such method is shown in .
[0085] Figure 6 6 is a diagram showing another implementation for connectivity between computing instances and communications service provider networks according to some embodiments. In this example, PSE 608 includes a local gateway 616 (e.g., local gateway 146, local gateway 316) that provides connectivity between instances connected to customer IVNs and CSP networks. Here, local gateway 616 connects computing instance 611 in IVN 610 ("Network X") to CSP network 601 ("Network M"), and connects computing instance 613 in IVN 612 to CSP network 602 ("Network M"). CSP networks 601 and 602 can be different slices or networks of a single CSP (e.g., 4G network, fixed line network, 5G network, etc.) or networks of different CSPs.
[0086] In some embodiments, the local gateway 616 has an IP address on the underlying layer of the cloud provider network (although residing within the PSE 608 as hardware, software, or some combination thereof). Compute instances 611 and 613 are hosted by servers of the PSE 608, which also have IP addresses on the underlying layer of the provider network. One or more local network managers 614 can provide routing for the IVN. For example, the local network manager 614 can receive a packet sent from the compute instance 611 to another compute instance (not shown) of the IVN 610, look up the underlying IP address of the server hosting the other compute instance, and encapsulate the packet in another packet addressed to the server hosting the other compute instance (so that another local network manager decapsulates and passes it to the other instance). As another example, the local network manager 614 can receive a packet to the IP address of the logical CSP gateway of the IVN and route the packet to the local gateway along with an indication of the IVN.
[0087] For this example, assume that a customer has requested connectivity between compute instance 611 and CSP network 601, and that the same customer or another customer has requested connectivity between compute instance 613 and CSP network 602. Additionally, the CSP has assigned 10.0.3.0 / 24 addresses to PSE 608, which local gateway 616 advertises to network components of the CSP network (e.g., to CSP NC 622 of CSP network 601 and CSP NC 624 of CSP network 602). The customer has requested IP addresses from the pool, and IP address 10.0.3.55 has been assigned to compute instance 611, and IP address 10.0.3.200 has been assigned to compute instance 613.
[0088] To provide this connectivity, the CSP gateway service (not shown) configures a local gateway. For example, the CSP gateway service can associate the computing instance with its assigned IP address from the network address pool allocated by the CSP by updating the NAT table 618. As shown, the NAT table 618 associates the computing instance 611 with the CSP network address 10.0.3.55 and associates the computing instance 613 with the CSP network address 10.0.3.200. In addition, the CSP gateway service configures routing to allow the computing instance 611 to communicate with a device (not shown) connected to the CSP network 601, and to allow the computing instance 613 to communicate with a device (not shown) connected to the CSP network 602.
[0089] Because the specific connection details of various CSP networks may vary from one CSP network to another, configuration data (e.g., stored in a database of database service 592) may include configuration details for the various CSP networks. Such configuration details may include whether the CSP network uses virtual local area network ("vlan") tagging and which vlans correspond to which networks, the gateway of a given CSP network through which the PSE connects, whether the CSP networks are connected through different physical interfaces of the PSE, etc. In this example, the configuration data for CSP network 601 indicates that it includes CSP network address 10.0.1.0 / 24, which can be accessed via a gateway at address 10.0.1.1 on vlan=10 via interface=eth0. Additionally, the configuration data for CSP network 602 indicates that it includes CSP network address 10.0.2.0 / 24, which can be accessed via a gateway at address 10.0.2.1 without vlan tagging via interface=eth1. To facilitate routing, the CSP gateway service may configure routing table 617 as indicated. Note that source IVN entries are included to distinguish between different IVN networks and CSP networks. Here, routing table 617 allows IVN 610 or devices on network "X" to communicate with CSP network 601, but not with CSP network 602. Routing table 617 also allows IVN 612 or devices on network "Y" to communicate with CSP network 602, but not with CSP network 601. Routing table 617 also includes local routing entries to indicate that packets addressed to destinations within the range of a given IVN are routed within the IVN.
[0090] As indicated herein, one benefit provided by utilizing a PSE with a backing cloud provider network is that the computing resources implemented within the PSE are "closer" to the end-user, and thus such architectures provide for extremely low latency interactions preferred by modern computing applications such as video games, communications, etc. Thus, a system employing a PSE can satisfy customer use cases requiring low client latency, and can allow users to launch computing instances (and other computing resources) around a geographic region to guarantee reliable low latency to all clients in the coverage area.
[0091] However, many applications require providing low-latency access to end users who obtain network connectivity from different communication service providers (e.g., a first mobile / cellular network operator and a second mobile / cellular network operator). This creates a challenge of deploying PSEs within the network of a first CSP to provide its users with low-latency access to PSE resources, while also allowing other users of other CSPs to similarly obtain low-latency access to these PSE resources, which may be located within proprietary networks that are not generally accessible to entities outside the first CSP (or its customers).
[0092] Thus, embodiments disclosed herein provide techniques for implementing multi-operator access to a provider underlay extension deployed within a communications service provider network. Figure 7 An exemplary communication service provider network and a provider bottom layer extension to be deployed in the communication service provider network according to some embodiments are shown. A modern communication service provider (or "operator") network or mobile network typically has a radio tower or base station 214 at one end of its network 200, which communicates with an electronic device 212 using wireless communication technology. At or near the base station 214, traffic initiated by the electronic device 212 reaches the CSP site 715 (e.g., a complete or small data center, etc.) of the communication provider via an overlay network through one or more transport networks 725 / aggregation networks (e.g., via fiber optic lines), and the overlay network can, for example, use a GPRS tunneling protocol (GTP) tunnel to carry traffic. The overlay network (and tunnel) typically terminates at a mobile core network 210, which includes a control plane function 705 and a data plane function 710 for providing communication services. The mobile overlay network can be considered a closed network that is proprietary to the communication service provider, and only users / entities authorized by the operator can send traffic on this part of their network.
[0093] The specific data plane functions 710 and control plane functions 705 implemented in various embodiments may vary based on the generation or type of communication service being provided. For example, in some 4G Long Term Evaluation (LTE) networks, the control plane functions 705 and data plane functions 710 may be implemented by well-known entities such as a mobile management entity (MME), a serving gateway (SGW), a public data network gateway (PDN-GW or PGW), a home subscriber server (HSS), and many other entities known to those skilled in the art, but in other types of networks, other entities that may perform control plane functions and / or data plane functions may be implemented. In some embodiments, these entities may perform only control plane operations or data plane operations, but in many embodiments, some of these entities may perform some control plane operations and some data plane operations.
[0094] In general, the data plane functions 710 include operations associated with transferring packets between a user device and other networks (e.g., the Internet) or between devices, which operations may include US IP address allocation, packet filtering and forwarding, etc., while the control plane functions 705 may include operations for facilitating data transfer, such as radio resource allocation, mobility management, session management, paging control, charging, security management, configuration management, connection management, broadcasting, measurement configuration, measurement reporting, cell selection and / or reselection, etc.
[0095] Upon leaving the mobile core network 210, traffic may be sent via one or more networks to an Internet Exchange Point (IXP), where the traffic may be routed as IP traffic (e.g., using other infrastructure of other providers, using globally routable network addresses) via the Internet 700. An IXP (or "IX") is the physical infrastructure through which communications service providers (e.g., cellular providers, or other wired or wireless Internet service providers (ISPs) and content delivery networks (CDNs)) exchange traffic between their networks / autonomous systems.
[0096] As disclosed herein, it is beneficial to deploy PSEs as close to the "edge" of these CSP networks 200A to 200N (i.e., close to end-user electronic devices 212A to 212B) as possible so that these devices can access resources within the PSE 142 as quickly as possible. However, by deploying CSP PSEs 142 at most locations in a carrier network (e.g., a first CSP network 200A), the PSE 142 is typically only reachable by electronic devices of that carrier or other carrier entities because, again, most of these networks are closed networks. In this naive deployment of PSEs 142, if an application (such as a game server) is deployed to the PSE 142 within the first telecommunications service provider's network 200A, users (e.g., electronic devices 221B) of other telecommunications service provider networks 200B to 200N may not be able to access the application because it is located within the first telecommunications service provider's closed network.
[0097] This leads to a problem in that many applications need to support users who get network connectivity from multiple sources (e.g., wireless cellular service via this telecommunications service provider, wireless cellular service via other telecommunications service providers, wired access via another ISP over the Internet, etc.). Therefore, in order to serve a broad user base, application owners must deploy these types of applications back to more universally accessible locations, such as within a cloud provider network or other data centers. However, while this solves the heterogeneous connectivity problem, it still results in poor user-to-application latency.
[0098] Therefore, it is desirable to deploy a computing environment such as a PSE within a first operator network to provide low latency access to these resources from multiple connectivity options (such as through other wireless or wired communication services).
[0099] Figure 8 An exemplary deployment of a provider underlay extension within a communications service provider network is shown in accordance with some embodiments. Figure 8 8, the PSE 508 is deployed within the telecommunications service provider network 200A data center 800, which implements the mobile core network 210 control plane functions 805 and data plane functions 810. However, in this example, the PSE 508 is deployed "behind" the mobile core network 210 (from the vantage point of the user electronic device 212A), on the "Internet-facing" side of the core, but before the IXP 720, which allows the second CSP network 200B to directly access the resources of the PSE 508 (assuming appropriate routing advertisements are made, for example, on the gateway 616 and portions of similarly located edge routers of the CSP network 200A and / or CSP network 200B).
[0100] As about Figure 6 As described, the PSE 608 may utilize a gateway 616 configured to allow resources within the PSE 608 to connect to different networks, such as the CSP network 200A and the CSP network 200B, such as via the use of an allocated IP space that may be split between these networks, the use of different VLANs, etc. In some embodiments, the PSE 508 at its edge (e.g., in a top-of-rack (ToR) switch) may implement peering sessions with components of the first CSP network 200A and the second CSP network 200B (e.g., via the Border Gateway Protocol (BGP) or similar edge routing protocols, for example, using autonomous system interconnection technology that allows different systems to talk to each other). Thus, this placement arrangement provides significant end-user latency gains (especially for wireless mobile network clients) compared to more traditional application placement environments (e.g., in a typical data center or cloud environment) without requiring significant changes to the CSP network 200A.
[0101] Fig. 9 Another exemplary deployment possibility is shown in , which shows an exemplary deployment of a provider underlay extension together with data plane functionality within a transport network of a communications service provider network according to some embodiments.
[0102] In some embodiments, the PSE 508 can be placed closer to the edge by effectively splitting the control plane function 805 and the data plane function 810 into two. As shown, the data plane function 810 (e.g., the service architecture evolution gateway (SAE-GW) function) can be moved closer to the edge of the site 900 that may be located within the transport network 725A, and can be interrupted from the site to connect to the PSE 508. For example, the site 900 can be a network aggregation site that is only a few "hops" away from the base station 214. Since the data plane function 810 can terminate the overlay network, the user plane traffic effectively leaves the mobile overlay network, and therefore the traffic is converted to IP traffic, and can be sent directly to the PSE 508 again using the gateway 616 technology involving multiple operator networks and address spaces disclosed herein, or sent to the rest of the core (e.g., via backhaul). In some embodiments, the control plane function 805 can be placed farther away from the site 900, for example, back in the core network implemented in the CSP site 715A (e.g., data center), and still make the control signaling work well. In this environment, the PSE 508 may be peered with the second CSP network 200B at a peering point 905, which may be, for example, at the site 900 or a nearby site. For example, the peering point 905 may be a connection established between the second CSP network 200B and the PSE 508 via a physical network connection (e.g., a fiber optic line or similar network communication cable) or a wireless network connection, and may be an IXP point between the two CSP networks 200A to 200B.
[0103] Fig.10 Another exemplary deployment of a traffic disruption system within a transport network of a provider underlay extension and a communications service provider network according to some embodiments is shown. The architecture utilizes a traffic disruption steering system (traffic disruption 1010), which can be implemented in software, hardware, or both, that can monitor the traffic passing through it and separate out traffic intended for applications of the PSE 508, which is directed away from the mobile core and instead forwarded to the PSE 508. Other traffic (e.g., voice calls, general Internet traffic, etc.) can be passed to the core unmodified.
[0104] In some embodiments, the traffic interrupt 1010 can be deployed as a transparent proxy that can be identified based on a traffic flow identifier (e.g., a combination of packet headers, such as protocol (e.g., TCP) values, source and destination port numbers, and / or source and destination IP addresses), based on an access point name (APN) configured for the user device when attempting to reach the PSE, etc. This information can be programmed or otherwise provided to the traffic interrupt 1010, such as by the control plane function 805A of the CSP network 200A. Advantageously, this architecture similarly places the PSE 508 very close to the end-user electronic device 212A, resulting in very low latency, and involves introducing the traffic interrupt 1010, rather than placing the DP function (e.g., Fig. 9 ) is moved from its typical deployment location in a data center, which may be more or less preferred by the CSP.
[0105] Fig.11 1 to 604, and wherein the resources are very close to the users of the CSP networks 601 to 604 from a latency perspective, and wherein the resources can be accessed by end users from a large number of different networks and network types.
[0106] Fig.12 1200 (or other processes described herein, or variations thereof, and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions, and are implemented as codes (e.g., executable instructions, one or more computer programs, or one or more applications) executed together on one or more processors, implemented by hardware, or implemented by a combination thereof. The code is stored on a computer-readable storage medium, for example, in the form of a computer program including instructions executable by one or more processors. The computer-readable storage medium is non-transitory. In some embodiments, one or more operations 1200 are performed by the CSP networks of other figures (e.g., the first CSP network 200A).
[0107] Operation 1200 includes, at block 1205, launching a compute instance on a provider underlay extension of a cloud provider network, wherein the provider underlay extension is deployed at a site implementing at least a portion of a first CSP network, coupled with the first CSP network, and managed at least in part by a control plane service of the cloud provider network.
[0108] However, in some embodiments, the provider underlay extension may not be deployed at the site where at least a portion of the CSP network is implemented, and may be a cloud provider network managed PSE 140, a customer managed PSE 144, or the like.
[0109] In some embodiments, the provider underlay extension is implemented between one or more mobile core network functions of the first CSP network and an Internet exchange point present at the edge of the second CSP network. The site may include a data center of the CSP, and in some embodiments, the data center also implements one or more mobile core network functions of the first CSP network.
[0110] Operation 1200 includes causing a first route associated with a compute instance to be advertised to a first CSP network (e.g., via a peering session between a gateway / switch extending over a provider underlay using an edge routing protocol (e.g., BGP) and a network entity of the first CSP), at block 1210, and causing a second route associated with the compute instance to be advertised to a second CSP network peered with at least a portion of the first CSP network (e.g., via a peering session between a gateway / switch extending over a provider underlay (or the first CSP) and a network entity of the second CSP using an edge routing protocol (e.g., BGP), at block 1215. Blocks 1210, 1215 may be performed by a gateway extending over a provider underlay (e.g., by using another network component to advertise the routes), or may be performed by a cloud provider network (or another entity) when configuring the gateway to advertise the routes.
[0111] At block 1220, operation 1200 includes sending, through a gateway of a provider underlay extension, to a computing instance a first set of packets associated with a first CSP initiated by a first one or more user equipment (UE) devices and a second set of packets associated with a second CSP initiated by a second one or more UE devices. In some embodiments, the first set of packets and the second set of packets do not transit the public Internet. For example, in some embodiments, the PSE is directly connected to the first CSP network and the second CSP network, and network traffic transited between the PSE and the first CSP network or the second CSP network can be sent entirely within these networks, respectively.
[0112] In some embodiments, the provider underlay extension is implemented within a transport network of a first CSP network, wherein the transport network is between one or more base stations and a data center hosting one or more control plane mobile core network functions. The one or more data plane mobile core network functions may be implemented within the transport network, and the one or more data plane mobile core network functions terminate the one or more tunnels by decapsulating network traffic sent via one or more tunnels with the one or more base stations initiated by the first one or more UE devices into a first set of packets for sending to a gateway of a PSE. In some embodiments, the data center also hosts the one or more data plane mobile core network functions, and in some embodiments, the transport network further includes a traffic disruption system, wherein the traffic disruption system monitors encapsulated network traffic sent via one or more tunnels between the one or more base stations and the one or more data plane mobile core network functions, identifies portions of the encapsulated network traffic destined for the PSE, and causes those identified portions of the encapsulated network traffic to be directed to the PSE. In order to identify portions of the encapsulated network traffic destined for the PSE, in some embodiments, the traffic disruption system identifies traffic with a known flow identifier (e.g., a combination of one or more values of one or more headers, such as a combination of a protocol value, a source port and a destination port, and a source IP address and a destination IP address) or an access point name (APN).
[0113] In some embodiments, a first CSP provides wireless communication services to a first one or more UEs, wherein the first one or more UEs include wireless computing devices that communicate with a network of the first CSP via one or more base stations.
[0114] In some embodiments, operation 1200 also includes: causing a third route associated with the computing instance to be advertised to a third CSP network that is peered with at least a portion of the first CSP network; and sending a third group of packets originated by a third one or more UE devices associated with the third CSP to the computing instance through the provider underlay extended gateway.
[0115] Fig.13An exemplary provider network (or "service provider system") environment according to some embodiments is shown. The provider network 1300 can provide resource virtualization to customers via one or more virtualization services 1310, which allow customers to purchase, rent, or otherwise obtain instances 1312 of virtualized resources (including but not limited to computing resources and storage resources) implemented on devices within one or more provider networks in one or more data centers. A local Internet Protocol (IP) address 1316 can be associated with the resource instance 1312; the local IP address is the internal network address of the resource instance 1312 on the provider network 1300. In some embodiments, the provider network 1300 can also provide public IP addresses 1314 and / or public IP address ranges (e.g., Internet Protocol version 4 (IPv4) or Internet Protocol version 6 (IPv6) addresses) that customers can obtain from the provider 1300.
[0116] Conventionally, the provider network 1300 can allow, via the virtualization service 1310, customers of the service provider (e.g., customers operating one or more client networks 1350A-1350C including one or more client devices 1352) to dynamically associate at least some public IP addresses 1314 allocated or assigned to the customers with specific resource instances 1312 assigned to the customers. The provider network 1300 can also allow customers to remap a public IP address 1314 previously mapped to one virtualized computing resource instance 1312 assigned to the customer to another virtualized computing resource instance 1312 also assigned to the customer. For example, a customer of a service provider (such as an operator of customer networks 1350A-1350C) can use the virtualized computing resource instances 1312 and public IP addresses 1314 provided by the service provider to implement customer-specific applications and present the customer's applications over an intermediate network 1340 such as the Internet. Other network entities 1320 on the intermediate network 1340 can then generate traffic to the destination public IP address 1314 published by the customer networks 1350A to 1350C; the traffic is routed to the service provider data center and there, via the network underlay, to the local IP address 1316 of the virtualized computing resource instance 1312 that is currently mapped to the destination public IP address 1314. Similarly, response traffic from the virtualized computing resource instance 1312 can be routed back onto the intermediate network 1340 via the network underlay to reach the source entity 1320.
[0117] As used herein, a local IP address refers to an internal or "private" network address of a resource instance, such as in a provider network. A local IP address may be within an address block reserved by Internet Engineering Task Force (IETF) Request for Comments (RFC) 1918 and / or have an address format specified by IETF RFC 4193, and may be variable within a provider network. Network traffic originating from outside the provider network is not routed directly to a local IP address; instead, traffic uses a public IP address that is mapped to the local IP address of the resource instance. The provider network may include network devices or appliances that provide network address translation (NAT) or similar functionality to perform mappings from public IP addresses to local IP addresses, and vice versa.
[0118] A public IP address is an Internet mutable network address assigned to a resource instance by a service provider or by a customer. Traffic routed to a public IP address is translated and forwarded to the corresponding local IP address of the resource instance, for example, via 1:1 NAT.
[0119] Some public IP addresses can be assigned to specific resource instances by the provider network infrastructure; these public IP addresses can be referred to as standard public IP addresses, or simply standard IP addresses. In some embodiments, the mapping of standard IP addresses to the local IP addresses of resource instances is the default startup configuration for all resource instance types.
[0120] At least some public IP addresses may be assigned to or obtained by customers of the provider network 1300; the customer may then assign the public IP address assigned to it to a specific resource instance assigned to the customer. These public IP addresses may be referred to as customer public IP addresses, or simply customer IP addresses. Unlike being assigned to a resource instance by the provider network 1300 as in the case of a standard IP address, a customer IP address may be assigned to a resource instance by a customer, for example, via an API provided by a service provider. Unlike a standard IP address, a customer IP address is assigned to a customer account and may be remapped to other resource instances by the corresponding customer as needed or desired. A customer IP address is associated with a customer account, rather than a specific resource instance, and the customer controls the IP address until the customer chooses to release it. Unlike conventional static IP addresses, customer IP addresses allow customers to shield resource instances or availability zone failures by remapping the customer's public IP address to any resource instance associated with the customer account. For example, a customer IP address enables a customer to solve a problem with a customer resource instance or software by remapping the customer IP address to a replacement resource instance.
[0121] Fig.1414 is a block diagram of an exemplary provider network that provides storage services and hardware virtualization services to customers according to some embodiments. Hardware virtualization service 1420 provides multiple computing resources 1424 (e.g., VMs) to customers. For example, computing resources 1424 can be rented or leased to customers of provider network 1400 (e.g., customers implementing customer network 1450). Each computing resource 1424 can be provided with one or more local IP addresses. Provider network 1400 can be configured to route packets from the local IP addresses of computing resources 1424 to public Internet destinations, and from public Internet sources to the local IP addresses of computing resources 1424.
[0122] The provider network 1400 can provide a customer network 1450 coupled to the intermediate network 1440, for example, via a local network 1456, with the ability to implement a virtual computing system 1492 via a hardware virtualization service 1420 coupled to the intermediate network 1440 and the provider network 1400. In some embodiments, the hardware virtualization service 1420 can provide one or more APIs 1402 (e.g., web service interfaces) via which the customer network 1450 can access functionality provided by the hardware virtualization service 1420, for example, via a console 1494 (e.g., a web-based application, a standalone application, a mobile application, etc.). In some embodiments, at the provider network 1400, each virtual computing system 1492 at the customer network 1450 can correspond to a computing resource 1424 that is leased, rented, or otherwise provided to the customer network 1450.
[0123] A customer may access the functionality of the storage service 1410, for example, from an instance of a virtual computing system 1492 and / or another customer device 1490 (e.g., via a console 1494), via one or more APIs 1402, to access and store data from storage resources 1418A through 1418N of virtual data stores 1416 (e.g., folders or “buckets,” virtualized volumes, databases, etc.) provided by the provider network 1400. In some embodiments, a virtualized data storage gateway (not shown) may be provided at the customer network 1450, which may cache at least some data (e.g., frequently accessed or critical data) locally and may communicate with the storage service 1410 via one or more communication channels to upload new or modified data from the local cache such that a primary storage area for the data (the virtualized data store 1416) is maintained. In some embodiments, users via the virtual computing system 1492 and / or on another client device 1490 can mount and access virtual data storage 1416 volumes via a storage service 1410 acting as a storage virtualization service, and these volumes can appear to the user as local (virtualized) storage devices 1498.
[0124] Although in Fig.14 1400, but the virtualization service may also be accessed from a resource instance within the provider network 1400 via API 1402. For example, a customer, device service provider, or other entity may access the virtualization service via API 1402 from within a corresponding virtual network on the provider network 1400 to request the allocation of one or more resource instances within the virtual network or within another virtual network.
[0125] Descriptive System
[0126] In some embodiments, a system implementing part or all of the techniques described herein may include a general purpose computer system (such as Fig.15 ), the general purpose computer system includes, or is configured to access, one or more computer accessible media. In the illustrated embodiment, the computer system 1500 includes one or more processors 1510 coupled to a system memory 1520 via an input / output (I / O) interface 1530. The computer system 1500 also includes a network interface 1540 coupled to the I / O interface 1530. Although Fig.15 Computer system 1500 is shown as a single computing device, but in various embodiments, computer system 1500 may include one computing device or any number of computing devices configured to work together as a single computer system 1500 .
[0127] In various embodiments, computer system 1500 may be a uniprocessor system including one processor 1510 or a multiprocessor system including several processors 1510 (e.g., two, four, eight, or another suitable number). Processor 1510 may be any suitable processor capable of executing instructions. For example, in various embodiments, processor 1510 may be a general-purpose or embedded processor that implements any of a variety of instruction set architectures (ISAs), such as an x86, ARM, PowerPC, SPARC, or MIPS ISAs or any other suitable ISAs. In a multiprocessor system, each processor 1510 may typically, but not necessarily, implement the same ISA.
[0128] The system memory 1520 may store instructions and data that may be accessed by the processor 1510. In various embodiments, the system memory 1520 may be implemented using any suitable memory technology, such as random access memory (RAM), static RAM (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash-type memory, or any other type of memory. In the illustrated embodiment, program instructions and data that implement one or more desired functions, such as those methods, techniques, and data described above, are shown as stored within the system memory 1520 as service code 1525 and data 1526.
[0129] In one embodiment, the I / O interface 1530 may be configured to coordinate I / O traffic between the processor 1510, the system memory 1520, and any peripheral devices (including the network interface 1540 or other peripheral interfaces) in the device. In some embodiments, the I / O interface 1530 may implement any necessary protocol, timing, or other data transformations to convert data signals from one component (e.g., the system memory 1520) into a format suitable for use by another component (e.g., the processor 1510). In some embodiments, the I / O interface 1530 may include support for devices attached through various types of peripheral buses (e.g., such as variants of the peripheral component interconnect (PCI) bus standard or the universal serial bus (USB) standard). In some embodiments, the functions of the I / O interface 1530 may be divided into two or more separate components, such as, for example, a north bridge and a south bridge. And, in some embodiments, some or all of the functionality of the I / O interface 1530 (such as an interface to the system memory 1520) may be directly incorporated into the processor 1510.
[0130] The network interface 1540 may be configured to allow data to be transferred between the computer system 1500 and other devices 1560 (e.g., such as Figure 1In various embodiments, network interface 1540 can support communication via any suitable wired or wireless general purpose data network (e.g., such as various types of Ethernet). Additionally, network interface 1540 can support communication via a telecommunications / telephone network such as an analog voice network or a digital fiber optic communications network, via a storage area network (SAN) such as a Fiber Channel SAN, or via I / O, any other suitable type of network and / or protocol.
[0131] In some embodiments, computer system 1500 includes one or more offload cards 1570 (including one or more processors 1575 and possibly one or more network interfaces 1540) connected using an I / O interface 1530 (e.g., a bus implementing a version of the Peripheral Component Interconnect-Express (PCI-E) standard or another interconnect such as Quick Path Interconnect (QPI) or Ultra Path Interconnect (UPI)). For example, in some embodiments, computer system 1500 can act as a host electronic device hosting a computing instance (e.g., operating as part of a hardware virtualization service), and one or more offload cards 1570 execute a virtualization manager that can manage computing instances executing on the host electronic device. As an example, in some embodiments, offload card 1570 can perform computing instance management operations, such as pausing and / or unpausing computing instances, starting and / or terminating computing instances, performing memory transfer / copy operations, etc. In some embodiments, these management operations may be performed by offload card 1570 in cooperation with hypervisors (e.g., pursuant to requests from the hypervisors) executed by other processors 1510A through 1510N of computer system 1500. However, in some embodiments, the virtualization manager implemented by offload card 1570 may accommodate requests from other entities (e.g., from the computing instances themselves) and may not cooperate with (or serve) any individual hypervisor.
[0132] In some embodiments, system memory 1520 may be an embodiment of a computer-accessible medium configured to store program instructions and data as described above. However, in other embodiments, program instructions and / or data may be received, sent, or stored on different types of computer-accessible media. In general, a computer-accessible medium may include a non-transitory storage medium or memory medium, such as a magnetic or optical medium, such as a disk or DVD / CD coupled to computing device 1500 via I / O interface 1530. A non-transitory computer-accessible storage medium may also include any volatile or non-volatile medium, such as RAM (e.g., SDRAM, double data rate (DDR) SDRAM, SRAM, etc.), read-only memory (ROM), etc., which may be included in some embodiments of computer system 1500 as system memory 1520 or another type of memory. In addition, a computer-accessible medium may include a transmission medium or signal, such as an electrical signal, an electromagnetic signal, or a digital signal, communicated via a communication medium (such as a network and / or a wireless link, such as may be implemented via network interface 1540).
[0133] The various embodiments discussed or proposed herein can be implemented in a wide variety of operating environments, which in some cases can include one or more user computers, computing devices, or processing devices that can be used to operate any of a number of applications. User or client devices can include: any of a number of general-purpose personal computers, such as desktop computers or notebook computers running standard operating systems; and cellular devices, wireless devices, and handheld devices that run mobile software and can support a number of networking protocols and messaging protocols. Such a system can also include a number of workstations that run a variety of commercially available operating systems and any of other known applications for purposes such as development and database management. These devices can also include other electronic devices, such as virtual terminals, thin clients, gaming systems, and / or other devices that can communicate via a network.
[0134] Most embodiments utilize at least one network with which those skilled in the art will be familiar to support communications using any of a variety of widely available protocols, such as Transmission Control Protocol / Internet Protocol (TCP / IP), File Transfer Protocol (FTP), Universal Plug and Play (UPnP), Network File System (NFS), Common Internet File System (CIFS), Extensible Messaging and Presence Protocol (XMPP), AppleTalk, etc. The network may include, for example, a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), the Internet, an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network, and any combination thereof.
[0135] In embodiments utilizing a web server, the web server may run any of a variety of server or middle-tier applications, including HTTP servers, file transfer protocol (FTP) servers, common gateway interface (CGI) servers, data servers, Java servers, streaming application servers, etc. The server may also be capable of responding to requests from user devices (such as by executing a program that may be implemented in any programming language (such as The server may also include a database server, including but not limited to commercially available database servers from Oracle (R), Microsoft (R), Sybase (R), IBM (R), etc. The database server may be relational or non-relational (e.g., "NoSQL"), distributed or non-distributed, etc.
[0136] The environment disclosed herein may include a variety of data storage areas and other memories and storage media as discussed above. These may reside in a variety of locations, such as on a storage medium that resides locally (and / or resides therein) on one or more computers, or on a storage medium that resides across a network away from any or all of the computers. In a particular concentration of the embodiment, information may reside in a storage area network (SAN) familiar to those skilled in the art. Similarly, any necessary files for performing functions belonging to a computer, server, or other network device may be stored locally or remotely as appropriate. In the case where the system includes a computerized device, each such device may include a hardware element that may be electrically coupled via a bus, the element including, for example, at least one central processing unit ("CPU"), at least one input device (e.g., a mouse, keyboard, controller, touch screen, or keypad) and / or at least one output device (e.g., a display device, printer, or speaker). Such a system may also include one or more storage devices, such as a hard disk drive, an optical storage device, and a solid-state storage device such as a random access memory (RAM) or a read-only memory (ROM), as well as a removable media device, a memory card, a flash memory card, etc.
[0137] Such devices may also include a computer-readable storage medium reader, a communication device (e.g., a modem, a network card (wireless or wired), an infrared communication device, etc.), and a working memory, as described above. A computer-readable storage medium reader may be connected to or configured to receive a computer-readable storage medium, which represents a remote, local, fixed and / or removable storage device and a storage medium for temporarily and / or more permanently containing, storing, transmitting, and retrieving computer-readable information. Systems and various devices will also typically include a plurality of software applications, modules, services, or other elements located within at least one working memory device, including an operating system and an application, such as a client application or a web browser. It should be understood that alternative embodiments may have a large number of variations different from those described above. For example, custom hardware may also be used, and / or specific elements may be implemented in hardware, software (including portable software, such as applets), or both. In addition, connections to other computing devices such as network input / output devices may be employed.
[0138] Storage media and computer-readable media for containing code or portions of code may include any suitable media known or used in the art, including storage media and communication media, such as but not limited to volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing and / or transmitting information (such as computer-readable instructions, data structures, program modules or other data), including RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk memory (CD-ROM), digital versatile disk (DVD) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be used to store the desired information and can be accessed by the system device. Based on the disclosure and teachings provided herein, those skilled in the art will understand other ways and / or methods for implementing various embodiments.
[0139] In the above description, various embodiments are described. For explanation purposes, specific configurations and details are set forth to provide a thorough understanding of the embodiments. However, it will also be apparent to those skilled in the art that the embodiments may be practiced without the specific details. In addition, in order not to obscure the described embodiments, well-known features may be omitted or simplified.
[0140] Bracketed text and boxes with dashed borders (e.g., large dashes, small dashes, dot dashes, and dots) are used herein to illustrate optional operations that add additional features to some embodiments. However, this notation should not be taken to mean that these are the only options or optional operations and / or that boxes with solid borders are not optional in certain embodiments.
[0141] In various embodiments, reference numerals with suffix letters (e.g., 1418A through 1418N) may be used to indicate that there may be one or more instances of the referenced entity, and that when multiple instances exist, each instance need not be identical, but may instead share some general features or act conventionally. Furthermore, unless expressly stated to the contrary, the particular suffix used is not intended to imply the presence of a particular quantity of the entity. Thus, in various embodiments, two entities using the same or different suffix letters may or may not have the same number of instances.
[0142] References to "one embodiment," "an embodiment," "an exemplary embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Furthermore, such phrases are not necessarily referring to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be understood that it is within the knowledge of those skilled in the art to implement such feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.
[0143] Furthermore, in the various embodiments described above, unless specifically noted otherwise, disjunctive language such as the phrase "at least one of A, B, or C" is intended to be understood to mean A, B, or C, or any combination thereof (e.g., A, B, and / or C). Thus, disjunctive language is generally not intended and should not be understood to imply that a given embodiment requires that at least one A, at least one B, or at least one C each be present.
[0144] At least some embodiments of the disclosed technology may be described in terms of the following:
[0145] 1. A computer-implemented method comprising:
[0146] launching a compute instance on a provider underlay extension of a cloud provider network, wherein the provider underlay extension is deployed within a site of a first communications service provider (CSP), the site comprising computing resources implementing at least a portion of a mobile core network, the computing resources comprising one or more control plane functions and one or more data plane functions of a mobile telecommunications network of the first CSP, wherein the provider underlay extension is communicatively coupled between the mobile core network and an Internet Exchange Point (IXP), and wherein the provider underlay extension is at least partially managed by a control plane service of the cloud provider network;
[0147] causing a first route associated with the computing instance to be advertised to the first CSP network;
[0148] causing a second route associated with the computing instance to be advertised to a second CSP network that is peered with at least a portion of the first CSP network; and
[0149] receiving, by a gateway of the provider underlay extension, a first set of packets originated by a first one or more user equipment (UE) devices, wherein the first set of packets are sent through one or more transport networks via a tunnel of a mobile overlay network of the first CSP network, exit the tunnel at the mobile core network, and are routed to the provider underlay extension via the first CSP network;
[0150] receiving, by the gateway of the provider underlay extension, a second set of packets originated by a second one or more UE devices and routed to the provider underlay extension via the second CSP network through the IXP; and
[0151] The first group of packets and the second group of packets are forwarded to the computing instance through the gateway.
[0152] 2. The computer-implemented method of clause 1, wherein:
[0153] The first CSP provides wireless communication services to the first one or more UE devices;
[0154] each of the first one or more UE devices comprises a wireless computing device in communication with the first CSP network via first one or more base stations;
[0155] The second CSP provides wireless communication services to the second one or more UE devices; and
[0156] The second one or more UE devices include wireless computing devices that communicate with the second CSP network via second one or more base stations.
[0157] 3. The computer-implemented method of clause 2, wherein the first set of packets and the second set of packets do not transit the public Internet.
[0158] 4. A computer-implemented method comprising:
[0159] launching a compute instance on a provider underlay extension of a cloud provider network, wherein the provider underlay extension is deployed at a site implementing at least a portion of a first communications service provider (CSP) network, coupled to the first CSP network, and managed at least in part by a control plane service of the cloud provider network;
[0160] causing a first route associated with the computing instance to be advertised to the first CSP network;
[0161] causing a second route associated with the computing instance to be advertised to a second CSP network that is peered with at least a portion of the first CSP network; and
[0162] A first set of packets originated by first one or more user equipment (UE) devices associated with the first CSP and a second set of packets originated by second one or more UE devices associated with the second CSP are sent to the computing instance through the provider underlay extended gateway.
[0163] 5. The computer-implemented method of clause 4, wherein the provider underlay extension is implemented between one or more mobile core network functions of the first CSP network and an Internet exchange point present at an edge of the second CSP network.
[0164] 6. The computer-implemented method of clause 5, wherein the site also implements the one or more mobile core network functions of the first CSP network.
[0165] 7. A computer-implemented method as described in clause 4, wherein the provider underlay extension is implemented within a transport network of the first CSP network, wherein the transport network is between one or more base stations and a CSP site of the first CSP hosting one or more control plane mobile core network functions.
[0166] 8. A computer-implemented method according to clause 7, wherein one or more data plane mobile core network functions are implemented within the transport network, and wherein the one or more data plane mobile core network functions terminate one or more tunnels for one or more base stations by: decapsulating network traffic initiated by the first one or more UE devices and sent via the one or more tunnels into the first group of packets for sending to the gateway of the provider underlying extension.
[0167] 9. The computer-implemented method of any of clauses 7 to 8, wherein the CSP site also hosts one or more data plane mobile core network functions.
[0168] 10. A computer-implemented method according to clause 9, wherein the transport network further comprises a traffic disruption system, wherein the traffic disruption system monitors encapsulated network traffic sent via one or more tunnels between the one or more base stations and the one or more data plane mobile core network functions, identifies portions of the encapsulated network traffic destined for the provider underlying extension, and causes those identified portions of the encapsulated network traffic to be directed to the provider underlying extension.
[0169] 11. A computer-implemented method according to clause 10, wherein in order to identify the portion of the encapsulated network traffic destined for the provider underlying extension, the traffic disruption system identifies traffic with a known flow identifier based on one or more packet header values or a known access point name (APN).
[0170] 12. A computer-implemented method as described in any of clauses 4 to 11, wherein the first CSP provides wireless communication services to the first one or more UE devices, and wherein the first one or more UE devices include wireless computing devices that communicate with the first CSP network via one or more base stations.
[0171] 13. The computer-implemented method of any one of clauses 4 to 12, further comprising:
[0172] causing a third route associated with the computing instance to be advertised to a third CSP network that is peered with at least a portion of the first CSP network; and
[0173] A third set of packets originated by a third one or more UE devices associated with the third CSP network is sent to the computing instance through the gateway of the provider underlay extension.
[0174] 14. The computer-implemented method of any of clauses 4 to 13, wherein the first set of packets and the second set of packets do not transit the public Internet.
[0175] 15. A computer-implemented method according to any one of clauses 4 to 14, wherein:
[0176] The computing instance is accessible within the first CSP network via a first network address within an address space of the first CSP network; and
[0177] The computing instance is accessible within the second CSP network via a second network within an address space of the second CSP network.
[0178] 16. The computer-implemented method of clause 4, wherein:
[0179] The computing instance is accessible within the first CSP network via a first network address; and
[0180] The computing instance is accessible within the second CSP network via the first network address.
[0181] 17. A system comprising:
[0182] a cloud provider network implemented by the first one or more electronic devices, the cloud provider network comprising instructions that, when executed, cause the cloud provider network, upon receiving a request to launch a computing instance on behalf of a user of the cloud provider network, to send one or more messages to a provider underlay extension of the cloud provider network to launch the computing instance; and
[0183] The provider underlay extension of the cloud provider network, the provider underlay extension being implemented by a second one or more electronic devices, wherein one or more of the second one or more electronic devices provide capabilities for executing computing instances, and wherein the provider underlay extension is controlled at least in part by a control plane of the cloud provider network, the provider underlay extension comprising instructions that, when executed, cause the provider underlay extension to:
[0184] launching the computing instance in the provider underlay extension, wherein the provider underlay extension is deployed at a site implementing at least a portion of a first communications service provider (CSP) network and coupled to the first CSP network;
[0185] causing a first route associated with the computing instance to be advertised to the first CSP network;
[0186] causing a second route associated with the computing instance to be advertised to a second CSP network that is peered with at least a portion of the first CSP network; and
[0187] A first set of packets originated by first one or more user equipment (UE) devices associated with the first CSP and a second set of packets originated by second one or more UE devices associated with the second CSP are sent to the computing instance through the provider underlay extended gateway.
[0188] 18. A system according to clause 17, wherein:
[0189] The provider underlay extension is implemented between one or more mobile core network functions of the first CSP network and an Internet exchange point present at the edge of the second CSP network;
[0190] The site comprises a data center of the first CSP; and
[0191] The data center also implements the one or more mobile core network functions of the first CSP network.
[0192] 19. A system according to clause 17, wherein:
[0193] The provider underlay extension is implemented within a transport network of the first CSP network, wherein the transport network is between one or more base stations and a data center hosting one or more control plane mobile core network functions.
[0194] 20. A system according to clause 19, wherein:
[0195] One or more data plane mobile core network functions are implemented within the transport network, and wherein the one or more data plane mobile core network functions terminate the one or more tunnels by decapsulating network traffic initiated by the first one or more UE devices and sent via one or more tunnels with the one or more base stations into the first group of packets for sending to the gateway of the provider underlay extension.
[0196] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure as set forth in the appended claims.
Claims
1. A computer-implemented method comprising: launching a compute instance on a provider underlay extension of a cloud provider network, wherein the provider underlay extension is deployed within a site implementing at least a portion of a first communications service provider (CSP) network, wherein the provider underlay extension is communicatively coupled between one or more mobile core network functions and an Internet exchange point of the first CSP network and is at least partially managed by a control plane service of the cloud provider network; causing a first route associated with the computing instance to be advertised to the first CSP network; causing a second route associated with the computing instance to be advertised to a second CSP network that is peered with at least a portion of the first CSP network; as well as A first set of packets originated by first one or more user equipment (UE) devices associated with the first CSP and a second set of packets originated by second one or more UE devices associated with the second CSP are sent to the computing instance through the provider underlay extended gateway.
2. The computer-implemented method of claim 1 , wherein an edge of the second CSP network exists at the Internet exchange point.
3. The computer-implemented method of claim 2, wherein the site also implements the one or more mobile core network functions of the first CSP network.
4. The computer-implemented method of claim 1 , wherein the provider underlay extension is implemented within a transport network of the first CSP network, wherein the transport network is between one or more base stations and a CSP site of the first CSP hosting one or more control plane mobile core network functions.
5. The computer-implemented method of claim 4 , wherein one or more data plane mobile core network functions are implemented within the transport network, and wherein the one or more data plane mobile core network functions terminate one or more tunnels for one or more base stations by decapsulating network traffic initiated by the first one or more UE devices and sent via the one or more tunnels into the first set of packets for sending to the gateway of the provider underlay extension.
6. The computer-implemented method of claim 4, wherein the CSP site also hosts one or more data plane mobile core network functions.
7. The computer-implemented method of claim 6, wherein the transport network further comprises a traffic disruption system, wherein the traffic disruption system monitors encapsulated network traffic sent via one or more tunnels between the one or more base stations and the one or more data plane mobile core network functions, identifies portions of the encapsulated network traffic destined for the provider underlay extension, and causes those identified portions of the encapsulated network traffic to be directed to the provider underlay extension.
8. The computer-implemented method of claim 7, wherein to identify the portion of the encapsulated network traffic destined for the provider underlay extension, the traffic disruption system identifies traffic with a known flow identifier based on one or more packet header values or a known access point name (APN).
9. The computer-implemented method of any one of claims 1 to 8, wherein the first CSP provides wireless communication services to the first one or more UE devices, and wherein the first one or more UE devices include wireless computing devices that communicate with the first CSP network via one or more base stations.
10. The computer-implemented method of any one of claims 1 to 8, further comprising: causing a third route associated with the computing instance to be advertised to a third CSP network that is peered with at least a portion of the first CSP network; as well as A third set of packets originated by a third one or more UE devices associated with the third CSP network is sent to the computing instance through the gateway of the provider underlay extension.
11. The computer-implemented method of any one of claims 1 to 8, wherein the first set of packets and the second set of packets do not transit the public Internet.
12. The computer-implemented method of any one of claims 1 to 8, wherein: The computing instance is accessible within the first CSP network via a first network address within an address space of the first CSP network; and The computing instance is accessible within the second CSP network via a second network within an address space of the second CSP network.
13. The computer-implemented method of claim 1 , wherein: The computing instance is accessible within the first CSP network via a first network address; and The computing instance is accessible within the second CSP network via the first network address.
14. A computer-implemented system comprising: a cloud provider network implemented by a first one or more electronic devices, the cloud provider network including instructions that, when executed, cause the cloud provider network, upon receiving a request to launch a computing instance on behalf of a user of the cloud provider network, to send one or more messages to a provider underlay extension of the cloud provider network to launch the computing instance; as well as The provider underlay extension of the cloud provider network, the provider underlay extension being implemented by a second one or more electronic devices, wherein one or more of the second one or more electronic devices provide capabilities for executing computing instances, and wherein the provider underlay extension is controlled at least in part by a control plane of the cloud provider network, the provider underlay extension comprising instructions that, when executed, cause the provider underlay extension to: launching the computing instance in the provider underlay extension, wherein the provider underlay extension is deployed within a site implementing at least a portion of a first communications service provider (CSP) network and wherein the provider underlay extension is communicatively coupled between one or more mobile core network functions and an Internet exchange point of the first CSP network; causing a first route associated with the computing instance to be advertised to the first CSP network; causing a second route associated with the computing instance to be advertised to a second CSP network that is peered with at least a portion of the first CSP network; as well as A first set of packets originated by first one or more user equipment (UE) devices associated with the first CSP and a second set of packets originated by second one or more UE devices associated with the second CSP are sent to the computing instance through the provider underlay extended gateway.
15. The computer-implemented system of claim 14, wherein: The provider underlay extension is implemented between one or more mobile core network functions of the first CSP network and an Internet exchange point present at the edge of the second CSP network; The site comprises a data center of the first CSP; and The data center also implements the one or more mobile core network functions of the first CSP network.
