Wireless private network management

Abstract

To manage a wireless private network.SOLUTION: In an embodiment, a cellular network includes at least one cell providing wireless private network coverage for an organization's site. A system further includes at least one computing device in a cloud provider network that implements one or more network functions for an associated core network of the wireless private network.SELECTED DRAWING: Figure 1B

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of, and priority to, a U.S. patent application entitled "MANAGING RADIO-BASED PRIVATE NETWORKS," filed on December 10, 2020, and assigned application serial number 17 / 118,563, which is incorporated herein by reference in its entirety. [Background technology]

[0002] 5G is the fifth-generation technology standard for broadband cellular networks and is planned to eventually replace the fourth-generation (4G) standard of Long-Term Evolution (LTE). 5G technology will offer significantly increased bandwidth, thereby expanding the cellular market beyond smartphones to provide last-mile connectivity to desktops, set-top boxes, laptops, Internet of Things (IoT) devices, and more. Some 5G cells may use frequency spectrum similar to 4G, while other 5G cells may use millimeter wave frequency spectrum. Millimeter wave cells have a relatively small coverage area but will offer much higher throughput than 4G. Summary of the Invention [Means for solving the problem]

[0003] Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views. [Brief explanation of the drawings]

[0004] [Figure 1A]1 is a drawing of an example communications network deployed and managed in accordance with various embodiments of the present disclosure. [Figure 1B] 1 is an example of a wireless private network used within the campus of a multi-building organization and deployed in accordance with various embodiments of the present disclosure. [Figure 2A] 1 illustrates an example of a network environment including a cloud provider network according to some embodiments of the present disclosure, and further including various provider substrate extensions of the cloud provider network that may be used at various locations within the communications network of FIG. [Figure 2B] 2 illustrates an example of cellularization and geographical distribution of the communication network of FIG. 1 to provide a highly available User Plane Function (UPF). [Figure 3] 2B illustrates an example of the network environment of FIG. 2A including geographically distributed provider substrate extensions, according to some embodiments of the present disclosure. [Figure 4] FIG. 2B is a schematic block diagram of the network environment of FIG. 2A in accordance with various embodiments of the present disclosure. [Figure 5] 5 is a flowchart illustrating example functionality implemented as part of a wireless private network management service executing in a computing environment within the network environment of FIG. 4, according to various embodiments of the present disclosure. [Figure 6] 5 is a flowchart illustrating example functionality implemented as part of a wireless private network management service executing in a computing environment within the network environment of FIG. 4, according to various embodiments of the present disclosure. [Figure 7] 5 is a flowchart illustrating example functionality implemented as part of a wireless private network management service executing in a computing environment within the network environment of FIG. 4, according to various embodiments of the present disclosure. [Figure 8]5 is a flowchart illustrating an example of functionality implemented as part of a capacity management service executing in a computing environment within the network environment of FIG. 4, according to various embodiments of the present disclosure. [Figure 9] 5 is a schematic block diagram providing an illustration of an example computing environment for use in the network environment of FIG. 4, in accordance with various embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0005] The present disclosure relates to the automated deployment, modification, and management of wireless private networks, such as 4G and 5G radio access networks, or portions of such wireless networks, and associated core networks, using cloud provider network infrastructure. Previous deployments of wireless networks relied on manual deployment and configuration at each step of the process, which proved to be extremely time-consuming and expensive. Furthermore, in previous generations, software was essentially tied to vendor-proprietary hardware, preventing customers from deploying alternative software. In contrast, 5G decouples the hardware from the software stack, allowing for greater flexibility and allowing components of wireless networks to run on cloud provider infrastructure. Using a cloud delivery model for wireless networks, such as 5G networks, can facilitate handling network traffic from hundreds to billions of connected devices and computationally intensive applications, while delivering faster speeds, lower latency, and greater capacity than other types of networks.

[0006] Until now, businesses have had to choose between performance and price when evaluating enterprise connectivity solutions. Cellular networks can offer high performance, excellent indoor and outdoor coverage, and advanced quality of service (QoS) connectivity features, but private cellular networks can be expensive and complex to manage. Ethernet and Wi-Fi require less upfront investment and are easier to manage, but businesses often find they are unreliable, require a lot of work to achieve optimal coverage, and do not offer QoS features such as guaranteed bit rate, latency, and reliability.

[0007] The disclosed private wireless network service offers businesses the best of both worlds: carrier-grade cellular network performance, coverage, and QoS, with the ease and cost of deployment and operation associated with Wi-Fi. The disclosed service provides the appropriate hardware in various form factors that businesses can deploy at their sites, integrated with the software that runs the entire network, from small cell sites to internet breakouts. Businesses can freely deploy various 5G devices and sensors across their enterprise (factory floors, warehouses, lobbies, and communications centers) and manage these devices, register users, and assign QoS from a management console. Using the disclosed technology, customers can allocate a certain bitrate of throughput to every device (such as a camera, sensor, or IoT device), a reliable, low-latency connection to devices operating on the factory floor, and a broadband connection to every handheld device. The disclosed service manages all the software necessary to provide connectivity that meets specified constraints and requirements. This enables an entirely new set of applications with stringent QoS or high IoT device density requirements that previously could not be run on Wi-Fi networks.

[0008] The disclosed service supports multiple deployment scenarios. In a cloud-only deployment, the service can provide a small wireless cell that an enterprise customer can place on-site, while network functions and other network software run in the nearest (or several nearest) cloud provider availability zones or edge locations. For enterprises preferring on-premises deployment, the disclosed service offers cloud provider hardware, such as the substrate extension described herein. In this mode, the network and applications remain on-premises for the enterprise, allowing the enterprise to securely store and process data that needs to remain local (e.g., for regulatory compliance, security concerns, etc.). Furthermore, the disclosed service allows any compute and storage not being used to run the wireless network to be used to run any local workloads via the same APIs that customers can use to run workloads in traditional cloud provider regions. Thus, as an advantage, the service allows any excess capacity to be used for local processing, and enterprises do not need to worry about overscaling and wasting capacity, as new hardware and software is provisioned as network needs change. Additionally, the disclosed service can provide application development APIs that expose and manage 5G features such as QoS, enabling customers to build applications that can take full advantage of the network's latency and bandwidth capabilities without needing to understand the network details.

[0009] Additionally, the disclosed service can provide a private zone for running local applications within the cloud provider network. This private zone can be connected to a broader regional zone, effectively making it a part of it, allowing customers to manage the private zone using the same APIs and tools used in the cloud provider network. Similar to availability zones, private zones can be assigned virtual private network subnets. APIs can be used to create subnets and assign them to all zones a customer wants to use, including private zones and other existing zones. The management console may provide a simplified process for creating private zones. Virtual machine instances and containers can be launched in private zones just like in regional zones. Customers can configure network gateways to define routes, assign IP addresses, set up network address translation (NAT), and so on. Autoscaling allows the private zone to scale its virtual machine instance or container capacity as needed. The same management and authentication APIs as the cloud provider network can be used within the private zone. In some cases, cloud services available in the regional zone can be accessed remotely from the private zone over a secure connection, allowing access to these cloud services without having to upgrade or modify local deployments.

[0010] Various embodiments of the present disclosure introduce an approach that enables customers to order and deploy wireless private networks and associated core networks in an automated manner. Customers may include businesses or organizations that want to set up wireless networks (e.g., private 5G networks) for their internal use. Through various user interfaces, customers can specify their network plans or needs (e.g., physical site layout, device / application types and quantities), and the various components needed to implement the wireless private network for the customer are automatically determined and provisioned. Hardware such as antennas, radios, and computer servers may be pre-configured for the customer's wireless private network and shipped to the customer. The process of installing the pre-configured hardware is primarily plug-and-play, and the wireless private network can be activated through a user interface or API. In addition to deploying wireless private networks, such as all or part of a new radio access network, various embodiments of the present disclosure may facilitate modifying and managing wireless private networks, such as deploying pre-configured equipment for additional cells or assigning QoS constraints to specific devices or applications on the wireless private network.

[0011] Various embodiments of the present disclosure may also incorporate concepts of elasticity and utility computing from the cloud computing model into wireless private networks and associated core networks. For example, the disclosed techniques may execute core and radio access network functions and associated control plane management functions on cloud provider infrastructure, creating a cloud-native core network and / or a cloud-native radio access network (RAN). Such core and RAN network functions may, in some implementations, be based on 3rd Generation Partnership Project (3GPP) specifications. Providing a cloud-native wireless network allows customers to dynamically scale their wireless private networks based on usage, latency requirements, and / or other factors. In some cases, the hardware delivered to a customer may include sufficient capacity to run both programs for operating and managing the wireless network and the customer's other workloads (e.g., its applications), and any capacity not used by the wireless network may be accessible for running workloads under a utility computing model. Advantageously, the customer's wireless network can scale up to this excess capacity as needed, allowing, for example, the hardware usage requirements of the wireless network to increase even before new physical hardware is provisioned for the customer. Customers may also configure thresholds to receive alerts regarding over-the-air network usage and excess capacity usage of provisioned infrastructure to more effectively manage the provisioning of new infrastructure or de-provisioning of existing infrastructure based on dynamic networking and workload requirements.

[0012] As one of ordinary skill in the art will appreciate in light of this disclosure, particular embodiments may be capable of achieving certain advantages, including some or all of the following:(2) improving the flexibility of computer systems by allowing computing hardware previously dedicated to network functions in a wireless network and associated core network to be reused for other applications; (3) improving the flexibility of computer systems by allowing computing hardware previously dedicated to a first wireless private network to be automatically reused for a second wireless private network (or between RAN and core network functions of the same wireless network, as appropriate); (4) improving the user experience when deploying wireless private networks by pre-configuring antennas, radios, and other hardware, thereby providing a plug-and-play installation experience; and (5) improving the user experience when deploying wireless private networks by allowing organizations to easily deploy and manage cell deployment and spectrum. (6) improving wireless network performance and management by monitoring performance metrics and adding, removing, and reconfiguring cells as needed to maintain acceptable performance; (7) improving the scalability and overall performance of the wireless private network by transferring network functions previously provided by proprietary hardware to virtual machine instances operated by a cloud computing provider with elasticity under a utility computing model; (8) reducing latency in the wireless private network by transferring network functions to virtual machine instances running on cloud service provider computing devices located at cell sites; and (9) improving the security of the wireless private network by configuring network function workloads to remain on the customer's premises.

[0013] Among the advantages of the present disclosure is the ability to deploy and chain network functions to deliver end-to-end services that meet specified constraints and requirements. According to the present disclosure, network functions organized into microservices work together to provide end-to-end connectivity. One set of network functions is part of the wireless network, operating within base stations and performing radio signal to IP conversion. Other network functions run in large data centers that execute subscriber-related business logic and route IP traffic to and from the Internet. For applications to use the new features of 5G, such as low-latency communications and reserved bandwidth, both of these types of network functions must work together to properly schedule and reserve wireless spectrum and perform real-time computations and data processing. The techniques disclosed herein provide edge location hardware (described further below) integrated with network functions that run throughout the network, from cell sites to internet breakouts, to orchestrate the network functions to meet required quality of service (QoS) constraints. This enables an entirely new set of applications with stringent QoS requirements that were previously not possible to run over mobile networks, ranging from factory-based Internet of Things (IoT) to augmented reality (AR), virtual reality (VR), game streaming, and autonomous navigation support for connected vehicles.

[0014] The described "Elastic 5G" service provides and manages all the hardware, software, and network functions necessary to build a network. In some embodiments, network functions may be developed and managed by a cloud service provider, but the described control plane can manage network functions across various providers, allowing customers to invoke and manage selected network functions on their cloud infrastructure using a single set of APIs. The Elastic 5G service has the advantage of automating the creation of end-to-end 5G networks, from hardware to network functions, thereby reducing deployment time and operational costs for network operation. By providing APIs that expose network functions, the disclosed Elastic 5G service enables applications to specify desired QoS as constraints and then simply deploy and chain network functions to deliver end-to-end services that meet the specified requirements, thereby facilitating the building of new applications.

[0015] This disclosure describes embodiments related to creating and managing a cloud-native 5G core and / or cloud-native 5G RAN and associated control plane components. Cloud-native refers to an approach to building and running applications that leverage the benefits of the cloud computing delivery model, such as dynamic scalability, distributed computing, and high availability (including geographic distribution, redundancy, and failover). Cloud-native refers to how these applications are created and deployed to make them suitable for deployment in a public cloud. Cloud-native applications can (and often do) run in a public cloud, but they can also run in on-premises data centers. Some cloud-native applications can be containerized, e.g., different parts, functions, or subunits of an application are packaged into their own containers, which can be dynamically orchestrated so that each part is actively scheduled and managed to optimize resource utilization. These containerized applications can be built using a microservices architecture to improve the overall agility and maintainability of the application.

[0016] In a microservices architecture, an application is arranged as a collection of smaller subunits (“microservices”) that can be deployed and scaled independently of each other and can communicate with each other over a network. These microservices typically have a specific technical and functional granularity and often implement lightweight communication protocols, resulting in fine-grained granularity. An application's microservices can perform different functions, may be independently deployable, and may use different programming languages, databases, and hardware / software environments. Decomposing an application into smaller services beneficially increases the modularity of the application, enables individual microservices to be replaced as needed, and parallelizes development by allowing teams to develop, deploy, and maintain them independently of each other. Microservices may, in some examples, be deployed using virtual machines, containers, or serverless functions. The disclosed core and RAN software may follow a microservices architecture such that the described wireless network is composed of independent subunits that can be deployed and scaled on demand.

[0017] 1A , an example of a communications network 100 deployed and managed in accordance with various embodiments of the present disclosure is shown. The communications network 100 includes a wireless private network 103 that may correspond to a cellular network, such as a fourth-generation (4G) Long-Term Evolution (LTE) network, a fifth-generation (5G) network, a 4G-5G hybrid core with both 4G and 5G RAN, or another network that provides wireless network access. The wireless private network 103 may be operated by a cloud service provider for a business, nonprofit organization, school system, government agency, or other organization. Although referred to as a private network, the wireless private network 103 may use private or public network addresses in various embodiments.

[0018] Various deployments of the wireless private network 103 may include one or more of a core network and a RAN network, as well as a control plane for running the core network and / or the RAN network on a cloud provider infrastructure. As noted above, these components may be developed in a cloud-native manner, for example, using a microservices architecture, so that centralized control and distributed processing are used to efficiently scale traffic and transactions. These components may be based on 3GPP specifications by following an application architecture with separated control and user plane processing (CUPS architecture).

[0019] The wireless private network 103 provides wireless network access to multiple wireless devices 106, which may be mobile devices or fixed location devices. In various examples, the wireless devices 106 may include devices such as smartphones, connected vehicles, IoT devices, sensors, machines (such as in a manufacturing facility), hotspots, etc. The wireless devices 106 may also be referred to as user equipment (UE) or customer premises equipment (CPE).

[0020] The wireless private network 103 may include a radio access network (RAN) that provides wireless network access to multiple wireless devices 106 through multiple cells 109. Each of the cells 109 may be equipped with one or more antennas and one or more radio units that transmit and receive wireless data signals to and from the wireless devices 106. The antennas may be configured for one or more frequency bands, and the radio units may also be frequency agile or frequency tunable. To focus signals in a particular direction or azimuth range, the antennas may be associated with a particular gain or beamwidth, which may enable frequency reuse in different directions. Furthermore, the antennas may be horizontally, vertically, or circularly polarized. In some examples, the radio units may transmit and receive signals utilizing multiple-input, multiple-output (MIMO) technology. Thus, the RAN implements radio access technologies that enable wireless connections with the wireless devices 106 and provides connectivity to the wireless private network's core network. Components of the RAN include base stations and antennas covering a given physical area, as well as the necessary core network items for managing connections to the RAN.

[0021] Data traffic is often routed to the core network over a fiber transport network consisting of multiple hops of Layer 3 routers (e.g., at aggregation sites). The core network is typically housed in one or more data centers. Typically, the core network aggregates data traffic from end devices, authenticates subscribers and devices, applies personalized policies, and manages device mobility before routing traffic to operator services or the Internet. For example, the 5G core can be decomposed into several microservice elements, separating the control plane and the user plane. Rather than physical network elements, the 5G core can include virtualized, software-based network functions (e.g., deployed as microservices) and thus can be instantiated within a multi-access edge computing (MEC) cloud infrastructure. Network functions in the core network can include a user plane function (UPF), an access and mobility management function (AMF), and a session management function (SMF), which are described in more detail below. For data traffic destined for locations external to communications network 100, network functions typically include firewalls to external networks, such as the Internet or cloud provider networks, through which traffic can enter and exit communications network 100. Note that in some embodiments, communications network 100 can include facilities that allow traffic to enter and exit sites further downstream from the core network (e.g., aggregation sites or wireless private networks 103).

[0022] The UPF provides the interconnection point between the mobile infrastructure and the data network (DN), i.e., General Packet Radio Service (GPRS) Tunneling Protocol for User Plane (GTP-U) encapsulation and decapsulation. The UPF may also provide a session anchor point for providing mobility within the RAN, such as sending one or more end marker packets to the RAN base station. The UPF may also handle packet routing and forwarding, such as steering flows to specific data networks based on traffic match filters. Another function of the UPF includes per-flow or per-application QoS handling, such as uplink (UL) and downlink (DL) transport-level packet marking and rate limiting. The UPF can be implemented as a cloud-native network function using modern microservices techniques, for example, deployed within a serverless framework (which abstracts the underlying infrastructure on which code runs via managed services).

[0023] The AMF may receive connection and session information from the wireless device 106 or the RAN and may handle connection and mobility management tasks. For example, the AMF may manage handovers between base stations within the RAN. In some examples, the AMF may be considered an access point to the 5G core by terminating traffic for a particular RAN control plane and wireless device 106. The AMF may also implement encryption and integrity protection algorithms.

[0024] The SMF may handle session establishment or modification, for example, by creating, updating, and deleting Protocol Data Unit (PDU) sessions and managing session context within the UPF. The SMF may also implement Dynamic Host Configuration Protocol (DHCP) and IP Address Management (IPAM). The SMF may also be implemented as a cloud-native network function using modern microservices approaches.

[0025] Various network functions for implementing wireless private network 103 may be deployed within distributed computing device 112, which may correspond to a general-purpose computing device configured to perform network functions. For example, distributed computing device 112 may run one or more virtual machine instances that are in turn configured to run one or more services that implement the network functions. In one embodiment, distributed computing device 112 is a ruggedized machine deployed at each cell site.

[0026] In contrast, one or more centralized computing devices 115 may perform various network functions at a central site operated by a customer. For example, the centralized computing devices 115 may be centrally located on the customer's premises in a coordinated server room. The centralized computing devices 115 may run one or more virtual machine instances that are in turn configured to run one or more services that implement the network functions.

[0027] In one or more embodiments, network traffic from wireless private network 103 is backhauled to one or more core computing devices 118, which may be located in one or more data centers located remotely from customer sites. Core computing devices 118 may also perform various network functions, including routing network traffic to and from network 121, which may correspond to the Internet and / or other external public or private networks. Core computing devices 118 may perform functions related to management of communication network 100 (e.g., billing, mobility management, etc.) and transport functions for relaying traffic between communication network 100 and other networks.

[0028] 1B, an example of a wireless private network 150 deployed in accordance with various embodiments of the present disclosure for use on the campus of an organization (e.g., the campus of a business, school, or other organization) having multiple buildings 153. While FIG. 1B shows an example with multiple buildings, it will be understood that the disclosed techniques are equally applicable to any layout of a site that may include one or more buildings and / or one or more outdoor spaces (e.g., a stadium or other outdoor venue).

[0029] In this non-limiting example, wireless private network 150 includes four cells 156a, 156b, 156c, and 156d to provide complete coverage within an organization's campus. Cells 156 may overlap somewhat to provide comprehensive coverage within each building 153. Adjacent or overlapping cells 156 are configured to operate on non-interfering frequencies. For example, cell 156a may use frequency A, cell 156b may use frequency B, and cell 156c may use frequency C, all of which are distinct frequencies because the coverage of each of cells 156a, 156b, and 156c overlaps. However, cell 156d's coverage does not overlap with that of cells 156a or 156b, so cell 156d may use frequency A or frequency B, for example.

[0030] It should be noted that cells 156 may be added or removed from the wireless private network 150 depending on usage or other network metrics. In some cases, signal strength to a cell 156 may be increased to reduce the number of cells 156 or decreased to increase the number of cells 156 while allowing spectrum reuse among the cells 156. Additionally, as desired, computing capacity may be added within a geographic area of ​​an organization's premises or within a cloud provider network to reduce latency, maintain security, and increase reliability of the wireless private network 150. In some cases, the computing capacity of the software implementing the wireless private network 150 may be largely or entirely provisioned within the cloud provider network, not on the customer's premises, such as in a cloud service provider's regional data center. This software may implement various network functions, such as UPF, AMF, and SMF, which may correspond to core network functions, central unit network functions, and distributed unit network functions. Some network functions, such as distributed unit network functions, may remain at the cell site.

[0031] FIG. 2A illustrates an example network environment 200, according to some embodiments, that includes a cloud provider network 203 and various provider substrate extensions of the cloud provider network that may be used in combination with on-premises customer deployments within the communications network 100 of FIG. 1 . Cloud provider network 203 (sometimes simply referred to as the “cloud”) refers to a network-accessible pool of computing resources (e.g., compute, storage, and networking resources, applications, and services), which may be virtualized or bare metal. A cloud may provide convenient, on-demand network access to a shared pool of configurable computing resources that can be programmatically provisioned and released in response to customer commands. These resources may be dynamically provisioned and reconfigured to adjust to fluctuating loads. Thus, cloud computing can be viewed as both applications delivered as services over publicly accessible networks (e.g., the Internet, cellular communications networks) and the hardware and software in the cloud provider's data centers that provide those services.

[0032] The cloud provider network 203 can provide users with an on-demand, scalable computing platform over the network, allowing them to have scalable “virtual computing devices” at their disposal, for example, through the use of compute servers (which provide compute instances through the use of central processing units (CPUs) and / or graphics processing units (GPUs), optionally with local storage) and block store servers (which provide virtualized persistent block storage for specified compute instances). These virtual computing devices have the attributes of personal computing devices, including hardware (various types of processors, local memory, random access memory (RAM), hard disk, and / or solid-state drive (SSD) storage), a selected operating system, network capabilities, and preloaded application software. Each virtual computing device may also virtualize its console input / output (e.g., keyboard, display, and mouse). This virtualization allows users to configure and use their virtual computing devices as if they were personal computing devices by connecting to them using computer applications, such as a browser, API, or software development kit (SDK). Unlike a personal computing device, where a user has a fixed amount of hardware resources available, the hardware associated with a virtual computing device can be scaled up or down depending on the resources needed by the user.

[0033] As previously mentioned, users can connect to virtualized computing devices and other cloud provider network 203 resources and services via intermediate network(s) 212 using various interfaces 206 (e.g., APIs) to configure and manage telecommunications networks, such as 5G networks. An API refers to an interface and / or communication protocol between client devices 215 and servers such that when a client makes a request in a predefined format, the client should receive a response in a specific format or initiate a defined action. In the context of a cloud provider network, an API provides a gateway for customers to access the cloud infrastructure by enabling customers to retrieve data from or perform actions within the cloud provider network, enabling the development of applications that interact with resources and services hosted in the cloud provider network. APIs can also enable various services in the cloud provider network to exchange data with each other. Users can choose to deploy their own virtual computing systems to provide network-based services for their own use and / or for use by their customers or clients.

[0034] Cloud provider network 203 may include a physical network (e.g., sheet metal boxes, cables, rack hardware) called a substrate. The substrate may be considered a network fabric that includes the physical hardware that runs the provider network's services. The substrate may be isolated from the rest of cloud provider network 203; for example, it may not be possible to route from a substrate network address to an address in the production network that runs the cloud provider's services or to a customer network that hosts customer resources.

[0035] Cloud provider network 203 may also include an overlay network of virtualized computing resources running on a substrate. In at least some embodiments, a hypervisor or other device or process on the network substrate may use encapsulation protocol techniques to encapsulate and route network packets (e.g., client IP packets) over the network substrate between client resource instances on different hosts in the provider network. The encapsulation protocol techniques may be used on the network substrate to route the encapsulated packets (also called network substrate packets) between endpoints on the network substrate via overlay network paths or routes. The encapsulation protocol techniques may be viewed as providing a virtual network topology overlaid on the network substrate. As such, network packets may be routed along the substrate network according to constructs in the overlay network (e.g., virtual networks, which may be called virtual private clouds (VPCs) and port / protocol firewall configurations, which may be called security groups). A mapping service (not shown) may coordinate the routing of these network packets. The mapping service may be a regionally distributed lookup service that maps overlay Internet Protocol (IP) and network identifier combinations to substrate IPs, allowing distributed substrate computing devices to look up packet destinations.

[0036] By way of example, each physical host device (e.g., compute server, block store server, object store server, control server) may have an IP address within the substrate network. Hardware virtualization techniques may enable multiple operating systems to run simultaneously on a host computer, e.g., as virtual machines (VMs) on a compute server. A hypervisor, or virtual machine monitor (VMM), on the host allocates the host's hardware resources to the various VMs on the host and monitors the execution of the VMs. Each VM may be assigned one or more IP addresses within the overlay network, and the VMM on the host may be aware of the IP addresses of the VMs on the host. The VMM (and / or other devices or processes on the network substrate) may use encapsulation protocol techniques to encapsulate and route network packets (e.g., client IP packets) over the network substrate between virtualized resources on different hosts within the cloud provider network 203. Encapsulation protocol techniques may be used on the network substrate to route encapsulated packets between endpoints on the network substrate via overlay network paths or routes. The encapsulation protocol technology may be viewed as providing a virtual network topology overlaid on a network substrate. The encapsulation protocol technology may include a mapping service that maintains a mapping directory that maps IP overlay addresses (e.g., customer-visible IP addresses) to substrate IP addresses (customer-invisible IP addresses), which may be accessed by various processes on the cloud provider network 203 to route packets between endpoints.

[0037] As shown, traffic and operations of the cloud provider network substrate can, in various embodiments, be broadly subdivided into two categories: control plane traffic, carried on a logical control plane 218, and data plane operations, carried on a logical data plane 221. The data plane 221 represents the movement of user data through the distributed computing system, and the control plane 218 represents the movement of control signals through the distributed computing system. The control plane 218 generally includes one or more control plane components or services distributed across and implemented by one or more control servers. Control plane traffic generally includes management operations such as establishing isolated virtual networks for various customers, monitoring resource usage and health, identifying specific hosts or servers on which requested compute instances should be launched, and provisioning additional hardware as needed. The data plane 221 includes customer resources (e.g., compute instances, containers, block storage volumes, databases, file storage) implemented on the cloud provider network. Data plane traffic generally includes non-management operations such as transferring data to and from customer resources.

[0038] Control plane components are typically implemented on a separate set of servers from data plane servers, and control plane traffic and data plane traffic may be transmitted over separate and distinct networks. In some embodiments, control plane traffic and data plane traffic may be supported by different protocols. In some embodiments, messages (e.g., packets) transmitted over cloud provider network 203 include a flag indicating whether the traffic is control plane traffic or data plane traffic. In some embodiments, the payload of the traffic may be inspected to determine its type (e.g., control plane or data plane). Other approaches to distinguishing traffic types are possible.

[0039] As shown, data plane 221 may include one or more compute servers, which may be bare metal (e.g., single-tenant) or virtualized by a hypervisor to run multiple VMs (sometimes referred to as “instances”) or micro-VMs for one or more customers. These compute servers may support the cloud provider network's virtualized computing services (or “hardware virtualization services”). The virtualized computing services may be part of control plane 218 and allow customers to issue commands via interfaces 206 (e.g., APIs) to launch and manage compute instances (e.g., VMs, containers) for their applications. The virtualized computing services may provide virtual compute instances with various compute and / or memory resources. In one embodiment, each of the virtual compute instances may correspond to one of several instance types. An instance type may be characterized by its hardware type, computational resources (e.g., the number, type, and configuration of CPUs or CPU cores), memory resources (e.g., the capacity, type, and configuration of local memory), storage resources (e.g., the capacity, type, and configuration of locally accessible storage), network resources (e.g., the characteristics and / or network capabilities of its network interfaces), and / or other suitable descriptive characteristics. The instance type selection function may be used to select an instance type for a customer, e.g., based (at least in part) on input from the customer. For example, a customer may select an instance type from a set of predefined instance types. As another example, a customer may specify the desired resources of the instance type and / or the requirements of the workload the instance will run, and the instance type selection function may select an instance type based on such specifications.

[0040] The data plane 221 may also include one or more block store servers, which may include persistent storage for storing volumes of customer data and software for managing these volumes. Such block store servers may support the cloud provider network's managed block storage service. The managed block storage service is part of the control plane 218 and allows customers to issue commands via interfaces 206 (e.g., APIs) to create and manage volumes for applications running on compute instances. Block store servers include one or more servers where data is stored as blocks. A block is a sequence of bytes or bits, typically containing an integer number of records with a maximum length of the block size. Blocked data is typically stored in a data buffer, and an entire block is read or written at a time. In general, a volume can correspond to a logical collection of data, such as a set of data maintained on behalf of a user. A user volume, which can be treated as an individual hard drive, ranging in size from 1 GB to 1 terabyte (TB) or more, is composed of one or more blocks stored on a block store server. Although treated as individual hard drives, it will be understood that volumes may be stored as one or more virtualized devices implemented on one or more underlying physical host devices. A volume may be partitioned a small number of times (e.g., up to 16 times), with each partition hosted by a different host. A volume's data may be replicated across multiple devices within a cloud provider network to provide multiple replicas of the volume (such replicas may collectively represent a volume on a computing system).Replicas of volumes in a distributed computing system can beneficially provide automatic failover and recovery, for example, by allowing users to access either a primary replica of a volume or a secondary replica of a volume synchronized at the block level with the primary replica, so that failure of either the primary or secondary replica does not prevent access to the volume's information. The role of the primary replica may be to facilitate reads and writes (sometimes referred to as "input / output operations" or simply "I / O operations") on the volume and to reflect any writes to the secondary (preferably synchronously in the I / O path, although sometimes using asynchronous replication). Secondary replicas are updated synchronously with the primary replica and can provide a seamless transition during a failover operation, whereby the secondary replica takes over the role of the primary replica and the previous primary is designated as the secondary, or a new replacement secondary replica is provisioned. While specific examples herein describe a primary replica and a secondary replica, it will be understood that a logical volume can include multiple secondary replicas. A compute instance may virtualize its I / O to the volume via a client. A client corresponds to instructions that enable a compute instance to connect to and perform I / O operations on a remote data volume (e.g., a data volume stored on a physically separate computing device accessed over a network). A client may be implemented on an offload card of a server that includes a processing unit (e.g., a CPU or GPU) of the compute instance.

[0041] Data plane 221 may also include one or more object store servers, which represent other types of storage within the cloud provider network. Object storage servers include one or more servers where data is stored as objects within resources called buckets and may be used to support the cloud provider network's managed object storage services. Each object typically includes the stored data, a variable amount of metadata that enables various functions of the object storage server regarding analysis of the stored objects, and a globally unique identifier or key that can be used to retrieve the object. Each bucket is associated with a given user account. Customers can store as many objects as they want within their buckets, write, read, and delete objects within their buckets, and control access to their buckets and the objects contained therein. Furthermore, in embodiments where several different object storage servers are distributed across different ones of the aforementioned regions, users can choose the region (or regions) in which their buckets are stored, e.g., to optimize latency. Customers can use buckets to store various types of objects, such as machine images that can be used to start VMs or snapshots that represent point-in-time views of a volume's data.

[0042] Provider substrate extension 224 (“PSE”) provides resources and services of cloud provider network 203 within a separate network, such as a telecommunications network, thereby extending the capabilities of cloud provider network 203 to new locations (e.g., for reasons related to latency in communications with customer devices, regulatory compliance, security, etc.). In some embodiments, PSE 224 may be configured to provide capacity for cloud-based workloads executing within the telecommunications network. In some embodiments, PSE 224 may be configured to provide core and / or RAN functionality for the telecommunications network and may be configured with additional hardware (e.g., radio access hardware). Some embodiments may be configured to enable both, for example, by making unused capacity by the core and / or RAN functionality available for running cloud-based workloads.

[0043] As shown, such provider substrate extensions 224 may include cloud provider network-managed provider substrate extensions 227 (e.g., formed by servers located within cloud provider-managed facilities separate from those associated with the cloud provider network 203), communications service provider substrate extensions 230 (e.g., formed by servers associated with communications service provider facilities), and customer-managed provider substrate extensions 233 (e.g., formed by servers located on-premise at customer or partner facilities), among other possible types of substrate extensions.

[0044] As shown in exemplary provider substrate extension 224, provider substrate extension 224 may similarly include a logical separation between control plane 236 and data plane 239, which extend cloud provider network 203's control plane 218 and data plane 221, respectively. Provider substrate extension 224 may be pre-configured, for example, by a cloud provider network operator, with an appropriate combination of hardware with software and / or firmware elements to support various types of computing-related resources and to reflect the experience of using a cloud provider network to do so. For example, one or more provider substrate extension location servers may be provisioned by a cloud provider for deployment within provider substrate extension 224. As noted above, cloud provider network 203 may offer a set of pre-defined instance types, each with different types and amounts of underlying hardware resources. Each instance type may also be offered in different sizes. The servers may be heterogeneous servers to allow customers to continue using the same instance types and sizes in provider substrate extension 224 as they use in their region. A heterogeneous server can simultaneously support multiple instance sizes of the same type and can also be reconfigured to host any instance type supported by its underlying hardware resources. Reconfiguration of a heterogeneous server can be performed on the fly using the server's available capacity, i.e., while other VMs are still running and consuming other capacity of the provider substrate extension location server.This can improve utilization of computing resources in edge locations by enabling better packing of running instances on servers, and can also provide a seamless experience for using instances across cloud provider network 203 and cloud provider network managed provider substrate extension 227.

[0045] A provider substrate extension server may host one or more compute instances. A compute instance may be a VM or container that packages code and all its dependencies, allowing an application to run quickly and reliably across computing environments (e.g., including VMs and micro-VMs). Additionally, depending on customer preferences, the server may host one or more data volumes. In a region of the cloud provider network 203, such volumes may be hosted on dedicated block store servers. However, because the capacity of the provider substrate extension 224 may be significantly smaller than within a region, a suboptimal usage experience may be provided if the provider substrate extension 224 includes such dedicated block store servers. Therefore, block storage services may be virtualized within the provider substrate extension 224, such that one of the VMs runs block store software to store the volume's data. Similar to the operation of block storage services in a region of the cloud provider network 203, volumes within the provider substrate extension 224 may be replicated for durability and availability. Volumes may be provisioned within their own isolated virtual network within the provider substrate extension 224. The compute instances and any volumes collectively constitute a data plane 239 extension of the provider network data plane 221 within the provider substrate extension 224 .

[0046] Servers within the provider substrate extension 224 may, in some embodiments, host certain local control plane components, such as components that allow the provider substrate extension 224 to continue functioning if connectivity back to the cloud provider network 203 is lost. Examples of these components include a migration manager that can move compute instances between provider substrate extension servers as needed to maintain availability, and a key-value data store that indicates the locations of volume replicas. In general, however, the provider substrate extension's control plane 236 functions remain in the cloud provider network 203 to allow customers to use as much of the provider substrate extension's resource capacity as possible.

[0047] The migration manager may have a centralized coordination component running within the region as well as a local controller running on the PSE server (and servers in the cloud provider's data center). The centralized coordination component can identify the target edge location and / or target host when a migration is triggered, and the local controller can coordinate the data transfer between the source and target hosts. The described movement of resources between hosts in different locations can take one of several migration forms. Migration refers to moving virtual machine instances (and / or other resources) between hosts within a cloud computing network or between hosts outside the cloud computing network and hosts within the cloud. There are various types of migrations, such as live migration and reboot migration. During a reboot migration, a customer experiences a shutdown and effective power cycle of the virtual machine instance. For example, a control plane service may coordinate a reboot migration workflow that involves destroying the current domain on the original host and then creating a new domain for the virtual machine instance on a new host. The instance is restarted by shutting it down on the original host and starting it up again on the new host.

[0048] Live migration refers to the process of moving a running virtual machine or application between different physical machines without significantly disrupting the availability of the virtual machine (e.g., virtual machine downtime is not noticeable to end users). When the control plane executes a live migration workflow, it can create a new "inactive" domain associated with the instance, while the instance's original domain continues to run as the "active" domain. The virtual machine's memory (including any in-memory state of running applications), storage, and network connectivity are transferred from the original host with the active domain to the destination host with the inactive domain. The virtual machine may be briefly paused to prevent state changes while its memory contents are transferred to the destination host. The control plane may transition the inactive domain to become the active domain, demote the original active domain to become the inactive domain (also known as a "flip"), and then destroy the inactive domain.

[0049] Various types of migration techniques involve managing critical phases (times when a virtual machine instance is unavailable to customers), which should be as short as possible. This management can be particularly challenging in currently disclosed migration techniques because resources are moved between hosts in geographically distant locations that may be connected through one or more intermediate networks. For live migrations, the disclosed techniques can dynamically determine how much memory state data to copy in advance (e.g., while the instance is still running on the source host) and how much memory state data to copy afterward (e.g., after the instance has started running on the destination host) based on, for example, the latency between the locations, network bandwidth / usage patterns, and / or which memory pages are most frequently used by the instance. Furthermore, the specific time at which the memory state data is transferred can be dynamically determined based on the network conditions between the locations. This analysis may be performed by a migration management component within the region or by a migration management component running locally at the source edge location. If the instance has access to virtualized storage, both the source and target domains may be simultaneously attached to the storage to enable uninterrupted access to that data during migration or if a rollback to the source domain is required.

[0050] Server software running on the provider substrate extension 224 may be designed by the cloud provider to run on the cloud provider substrate network, and this software may be enabled to run unmodified within the provider substrate extension 224 by creating a private replica of the substrate network (a "shadow substrate") within an edge location using local network manager(s) 242. The local network manager(s) 242 operate on the provider substrate extension 224 servers and can bridge the shadow substrate with the provider substrate extension 224 network, for example, by acting as one or more virtual private network (VPN) endpoints between the provider substrate extension 224 and proxies 245, 248 within the cloud provider network 203, and by implementing a mapping service (for traffic encapsulation and de-encapsulation) that associates data plane traffic (from the data plane proxy 248) and control plane traffic (from the control plane proxy 245) with the appropriate server(s). By implementing a local version of the provider network's substrate overlay mapping service, local network manager(s) 242 enable resources in provider substrate extension 224 to seamlessly communicate with resources in cloud provider network 203. In some embodiments, a single local network manager 242 may perform these actions for all servers hosting compute instances in provider substrate extension 224. In other embodiments, each server hosting a compute instance may have its own dedicated local network manager 242.In multi-rack edge locations, local network managers maintain open tunnels with each other so that inter-rack communications can be routed through the local network manager 242 .

[0051] The provider substrate extension locations may utilize secure network tunnels through the provider substrate extension 224 network to the cloud provider network 203, for example, to maintain the security of customer data as it traverses the provider substrate extension 224 network and other intermediate networks (potentially including the public Internet). Within the cloud provider network 203, these tunnels are comprised of virtual infrastructure components, including isolated virtual networks (e.g., within an overlay network), a control plane proxy 245, a data plane proxy 248, and substrate network interfaces. Such proxies 245, 248 may be implemented as containers running on the compute instances. In some embodiments, each server at the provider substrate extension 224 location that hosts a compute instance may utilize at least two tunnels: one for control plane traffic (e.g., Constrained Application Protocol (CoAP) traffic) and one for encapsulated data plane traffic. A connection manager (not shown) within the cloud provider network 203 manages the lifecycle of the cloud provider network side of these tunnels and their components, for example, by automatically provisioning them as needed and maintaining them in a healthy operational state. In some embodiments, a direct connection between the provider substrate extension 224 location and the cloud provider network 203 may be used for control plane and data plane communications. Compared to VPNs over other networks, a direct connection may provide constant bandwidth and more consistent network performance because its network path is relatively fixed and stable.

[0052] A control plane (CP) proxy 245 may be provisioned within the cloud provider network 203 to represent a specific host(s) at an edge location. The CP proxy 245 is an intermediary between the control plane 218 within the cloud provider network 203 and control plane targets within the control plane 236 of the provider substrate extension 224. That is, the CP proxy 245 provides the infrastructure for tunneling management API traffic destined for a provider substrate extension server from the regional substrate to the provider substrate extension 224. For example, a virtualized computing service in the cloud provider network 203 may issue a command to the VMM of a server in the provider substrate extension 224 to launch a compute instance. The CP proxy 245 maintains a tunnel (e.g., a VPN) to the provider substrate extension's local network manager 242. Software implemented within the CP proxy 245 ensures that only eligible API traffic leaves the substrate and returns to the substrate. The CP proxy 245 provides a mechanism to expose remote servers on the cloud provider substrate while still protecting the substrate's security material (e.g., encryption keys, security tokens) from leaving the cloud provider network 203. The unidirectional control plane traffic tunnel imposed by the CP proxy 245 also prevents arbitrary (potentially compromised) devices from calling back into the substrate. The CP proxy 245 may be instantiated one-to-one with a server in the provider substrate extension 224, or may be able to manage control plane traffic for multiple servers within the same provider substrate extension.

[0053] Data plane (DP) proxies 248 may also be provisioned within the cloud provider network 203 to represent specific server(s) within the provider substrate extension 224. The DP proxies 248 act as shadows or anchors for the server(s) and may be used by services within the cloud provider network 203 to monitor the host's health (including availability, used / free compute and capacity, used / free storage and capacity, and network bandwidth usage / availability). The DP proxies 248 also act as proxies for the server(s) within the cloud provider network 203, thereby enabling isolated virtual networks to span the provider substrate extension 224 and the cloud provider network 203. Each DP proxy 248 may be implemented as a packet forwarding compute instance or container. As shown, each DP proxy 248 may maintain a VPN tunnel with a local network manager 242, which manages traffic to the server(s) it represents. This tunnel may be used to transmit data plane traffic between the provider substrate extension server(s) and the cloud provider network 203. Data plane traffic flowing between the provider substrate extension 224 and the cloud provider network 203 may pass through the DP proxy 248 associated with that provider substrate extension 224. For data plane traffic flowing from the provider substrate extension 224 to the cloud provider network 203, the DP proxy 248 may receive the encapsulated data plane traffic, verify its accuracy, and allow it to enter the cloud provider network 203. The DP proxy 248 may forward the encapsulated traffic from the cloud provider network 203 directly to the provider substrate extension 224.

[0054] The local network manager(s) 242 can provide secure network connectivity with proxies 245, 248 established within the cloud provider network 203. After connectivity between the local network manager 242 and the proxies 245, 248 is established, the customer may issue commands via interface 206 to instantiate compute instances (and / or perform other operations using the compute instances) using the provider substrate extension resources in the same manner as commands would be issued for compute instances hosted within the cloud provider network 203. From the customer's perspective, the customer can now seamlessly use local resources within the provider substrate extension (and resources in the cloud provider network 203, if desired). Compute instances set up on servers at the provider substrate extension 224 may communicate with both electronic devices in the same network and other resources set up within the cloud provider network 203, as desired. A local gateway 251 may be implemented to provide network connectivity between the provider substrate extension 224 and the network coupled to that extension (e.g., the communications service provider network in the example of communications service provider substrate extension 230).

[0055] In some situations, data transfer between an object storage service and a provider substrate extension (PSE) 224 may be required. For example, an object storage service may store machine images used to boot VMs and snapshots representing point-in-time backups of volumes. An object gateway may provide a configurable per-bucket cache of object storage bucket contents in the PSE 224 to customers on a PSE server or specialized storage device to minimize the impact of PSE regional latency on customer workloads. The object gateway may also temporarily store snapshot data from volume snapshots in the PSE 224 and synchronize it with an object server in the region when possible. The object gateway may also store machine images designated by the customer for use in the PSE 224 or on the customer's premises. In some implementations, data in the PSE 224 may be encrypted with a unique key, and the cloud provider may restrict the key from being shared from the region to the PSE 224 for security reasons. Thus, data exchanged between the object store server and the object gateway may utilize encryption, decryption, and / or re-encryption to maintain security boundaries regarding encryption keys or other sensitive data. A means of transformation can perform these operations, and a PSE bucket can be created (on the object store server) to store the snapshot data and machine image data using the PSE encryption key.

[0056] As described above, PSE 224 forms an edge location in that it provides cloud provider network 203 resources and services closer to customer devices outside of traditional cloud provider datacenters. The edge locations referred to herein can be structured in several ways. In some implementations, an edge location can be an extension of the cloud provider network substrate, including a limited amount of capacity provided outside of an availability zone (e.g., in a smaller datacenter located closer to customer workloads and potentially farther from any availability zones, or in other facilities of the cloud provider). Such edge locations are sometimes referred to as “far zones” (because they are far from other availability zones) or “near zones” (because they are closer to customer workloads). Near zones can be connected to publicly accessible networks such as the Internet in various ways, for example, directly, through another network, or through a private connection to a region. Typically, near zones have limited capacity compared to regions, but in some cases, near zones can have significant capacity, such as thousands of racks or more.

[0057] In some implementations, an edge location may be an extension of a cloud provider network substrate formed by one or more servers located on-premises at a customer or partner facility, with such server(s) communicating over a network (e.g., a publicly accessible network such as the Internet) with a nearby availability zone or region of the cloud provider network. This type of substrate extension located outside a cloud provider network datacenter may be referred to as an "outpost" of the cloud provider network. Some outposts may be integrated into a communications network, for example, as a telecommunications datacenter, a telecommunications aggregation site, and / or a multi-access edge computing (MEC) site with physical infrastructure spanning telecommunications base stations within the telecommunications network. In an on-premises example, the outpost's limited capacity may be available only to the customer that owns the facility (and any other accounts authorized by the customer). In a telecommunications example, the outpost's limited capacity may be shared among multiple applications (e.g., games, virtual reality applications, healthcare applications) that transmit data to users of the telecommunications network.

[0058] An edge location can include data plane capacity that is controlled at least in part by the control plane of a nearby availability zone in the provider network. Thus, an availability zone group may include a “parent” availability zone and any “child” edge locations that are home to the parent availability zone (e.g., are controlled at least in part by its control plane). Certain limited control plane functions (e.g., functions requiring low-latency communication with customer resources and / or functions that allow the edge location to continue functioning when disconnected from the parent availability zone) may also be present in some edge locations. Thus, in the above example, an edge location refers to at least an extension of data plane capacity located at the edge of the cloud provider network, close to customer devices and / or workloads.

[0059] In the example of FIG. 1A , the distributed computing device 112 ( FIG. 1A ), the centralized computing device 115 ( FIG. 1A ), and the core computing device 118 ( FIG. 1A ) may be implemented as provider substrate extensions 224 of the cloud provider network 203. The placement or location of the provider substrate extensions 224 within the communications network 100 may vary depending on the particular network topology or architecture of the communications network 100. The provider substrate extensions 224 may generally connect anywhere the communications network 100 can generate packet-based traffic (e.g., IP-based traffic). Furthermore, communications between a given provider substrate extension 224 and the cloud provider network 203 typically traverse at least a portion of the communications network 100 securely (e.g., via a secure tunnel, a virtual private network, a direct connection, etc.).

[0060] In 5G wireless network development efforts, edge locations may be considered as a possible implementation of multi-access edge computing (MEC). Such edge locations may be connected to various points within the 5G network that provide data traffic breakout as part of the user plane function (UPF). Even older wireless networks may incorporate edge locations. For example, in a 3G wireless network, an edge location may be connected to a packet-switched network portion of the communications network 100, such as a serving general packet radio service support node (SGSN) or gateway general packet radio service support node (GGSN). In a 4G wireless network, an edge location may be connected to a serving gateway (SGW) or packet data network gateway (PGW) as part of the core network or evolved packet core (EPC). In some embodiments, traffic between the provider substrate extension 224 and the cloud provider network 203 may be decoupled from the communications network 100 without being routed through the core network.

[0061] In some embodiments, provider substrate extension 224 may connect to multiple communication networks associated with each customer. For example, if two communication networks of each customer share or route traffic through a common point, provider substrate extension 224 may be connected to both networks. For example, each customer may assign a portion of its network address space to the provider substrate extension, and the provider substrate extension may include a router or gateway capable of distinguishing traffic exchanged with each of communication networks 100. For example, traffic destined for provider substrate extension 224 from one network may have a different destination IP address, source IP address, and / or virtual local area network (VLAN) tag than traffic received from another network. Traffic from the provider substrate extension destined for a destination in one of the networks may likewise be encapsulated to have the appropriate VLAN tag, source IP address (e.g., from a pool allocated to the provider substrate extension from the destination network address space), and destination IP address.

[0062] FIG. 2B illustrates an example cellularization and geographic distribution 253 of communication network 100 (FIG. 1) for providing a highly available user plane function (UPF). In FIG. 2B, a user device 254 communicates with a request router 255 to route requests to one of multiple control plane cells 257a and 257b. Each control plane cell 257 may include a network service API gateway 260, a network slice configuration 262, network service monitoring functions 264, site planning data 266 (including layouts describing customer site requirements, device types, device quantities, etc.), a network service / function catalog 268, orchestration functions 270, and / or other components. To reduce the likelihood that a large-scale error will affect a wide range of customers, a large control plane may be divided into cells, with one or more cells operating independently, for example, per customer, per network, or per region.

[0063] The network service / function catalog 268 is also referred to as the NF repository function (NRF). In a service-based architecture (SBA) 5G network, control plane functions and a common data repository may be delivered through a set of interconnected network functions built using a microservices architecture. The NRF may maintain a record of available NF instances and their supported services, allowing other NF instances to subscribe to and be notified of registrations from NF instances of a given type. Thus, the NRF may support service discovery by receiving discovery requests from NF instances and specify which NF instances support a particular service. The network function orchestrator 270 may perform NF lifecycle management, including instantiation, scale-out / in, performance measurement, event correlation, and termination. The network function orchestrator 270 may onboard new NFs, manage the migration of existing NFs to new or updated versions, identify a suitable set of NFs for a particular network slice or larger network, and orchestrate NFs across the various computing devices and sites that comprise the wireless private network 103.

[0064] The control plane cell 257 may communicate with one or more cell sites 272, one or more customer local data centers 274, one or more local zones 276, and one or more regional zones 278. The cell site 272 includes computing hardware 280 that runs one or more distributed unit (DU) network functions 282. The customer local data center 274 includes computing hardware 283 that runs one or more DU or central unit (CU) network functions 284, a network controller, a UPF 286, one or more edge applications 287 that support customer workloads, and / or other components.

[0065] The local zone 276 may reside in a data center operated by a cloud service provider and may run one or more core network functions 288, such as the AMF, SMF, a network publishing function (NEF) that securely exposes the services and capabilities of other network functions, and a unified data management (UDM) function that manages subscriber data for authorization, registration, and mobility management. The local zone 276 may also run the UPF 286, a service for metric processing 289, and one or more edge applications 287.

[0066] The regional zone 278 may be located within a data center operated by a cloud service provider and may run one or more core network functions 288; a UPF 286; an operations support system (OSS) 290 that supports network management systems, service delivery, service fulfillment, service assurance, and customer care; an Internet Protocol Multimedia Subsystem (IMS) 291; a business support system (BSS) 292 that supports product management, customer management, revenue management, and / or order management; one or more portal applications 293, and / or other components.

[0067] In this example, communication network 100 employs a cellular architecture to reduce the blast radius of individual components. At the highest level, the control plane resides within multiple control plane cells 257 to prevent failure of an individual control plane from affecting the entire deployment.

[0068] Within each control plane cell 257, multiple redundant stacks may be provided, with the control plane shifting traffic to a secondary stack as needed. For example, cell site 272 may be configured to utilize a nearby local zone 276 as its default core network. If the local zone 276 experiences an outage, the control plane can redirect cell site 272 to use a backup stack in the regional zone 278. Traffic normally routed from the Internet to the local zone 276 may be shifted to an endpoint in the regional zone 278. Each control plane cell 278 may implement a "stateless" architecture that shares a common session database across multiple sites (e.g., across availability zones or edge sites).

[0069] FIG. 3 illustrates an exemplary cloud provider network 203 including geographically distributed provider substrate extensions 224 (FIG. 2A) (or “edge locations 303”), according to some embodiments. As shown, the cloud provider network 203 may be formed as multiple regions 306, where a region is a distinct geographic area in which the cloud provider has one or more data centers 309. Each region 306 may include two or more availability zones (AZs) connected to each other via a private high-speed network, such as a fiber optic connection. An availability zone refers to an isolated failure domain that includes one or more data center facilities with separate power sources, separate networks, and separate cooling from other availability zones. Cloud providers may strive to locate availability zones sufficiently far from each other within a region to prevent a natural disaster, major power outage, or other unforeseen event from taking multiple availability zones offline at the same time. Customers can connect to resources within an availability zone of the cloud provider network via a publicly accessible network (e.g., the Internet, a cellular network, a communications service provider network). Transit centers (TCs) are primary backbone locations that link customers to the cloud provider network and may be co-located with other network provider facilities (e.g., internet service providers, telecommunications providers). Each region can operate two or more TCs for redundancy. The regions 306 are connected to a global network that includes a private network infrastructure (e.g., fiber connections controlled by the cloud service provider) that connects each region 306 to at least one other region. The cloud provider network 203 can deliver content from points of presence ("PoPs") that are external to, but networked with, these regions 306 via edge locations 303 and regional edge cache servers.This partitioning and geographic distribution of computing hardware enables cloud provider network 203 to offer customers global, low-latency resource access with a high degree of fault tolerance and stability.

[0070] The number of edge locations 303 can be much greater than the number of regional data centers or availability zones. This widespread deployment of edge locations 303 can provide low-latency connectivity to the cloud for a much larger group of end-user devices (compared to end-user devices that happen to be very close to a regional data center). In some embodiments, each edge location 303 may peer with a portion of the cloud provider network 203 (e.g., a parent availability zone or regional data center). Such peering allows various components operating within the cloud provider network 203 to manage the computing resources of the edge locations 303. In some cases, multiple edge locations 303 may be located or housed in the same facility (e.g., separate racks of computer systems) and managed by different zones or data centers to provide additional redundancy. It should be noted that although the edge location 303 is generally shown herein as being within the communications service provider network or wireless private network 103 (FIG. 1A), in some cases, such as when the cloud provider network facilities are relatively close to the communications service provider facilities, the edge location 303 may remain within the physical premises of the cloud provider network 203 while connected to the communications service provider network via fiber or other network links.

[0071] Edge locations 303 may be structured in several ways. In some implementations, edge locations 303 may be an extension of the cloud provider network substrate, including a limited amount of capacity provided outside of availability zones (e.g., in a small data center located close to customer workloads and potentially far from any availability zones, or in other cloud provider facilities). Such edge locations 303 may be referred to as local zones (because they are closer to a local area or group of users than traditional availability zones). Local zones may be connected to publicly accessible networks such as the Internet in various ways, for example, directly, through another network, or through a private connection to a region 306. Typically, local zones have limited capacity compared to a region 306, but in some cases, local zones may have significant capacity, such as thousands of racks or more. Some local zones may use infrastructure similar to a typical cloud provider data center instead of the edge location 303 infrastructure described herein.

[0072] As shown herein, the cloud provider network 203 can be formed as several regions 306, each representing a geographic area where the cloud provider clusters its data centers. Each region may further include multiple (e.g., two or more) availability zones (AZs) connected to each other via a private high-speed network, e.g., fiber communications connections. An AZ may provide an isolated failure domain, including one or more data center facilities with separate power sources, separate networks, and separate cooling from another AZ. The AZs within a region 306 are preferably located in locations far enough apart from each other that the same natural disaster (or other failure-causing event) does not affect or take multiple AZs offline at the same time. Customers can connect to the AZs of the cloud provider network via a publicly accessible network (e.g., the Internet, a cellular communications network, etc.).

[0073] The parenting of a given edge location 303 to an AZ or region 306 of the cloud provider network 203 may be based on several factors. One such parenting factor is data sovereignty. For example, an edge location 303 deployed within a country's communications network may be parented to an AZ or region 306 within that country to keep data originating from that country's communications network within that country. Another factor is service availability. For example, some edge locations 303 may have different hardware configurations, such as the presence or absence of components such as local non-volatile storage for customer data (e.g., solid-state drives), graphics accelerators, etc. Some AZs or regions 306 may lack services to utilize those additional resources, so an edge location may be parented to an AZ or region 306 that supports the use of those resources. Another factor is the latency between the AZ or region 306 and the edge location 303. While deploying edge locations 303 within a communications network provides benefits in terms of latency, those benefits can be negated by parenting the edge location 303 to a distant AZ or region 306, which introduces significant latency to the edge location 303's regional traffic. Therefore, edge locations 303 are often parented to nearby (in terms of network latency) AZs or regions 306.

[0074] 4, a network environment 400 according to various embodiments is shown. The network environment 400 includes a computing environment 403, one or more client devices 406, one or more pre-deployed devices 409, a spectrum reservation service 410, and one or more wireless private networks 103 in data communication with each other via a network 412. The network 412 may include, for example, the Internet, an intranet, an extranet, a wide area network (WAN), a local area network (LAN), a wired network, a wireless network, a cable network, a satellite network, or other suitable network, or any combination of two or more such networks.

[0075] Computing environment 403 may include, for example, a server computer or any other system that provides computing capacity. Alternatively, computing environment 403 may employ multiple computing devices that may be arranged, for example, in one or more server or computer banks or other arrangements. Such computing devices may be located in a single facility or distributed among many different geographic locations. For example, computing environment 403 may include multiple computing devices that together comprise host computing resources, grid computing resources, and / or any other distributed computing arrangement. In some cases, computing environment 403 may correspond to elastic computational resources in which allocated capacity of processing, network, storage, or other computing-related resources may change over time. For example, computing environment 403 may correspond to cloud provider network 203 (FIG. 2A), where customers are billed according to their use of computing resources based on a utility computing model.

[0076] In some embodiments, computing environment 403 may correspond to a virtualized private network within a physical network that includes virtual machine instances running on physical computing hardware, for example, by a hypervisor. The virtual machine instances and containers running on those instances may be provided network connectivity through virtualized network components made available by physical network components such as routers and switches.

[0077] Various applications and / or other functions may be executed in computing environment 403 according to various embodiments. Also, various data may be stored in data store 415, which is accessible to computing environment 403. Data store 415 may be representative of multiple data stores 415, as can be appreciated. Data stored in data store 415 may be associated with the operation of various applications and / or functional entities, for example, as described below.

[0078] The computing environment 403, as part of a cloud provider network offering utility computing services, includes computing devices 418 and other types of computing devices. The computing devices 418 may correspond to different types of computing devices 418 and may have different computing architectures. The computing architectures may differ by utilizing processors with different architectures, such as x86, x86_64, ARM, Scalable Processor Architecture (SPARC), PowerPC, etc. For example, one computing device 418 may have an x86 processor, while another computing device 418 may have an ARM processor. The computing devices 418 may also differ in available hardware resources, such as local storage, graphics processing units (GPUs), machine learning extensions, and other characteristics.

[0079] Computing devices 418 may have various forms of allocated computing capacity 421, which may include virtual machine (VM) instances, containers, serverless functions, etc. VM instances may be instantiated from VM images. To this end, a customer may specify that a virtual machine instance should be launched within a particular type of computing device 418 and not other types of computing devices 418. In various examples, one VM instance may run alone on a particular computing device 418, or multiple VM instances may run on a particular computing device 418. Also, a particular computing device 418 may run different types of VM instances, which may provide different amounts of resources available via the computing device 418. For example, one type of VM instance may provide more memory and processing power than another type of VM instance.

[0080] Components executing on the computing environment 403 include, for example, a wireless based private network (RBPN) management service 424, an RBPN hardware configuration service 427, an RBPN hardware deployment service 430, a capacity management service 433, and other applications, services, processes, systems, engines, or functions not described in detail herein.

[0081] The RBPN management service 424 executes to manage, configure, and monitor the wireless private network 103 operated by the cloud service provider on behalf of the customer. To this end, the RBPN management service 424 can generate a number of user interfaces that enable the customer to order a new wireless private network 103, scale up or down an existing wireless private network 103, modify the operation of an existing wireless private network 103, configure wireless devices 106 (FIG. 1A) that are authorized to use the wireless private network 103, provide statistics and metrics regarding the operation of the wireless private network 103, reserve frequency spectrum for the customer's private network via the spectrum reservation service 410, etc. For example, the RBPN management service 424 can generate one or more network pages, such as web pages, that include the user interfaces. The RBPN management service 424 can also support this functionality via APIs that can be called by client applications 436. In addition to facilitating user interaction, the RBPN management service 424 also implements the orchestration of deployment and configuration changes and the continuous monitoring of performance parameters of the wireless private network 103. In some cases, the RBPN management service 424 may generate a network plan 439 for a customer based at least in part on the customer's location specifications, automated site surveys by unmanned aerial vehicles, and / or other input parameters.

[0082] The RBPN hardware configuration service 427 executes to implement configuration changes on the hardware implementing the wireless private network 103. This may include radio units, antennas, VM instances or containers performing network functions, routers, switches, fiber termination devices, etc. For example, an antenna may be configured to operate at a particular frequency. A radio unit may be programmed to operate at a particular frequency, join a particular wireless private network 103, and backhaul traffic to a particular VM instance or container.

[0083] In some scenarios, the RBPN hardware configuration service 427 is executed to reconfigure hardware that is already present in the wireless private network 103. In other scenarios, the RBPN hardware configuration service 427 is executed to pre-configure a set of hardware to be deployed in an existing or new wireless private network 103. To this end, the RBPN hardware configuration service 427 can implement configuration on one or more pre-deployed devices 409 that are temporarily connected to the network 412 to facilitate pre-configuration before the pre-deployed devices 409 are shipped to a customer for deployment in the wireless private network 103.

[0084] The RBPN hardware deployment service 430 executes to automate and arrange for the deployment of hardware to implement the wireless private network 103. Based on a network plan 439 submitted by or generated for a customer, the RBPN hardware deployment service 430 can arrange for the procurement of hardware components necessary to implement the wireless private network 103 in accordance with the network plan 439. This can include automatically ordering new equipment from a vendor, reserving equipment already in the provider's inventory, or reallocating existing equipment for reallocation if it is no longer in use at the customer's site or another customer's site. In this regard, the RBPN hardware deployment service 430 can send instructions to the customer to return equipment that is no longer in use, and the equipment can be sent directly to another customer for use in another deployment. In another scenario, the RBPN hardware deployment service 430 can send instructions to the customer to move equipment from one site where the equipment is no longer in use to another site where the equipment will be used. RBPN hardware deployment service 430 may manage the connection of equipment to network 412 for pre-configuration as pre-deployed devices 409. RBPN hardware deployment service 430 may also arrange for the shipment of equipment to customer locations, potentially including multiple customer locations corresponding to respective cell sites.

[0085] Capacity management services 433 are implemented to manage computing capacity within the wireless private network 103 and associated core networks. This may include transferring computing capacity from network function workloads to customer workloads and vice versa. Additionally, unused computing capacity may be transferred from one customer to another. Network function workloads may also be transferred between distributed computing devices 112 at cell 109 (FIG. 1A) sites, centralized computing devices 115 at customer sites (FIG. 1A), and core computing devices 118 at data centers (FIG. 1A).

[0086] Data stored in data store 415 may include, for example, one or more network plans 439, one or more cellular topologies 442, one or more spectrum allocations 445, device data 448, one or more RBPN metrics 451, customer billing data 454, radio unit configuration data 457, antenna configuration data 460, network function configuration data 463, one or more network function workloads 466, one or more customer workloads 469, and potentially other data.

[0087] The network plan 439 is a specification of the wireless private network 103 to be deployed for a customer. For example, the network plan 439 may include the location or geographic area of ​​the premises to be covered, the number of cells, device identification and permissions, desired maximum network latency, desired bandwidth or network throughput for one or more classes of devices, one or more quality of service parameters for applications or services, and / or other parameters that can be used to create the wireless private network 103. The customer can manually specify one or more of these parameters via a user interface. One or more of the parameters may be pre-set as default parameters. In some cases, the network plan 439 may be generated for a customer based at least in part on an automated site survey using an unmanned aerial vehicle. The values ​​of the parameters that define the network plan 439 may be used as a basis for the cloud service provider to bill the customer under a utility computing model. For example, a service level agreement (SLA) may charge a customer a higher amount for lower latency targets and / or higher bandwidth targets, and the customer may be charged per device, per cell, etc., based on the geographic area served, spectrum availability, etc. In some cases, the network plan 439 may incorporate thresholds and reference parameters determined based at least in part on automated probing of the customer's existing private network.

[0088] The cellular topology 442 includes an arrangement of multiple cells for a customer, taking into account cell locations and, where possible, frequency spectrum reuse. The cellular topology 442 can be generated automatically by conducting a site survey. In some cases, the number of cells in the cellular topology 442 can be automatically determined based on the desired geographic area to be covered, the availability of backhaul connections at various sites, signal propagation, available frequency spectrum, and / or other parameters. In the case of a wireless private network 103, the cellular topology 442 can be developed to cover one or more buildings on an organizational campus, one or more schools in a school district, one or more buildings in a university or college system, and other areas.

[0089] Spectrum allocation 445 includes frequency spectrum available for allocation to wireless private networks 103 and frequency spectrum currently allocated to wireless private networks 103. Frequency spectrum may include spectrum that is publicly accessible without restrictions, spectrum that is privately owned or leased by customers, spectrum that is owned or leased by providers, spectrum that is free to use but requires reservations, etc.

[0090] The device data 448 corresponds to data describing the device 106 that is authorized to connect to the wireless private network 103. This device data 448 may include the corresponding user, account information, billing information, data plan, authorized applications or uses, an indication of whether the wireless device 106 is mobile or fixed, location, current cell, network address, device identifier (e.g., International Mobile Equipment Identity (IMEI) number, Equipment Serial Number (ESN), Media Access Control (MAC) address, Subscriber Identity Module (SIM) number, etc.).

[0091] The RBPN metrics 451 include various metrics or statistics indicative of the performance or health of the wireless private network 103. Such RBPN metrics 451 may include bandwidth metrics, dropped packet metrics, signal strength metrics, delay metrics, etc. The RBPN metrics 451 may be aggregated per device, per cell, per customer, etc.

[0092] The customer billing data 454 specifies the fees the customer will incur for the provider's operation of the wireless private network 103 for the customer. The fees may include a fixed fee based on the equipment deployed to the customer and / or a usage-based fee determined by tracked usage metrics. In some cases, the customer may have purchased the equipment upfront and may be charged only for bandwidth or back-end network costs. In other cases, the customer may not incur any upfront costs and may be charged purely on a usage basis. Because the equipment is provided to the customer based on a utility computing model, the cloud service provider can select the optimal configuration of equipment to meet the customer's target performance metrics while avoiding unnecessary hardware over-provisioning.

[0093] The radio unit configuration data 457 may correspond to configuration settings for radio units deployed in the wireless private network 103. Such settings may include the frequency to use, the protocol to use, modulation parameters, bandwidth, network routing and / or backhaul configuration, etc.

[0094] The antenna configuration data 460 may correspond to the configuration settings of the antenna, including the frequency to use, the azimuth angle, the vertical or horizontal orientation, the beam tilt, and / or other parameters that may be controlled automatically (e.g., by networked motors and controls on the antenna) or manually by instructing a user to mount the antenna in a particular manner or make physical changes to the antenna.

[0095] Network function configuration data 463 corresponds to configuration settings that configure the operation of various network functions of wireless private network 103. In various embodiments, network functions may be deployed on VM instances or containers located on computing devices 418 at cell sites, customer aggregation sites, or data centers located remotely from the customers. Non-limiting examples of network functions may include access and mobility management functions, session management functions, user plane functions, policy control functions, authentication server functions, integrated data management functions, application functions, network publishing functions, network function repositories, network slice selection functions, and / or others. Network function workloads 466 correspond to machine images, containers, or functions that will be launched within allocated computing capacity 421 to perform one or more network functions.

[0096] Customer workloads 469 correspond to customer machine images, containers, or functions that may be executed in parallel with or instead of network function workloads 466 within allocated computing capacity 421. For example, customer workloads 469 may provide or support customer applications or services. In various examples, customer workloads 469 are related to factory automation, autonomous robotics, augmented reality, virtual reality, engineering, monitoring, etc.

[0097] Client device 406 represents multiple client devices 406 that may be coupled to network 412. Client device 406 may include, for example, a processor-based system such as a computer system. Such a computer system may be embodied in the form of a desktop computer, a laptop computer, a personal digital assistant, a mobile phone, a smartphone, a set-top box, a music player, a web pad, a tablet computer system, a game console, an e-reader, a smart watch, a head-mounted display, a voice interface device, or other device. Client device 406 may include, for example, a display comprising one or more devices such as a liquid crystal display (LCD), a gas plasma-based flat panel display, an organic light-emitting diode (OLED) display, an electronic ink (E-ink) display, an LCD projector, or other type of display device.

[0098] The client device 406 may be configured to execute various applications, such as a client application 436 and / or other applications. The client application 436 may execute within the client device 406 and render a user interface on a display, for example, by accessing network content provided by the computing environment 403 and / or other servers. To this end, the client application 436 may comprise, for example, a browser, a dedicated application, etc., and the user interface may comprise a network page, an application screen, etc. The client device 406 may be configured to execute applications in addition to the client application 436, such as, for example, an email application, a social networking application, a word processor, a spreadsheet, and / or other applications.

[0099] In some embodiments, the spectrum reservation service 410 provides frequency spectrum reservations for customer private networks. In one scenario, the spectrum reservation service 410 is operated by an entity, such as a third party, to manage reservations and coexistence in publicly accessible spectrum. An example of such spectrum is the Citizens Broadband Radio Service (CBRS). In another scenario, the spectrum reservation service 410 is operated by a telecommunications service provider to sell or sublicense portions of the provider's owned or licensed spectrum.

[0100] 5, shown is a flowchart presenting an example of the operation of a portion of RBPN management service 424 according to various embodiments. It is understood that the flowchart of FIG. 5 merely presents an example of many different types of functional arrangements that may be utilized to implement the operation of a portion of RBPN management service 424 described herein. Alternatively, the flowchart of FIG. 5 may be considered to illustrate example elements of a method implemented within computing environment 403 (FIG. 4) according to one or more embodiments.

[0101] Starting in box 503, the RBPN management service 424 generates a user interface for ordering or provisioning the RBPN 103 (FIG. 1A). For example, the user interface may include components for specifying a network plan 439 (FIG. 4) or parameters of the network plan 439. Such parameters may include, for example, the number of cells, a map or site plan of the customer premises or geographic area to be covered, a target bandwidth, information about the wireless device 106 (FIG. 1A) or the user, a target minimum delay, a desired cost, and / or other parameters. The user interface may include components for uploading one or more data files containing this information. The user interface may be transmitted as a network page or other network data over the network 412 (FIG. 4) for rendering by a client application 436 (FIG. 4) executing on the client device 406 (FIG. 4). Alternatively, the client application 436 may make one or more API calls to order or provision an RBPN from a provider.

[0102] At box 506, the RBPN management service 424 receives a request from an organization to provision an RBPN. For example, a user may submit a form or otherwise interact with a user interface to submit the request. Alternatively, a client application 436 may make one or more API calls to request provisioning of an RBPN.

[0103] In box 507, the RBPN management service 424 can automatically initiate a probe of the organization's existing private network. For example, the organization may have an existing network, such as a wired, Ethernet network, a Wi-Fi network, or another type of network. Upon receiving the appropriate security credentials and accessing endpoints on the network, the RBPN management service 424 can automatically probe the network to ascertain customer requirements for the new RBPN 103 and associated core network. For example, the RBPN management service 424 may automatically determine the number of user devices on the existing network, the network bandwidth and demand associated with various applications or services, the existing delays observed to access various applications or services, the reliability of the existing system, etc. These observations may be used to set initial thresholds for delay, bandwidth, etc., in the RBPN to be deployed.

[0104] In box 509, the RBPN management service 424 determines the placement of cells 109 (FIG. 1A) within the RBPN 103. The placement may be determined to optimally cover one or more buildings of the organization. Both interior and exterior areas of the buildings can be covered as desired. This determination may include receiving information about the service area, including indoor and outdoor coverage, the number of devices to be connected, and the layout of physical network connections and power sources. In this regard, the RBPN management service 424 may determine the number of cells 109 from the network plan 439. Alternatively, the RBPN management service 424 may automatically determine the optimal number of cells 109 based at least in part on parameters such as target delay, bandwidth, signal strength, and reliability, taking into account the target customer area and / or site. In one embodiment, an unmanned aerial vehicle may be used to conduct a site survey of the area to be covered, possibly recording signal strength and observing the condition of the frequency spectrum, thereby determining available frequencies. The RBPN management service 424 may record the placement of cells 109 within the cellular topology 442 (FIG. 4). In some scenarios, the RBPN management service 424 may use machine learning to determine the optimal placement of cells 109 by deploying various placements of RBPNs 103 and evaluating their performance. Over time, by observing these deployments, the RBPN management service 424 can learn which placements perform better or worse than others and use these results to train a machine learning model.

[0105] In box 512, the RBPN management service 424 automatically reserves a frequency spectrum allocation for the RBPN 103. To this end, the RBPN management service 424 may automatically determine available frequencies in the cellular topology 442 from publicly available frequencies, customer-owned frequencies, and / or provider-owned frequencies. The frequency determination may take into account polarization, directivity, beam tilt, and / or other factors that may enable or hinder frequency reuse. The RBPN management service 424 may record the reservation in spectrum allocation 445 (FIG. 4). Additionally, the RBPN management service 424 may communicate with external services via network 412, such as spectrum reservation service 410 (FIG. 4) that implements a frequency spectrum reservation system, to make reservations and / or determine available frequencies.

[0106] In box 515, the RBPN management service 424 identifies the equipment needed to implement the RBPN 103. This may include antennas, radio units, computing devices implementing the provider substrate extension 224 (FIG. 2A), cables, switches, routers, fiber termination equipment, etc. In some cases, the computing devices may be included in outdoor units that will be installed outside the customer's premises. Such units may, in some instances, be self-contained except for power and network connections. In some scenarios, the RBPN management service 424 may use machine learning to determine the optimal placement of equipment by deploying and evaluating the performance of various placements of the RBPN 103. Over time, by observing these deployments, the RBPN management service 424 can learn which placements perform better or worse than others and use these results to train a machine learning model.

[0107] RBPN management service 424 may also determine, through machine learning, the optimal allocation of equipment for various customers. For example, for a certain type of customer, it may make sense to run network function workload 466 (FIG. 4) only on data center core computing device 118 (FIG. 1A), while for another type of customer, it may be optimal to run network function workload 466 on distributed computing device 112 (FIG. 1A) or centralized computing device 115 (FIG. 1A). Thus, RBPN management service 424 may determine not to deploy computing device 112 or 115 in favor of a cloud-only deployment of computing device 118.

[0108] In box 518, the RBPN management service 424, via the RBPN hardware deployment service 430 (FIG. 4), initiates the procurement of equipment for the RBPN 103. This may include automatically reserving equipment from the provider's existing inventory and / or placing one or more orders for equipment from one or more vendors.

[0109] In box 521, RBPN management service 424 causes RBPN hardware configuration service 427 (FIG. 4) to pre-configure one or more devices, such as computing devices that implement network functions, radio units, antennas, routers, etc. Such devices may be connected to network 412 as pre-deployed devices 409 (FIG. 4). RBPN hardware configuration service 427 uses radio unit configuration data 457 (FIG. 4), antenna configuration data 460 (FIG. 4), and network function configuration data 463 (FIG. 4) to perform the pre-configuration.

[0110] In box 522, the RBPN management service 424 can configure one or more network slices for the wireless private network 103 that can provide differentiated quality of service levels for different user devices, applications, or services. The quality of service levels can provide different latency, bandwidth / throughput, signal strength, reliability, and / or other service factors. For example, a customer may have a set of devices that require very low latency, so the RBPN management service 424 can configure a network slice that provides latency below a threshold for those devices. In another example, a first quality of service level can be provided for a first application and a second quality of service level can be provided for a second application.

[0111] In box 524, RBPN management service 424 causes RBPN hardware deployment service 430 to ship the equipment to the customer, including pre-configured and pre-deployed device 409. Various instructions for installing the equipment can be sent to the customer via client application 436. This portion of the operation of RBPN management service 424 then ends.

[0112] Turning to Figure 6, a flowchart is shown presenting an example of the operation of another portion of RBPN management service 424 according to various embodiments. It is understood that the flowchart of Figure 6 merely presents an example of many different types of functional arrangements that may be utilized to implement the operation of portions of RBPN management service 424 described herein. Alternatively, the flowchart of Figure 6 may be considered to illustrate example elements of a method implemented within computing environment 403 (Figure 4) according to one or more embodiments.

[0113] Starting at box 603, RBPN management service 424 receives a request from a customer to activate RBPN 103 (FIG. 1A). For example, the customer user may interact with a user interface generated by RBPN management service 424 to select an option to activate RBPN 103. Alternatively, client application 436 (FIG. 4) may make an API call to activate RBPN 103.

[0114] In box 606, the RBPN management service 424 launches one or more network function workloads 466 (FIG. 4) for the RBPN 103 within the allocated computing capacity 421 (FIG. 4). Note that the network function workloads 466 may be hosted on computing devices 418 (FIG. 4) on the provider substrate extension 224 (FIG. 2A), which may be located at a cell site, a customer site, and / or a data center for core network functions.

[0115] In some scenarios, a customer may prefer that their entire core network, or all of their network function workloads 466, be provisioned in a data center operated by the cloud service provider. This reduces the number of devices to maintain at the customer's site, which may reduce the customer's information technology staffing requirements. In other scenarios, a customer may want to keep as much data as possible on-site to ensure control over their data. In some scenarios, a customer may choose to migrate their network function workloads 466 from the cloud provider network 203 (FIG. 2A) to their premises or from their premises to the cloud provider network 203. An application programming interface (API) may be provided for the customer to manage where their network function workloads 466 are hosted. Thus, a customer may request that their network function workloads 406 be relocated from the cloud provider network 203 to a provider substrate extension 224 on their premises, and the cloud provider network 203 may implement the relocation in accordance with the request. Additionally, a customer may request that network function workloads 406 be transferred from provider substrate extension 224 on the customer's premises to cloud provider network 203, and cloud provider network 203 may implement the transfer in response to the request. In some scenarios, traffic within region 306 (FIG. 3) may be directed to computing capacity 421 allocated to a data center within region 306, while local network traffic may remain at the customer's premises.

[0116] In box 609, the RBPN management service 424 determines whether the devices and equipment on the RBPN 103 are properly connected. Because the devices and equipment may be configured for plug-and-play operation by the customer, the RBPN management service 424 may perform diagnostic evaluations to verify that instructions were followed and that the devices are powered and accessible via a network connection. In box 612, the RBPN management service 424 proceeds to activate the RBPN 103, thereby enabling the wireless device 106 to communicate with other hosts on the RBPN 103 and / or hosts on the Internet through the RBPN 103. This portion of the operation of the RBPN management service 424 then ends.

[0117] 7, a flowchart presenting an example of the operation of different portions of RBPN management service 424 according to various embodiments is shown. It is understood that the flowchart of FIG. 7 merely presents an example of many different types of functional arrangements that may be utilized to implement the operation of different portions of RBPN management service 424 as described herein. Alternatively, the flowchart of FIG. 7 may be considered to illustrate example elements of a method implemented within computing environment 403 (FIG. 4) according to one or more embodiments.

[0118] Beginning in box 703, the RBPN management service 424 monitors performance and usage metrics of the RBPN 103 (FIG. 1A). For example, the RBPN management service 424 may collect RBPN metrics 451 (FIG. 4) related to dropped packets, latency, bandwidth usage, signal strength, interference, etc. during operation of the RBPN 103. In box 706, the RBPN management service 424 determines to modify the RBPN 103 based at least in part on the performance and / or usage metrics and / or a customer request to modify the RBPN 103. For example, the RBPN management service 424 may determine whether observed performance is below a minimum threshold or whether observed usage is above a maximum threshold. In such cases, the RBPN management service 424 can automatically scale the quantity of VM instances, containers, functions, or other allocated computing capacity 421 (FIG. 4) that execute network functions on the RBPN 103. Alternatively, a customer can submit a request via a user interface or API to modify the RBPN 103. Such requests may include OSS and BSS management requests to specify access levels for specific devices or groups of devices on the RBPN 103. The RBPN management service 424 can perform network bandwidth and validation tests and provide monitoring and alerting capabilities for complete visibility into how the RBPN 103 is being used.

[0119] In box 709, the RBPN management service 424 determines an updated placement of cells 109 (FIG. 1A) within the RBPN 103. In this regard, the RBPN management service 424 may determine the number of cells 109 from an updated network plan 439 (FIG. 4). Alternatively, the RBPN management service 424 may automatically determine an updated optimal number of cells 109 based at least in part on parameters such as target delay, bandwidth, signal strength, and reliability, taking into account the target customer area and / or site. In one embodiment, an unmanned aerial vehicle is used to conduct an updated site survey of the area to be covered, possibly recording signal strength and observing frequency spectrum conditions to determine available frequencies or to determine locations where cells 109 may be underperforming. The RBPN management service 424 may record the updated placement of cells 109 within the cellular topology 442 (FIG. 4).

[0120] In box 712, the RBPN management service 424 can automatically identify and / or reserve updated allocations of frequency spectrum for the RBPN 103. To this end, the RBPN management service 424 can automatically determine available frequencies in the cellular topology 442 from publicly available frequencies, customer-owned frequencies, and / or provider-owned frequencies. The frequency determination may take into account polarization, directivity, beam tilt, and / or other factors that may enable or hinder frequency reuse. The RBPN management service 424 can record the updated reservation in spectrum allocation 445 (FIG. 4). Additionally, the RBPN management service 424 can communicate with external services via network 412 (FIG. 4), such as spectrum reservation service 410, which implements a frequency spectrum reservation system, to make updated reservations and / or determine available frequencies. Updated reservations may include releasing previously reserved spectrum and / or changing to a different spectrum in various scenarios.

[0121] In box 715, the RBPN management service 424 identifies the equipment needed to implement the modified RBPN 103. This may include antennas, radio units, computing devices implementing the provider substrate extension 224 (FIG. 2A), cables, switches, routers, fiber termination equipment, etc. In box 718, the RBPN management service 424 initiates procurement of equipment for the updated RBPN 103 via the RBPN hardware deployment service 430 (FIG. 4). This may include automatically reserving equipment from the provider's existing inventory and / or placing one or more orders for equipment from one or more vendors. Some or all of the existing equipment may be reused.

[0122] In box 721, RBPN management service 424 has RBPN hardware configuration service 427 (FIG. 4) pre-configure one or more new devices, such as computing devices implementing network functions, radio units, antennas, routers, etc. Such devices may be connected to network 412 as pre-deployed devices 409 (FIG. 4). RBPN hardware configuration service 427 uses radio unit configuration data 457 (FIG. 4), antenna configuration data 460 (FIG. 4), and network function configuration data 463 (FIG. 4) to perform the pre-configuration. The pre-configuration may include installing software, initializing the software, configuring the software to communicate with one or more services, setting frequencies, setting modulation types, setting signal strengths, setting directivity, and other types of pre-configuration. In box 724, RBPN management service 424 has RBPN hardware deployment service 430 ship equipment, including pre-configured and pre-deployed devices 409, to the customer. Various instructions for installing the equipment may be sent to the customer via client application 436.

[0123] In box 727, RBPN management service 424 can initiate a reconfiguration of existing equipment and devices via RBPN hardware configuration service 427. For example, existing radio units and / or antennas may change frequency, signal strength, and / or other parameters. Also, existing allocated computing capacity 421 for network function workloads 466 may be reprogrammed or terminated to be replaced with other allocated computing capacity 421 to perform network functions. In some cases, additional computing capacity may be deployed or provisioned via cloud provider network 203 (FIG. 2A).

[0124] In box 730, the RBPN management service 424 activates the modified RBPN 103. The correct state of the modified wireless private network 103 and associated core network may be verified before activation, after which this portion of the RBPN management service 424 operation ends.

[0125] 8, shown is a flowchart presenting an example of the operation of a portion of capacity management service 433 according to various embodiments. It is understood that the flowchart of FIG. 8 merely presents an example of many different types of functional arrangements that may be utilized to implement the operation of a portion of capacity management service 433 as described herein. Alternatively, the flowchart of FIG. 8 may be viewed as illustrating example elements of a method implemented within computing environment 403 (FIG. 4) according to one or more embodiments.

[0126] Beginning in box 803, the capacity management service 433 monitors the use of the RBPN 103 (FIG. 1A), including the use of individual cells 109 (FIG. 1A), devices, and / or communication links. In box 806, the capacity management service 433 determines to release capacity in the RBPN 103. For example, the capacity may be released based at least in part on underutilization or in response to a request from a customer. The request from the customer may be received using a user interface or via an API call.

[0127] In some scenarios, capacity in RBPN 103 may be released (or transferred to other locations) to provision capacity for customer workloads 469 (FIG. 4) that are higher priority or associated with a premium rate. For example, a customer may have an application that is highly sensitive to latency and is a high priority for the customer, and capacity management service 433 may determine that the optimal allocation is to move network function workloads 466 (FIG. 4) off the server hardware at the edge location to make room for the application to run at the edge location. Upon releasing capacity, capacity management service 433 may take one or more actions to reclaim and reuse the capacity.

[0128] In box 809, the capacity management service 433 may determine to remove one or more cells 109 from the RBPN 103. For example, the capacity management service 433 may identify a particular cell 109 that is underutilized relative to a threshold or that is relatively underutilized compared to other cells 109 in the RBPN 103. In box 812, the capacity management service 433 may automatically disable the cell equipment (e.g., radios, antennas), or the capacity management service 433 may transfer the equipment for use by another customer. In some cases, the capacity management service 433 may request the customer to return the equipment to the provider and / or ship the equipment to another customer so that the equipment can be used by the other customer. The capacity management service 433 may also automatically reduce bandwidth or reconfigure communication links to free up no longer needed bandwidth that was previously reserved for the RBPN 103.

[0129] In box 815, capacity management service 433 may terminate one or more network function workloads 466 (FIG. 4) that are performing one or more network functions for RBPN 103. Capacity management service 433 may determine that a VM instance, container, or function within allocated computing capacity 421 (FIG. 4) is no longer needed or is of lower priority than customer workloads 469 in terms of RBPN 103 usage or performance.

[0130] In box 818, capacity management service 433 initiates reallocation of computing capacity previously occupied by terminated network function workload 466. The computing capacity may be on computing devices 418 (FIG. 4) at the customer's location or at the provider's data center. In this regard, capacity management service 433 may use the freed capacity to launch VM instances, containers, or functions, thereby providing network functionality to another customer or another RBPN 103.

[0131] Alternatively, the capacity management service 433 may use the freed-up capacity to launch customer workloads 469 for customer applications other than network functions. For example, the capacity management service 433 may use the freed-up computing capacity to launch VM instances, containers, or functions for the customer workloads 469. Such VM instances, containers, or functions may be unrelated to the function of the RBPN 103. The VM instances, containers, or functions may be launched on a provider substrate extension 224 (FIG. 2A) already located at the cell site or otherwise at the customer premises, so that the computing capacity is not wasted or is instead used for higher priority workloads.

[0132] In box 821, the capacity management service 433 can automatically reallocate the released frequency spectrum to other customers or to other cells 109 within the RBPN 103. The capacity management service 433 can also reallocate the released network bandwidth.

[0133] In box 824, the capacity management service 433 reduces the charges to the customer associated with the RBPN 103 in consideration of the release of computing resources. In some cases, the charges may remain the same or increase if a higher level of service is provided to the customer or if a higher priority workload is run at the reclaimed capacity. This portion of the operation of the capacity management service 433 then ends.

[0134] 9, a schematic block diagram of a computing environment 403 is shown, according to an embodiment of the present disclosure. The computing environment 403 includes one or more computing devices 900. Each computing device 900 includes at least one processor circuit having, for example, a processor 903 and memory 906, both of which are coupled to a local interface 909. To this end, each computing device 900 may comprise, for example, at least one server computer or similar device. The local interface 909 may include, for example, a data bus or other bus structure including an associated address / control bus, as can be appreciated.

[0135] Stored in memory 906 are both data and several components executable by processor 903. In particular, stored in memory 906 and executable by processor 903 are RBPN management service 424, RBPN hardware configuration service 427, RBPN hardware deployment service 430, capacity management service 433, and potentially other applications. Also stored in memory 906 may be data store 415 and other data. Additionally, an operating system may be stored in memory 906 and executable by processor 903.

[0136] As can be appreciated, there may be other applications stored in memory 906 and executable by processor 903. If any component described herein is implemented in software, it may employ any one of several programming languages, such as, for example, C, C++, C#, ObjectiveC, Java, JavaScript, Perl, PHP, Visual Basic, Python, Ruby, Flash, or other programming languages.

[0137] Some software components are stored in memory 906 and executable by processor 903. In this regard, the term "executable" refers to a program file in a format that is ultimately executable by processor 903. Examples of executable programs may include, for example, a compiled program that can be loaded into a random access portion of memory 906 and converted into machine code in a format executable by processor 903; source code that may be expressed in a suitable format, such as object code, that can be loaded into a random access portion of memory 906 and executed by processor 903; or source code that can be interpreted by another executable program that generates instructions in the random access portion of memory 906 to be executed by processor 903. The executable program may be stored in any portion or component of memory 906, including, for example, random access memory (RAM), read-only memory (ROM), a hard drive, a solid-state drive, a USB flash drive, a memory card, an optical disk such as a compact disc (CD) or digital versatile disc (DVD), a floppy disk, a magnetic tape, or other memory component.

[0138] Memory 906 is defined herein to include both volatile and nonvolatile memory and data storage components. A volatile component is one that does not retain a data value upon loss of power. A nonvolatile component is one that retains data upon loss of power. Thus, memory 906 may include, for example, random access memory (RAM), read-only memory (ROM), a hard disk drive, a solid-state drive, a USB flash drive, a memory card accessed via a memory card reader, a floppy disk accessed via an associated floppy disk drive, an optical disk accessed via an optical disk drive, a magnetic tape accessed via an appropriate tape drive, and / or other memory components, or a combination of any two or more of these memory components. Additionally, RAM may include, for example, static random access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM), and other such devices. ROM may include, for example, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other similar memory devices.

[0139] Also, processor 903 may represent multiple processors 903 and / or multiple processor cores, and memory 906 may represent multiple memories 906 each operating in parallel processing circuitry. In such cases, local interface 909 may be a suitable network facilitating communication between any two of the multiple processors 903, between any processor 903 and any of the memories 906, or between any two of the memories 906, etc. Local interface 909 may include additional systems designed to coordinate this communication, including, for example, implementing load balancing. Processor 903 may have an electrical structure or some other available structure.

[0140] The RBPN management service 424, RBPN hardware configuration service 427, RBPN hardware deployment service 430, capacity management service 433, and various other systems described herein may be embodied in software or code executed by general-purpose hardware, as described above, but alternatively, the same may be embodied in dedicated hardware or a combination of software / general-purpose hardware and dedicated hardware. If embodied in dedicated hardware, each may be implemented as a circuit or state machine using any one or combination of numerous technologies. These technologies may include, but are not limited to, discrete logic circuits having logic gates to implement various logical functions upon application of one or more data signals, application-specific integrated circuits (ASICs) having appropriate logic gates, field-programmable gate arrays (FPGAs), or other components. Such technologies are generally well known to those skilled in the art and will not be described in detail herein.

[0141] The flowcharts in FIGS. 5-8 illustrate the functionality and operation of some implementations of the RBPN management service 424 and the capacity management service 433. If embodied in software, each block may represent a module, segment, or portion of code containing program instructions for implementing a particular logical function(s). The program instructions may be embodied in the form of source code, which includes human-readable statements written in a programming language, or machine code, which includes numerical instructions recognizable by a suitable execution system, such as the processor 903 in a computer system or other system. The machine code may be translated from the source code or the like. If embodied in hardware, each block may represent a circuit or multiple interconnected circuits for implementing a particular logical function(s).

[0142] While the flowcharts in FIGS. 5-8 depict a specific order of execution, it is understood that the order of execution may differ from that depicted. For example, the order of execution of two or more blocks may be swapped relative to the order depicted. Also, two or more blocks shown consecutively in FIGS. 5-8 may execute concurrently or with partial concurrence. Furthermore, in some embodiments, one or more blocks depicted in FIGS. 5-8 may be skipped or omitted. Additionally, any number of counters, state variables, warning semaphores, or messages may be added to the logic flows described herein for purposes such as improving usability, providing explanations, measuring performance, or providing clues for problem resolution. It is understood that all such variations are within the scope of the present disclosure.

[0143] Additionally, any logic or application described herein, including RBPN management service 424, RBPN hardware configuration service 427, RBPN hardware deployment service 430, and capacity management service 433, comprising software or code, can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system, such as, for example, processor 903 in a computer system or other system. In this sense, logic can include, for example, statements, including instructions and declarations, retrievable from a computer-readable medium and executable by an instruction execution system. In the context of this disclosure, a "computer-readable medium" can be any medium capable of containing, storing, or retaining the logic or application described herein for use by or in connection with an instruction execution system.

[0144] The computer-readable medium may include any one of numerous physical media, such as, for example, magnetic, optical, or semiconductor media. More specific examples of suitable computer-readable media would include, but are not limited to, magnetic tape, magnetic floppy disks, magnetic hard drives, memory cards, solid-state drives, USB flash drives, or optical disks. The computer-readable medium may also be random access memory (RAM), including, for example, static random access memory (SRAM) and dynamic random access memory (DRAM), or magnetic random access memory (MRAM). In addition, the computer-readable medium may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other types of memory devices.

[0145] Additionally, any logic or application described herein, including RBPN administration service 424, RBPN hardware configuration service 427, RBPN hardware deployment service 430, and capacity management service 433, may be implemented and structured in a variety of ways. For example, one or more applications described herein may be implemented as modules or components of a single application. Furthermore, one or more applications described herein may execute on a shared computing device or separate computing devices, or a combination thereof. For example, multiple applications described herein may execute on the same computing device 900 or on multiple computing devices 900 in the same computing environment 403.

[0146] Unless otherwise indicated, disjunctive language, such as the phrase "at least one of X, Y, or Z," is understood in context as generally used to indicate that an item, term, etc. may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended, and should not be intended, to imply that a particular embodiment requires that at least one of X, at least one of Y, or at least one of Z, respectively, be present.

[0147] Embodiments of the present disclosure can be described by at least the following clauses. Clause 1. A system comprising: at least one computing device in a cloud provider network; and instructions executable on the at least one computing device, which, when executed, cause the at least one computing device to at least receive a request from an organization to provision a wireless private network, including a radio access network and an associated core network, to cover one or more buildings of the organization; determine a placement of a plurality of cells covering the one or more buildings; automatically identify frequency spectrums for the plurality of cells via a spectrum reservation service; pre-configure cell equipment to implement the plurality of cells prior to shipping the cell equipment to the organization; and provision at least a portion of the associated core network for the organization on the cloud provider network.

[0148] Clause 2. The system of clause 1, wherein the instructions, when executed, further cause the at least one computing device to at least automatically probe the organization's existing private network to determine at least one requirement for the wireless private network or the associated core network.

[0149] Clause 3. The system described in clauses 1-2, wherein the instructions, when executed, further cause the at least one computing device to scale the amount of resources allocated to the associated core network within the cloud provider network based at least in part on at least one of a usage metric or a latency metric.

[0150] Clause 4. The system described in clauses 1 to 3, wherein the instructions, when executed, further cause the at least one computing device to at least implement differentiated quality of service levels in the wireless private network.

[0151] Clause 5. A cellular network comprising at least one cell providing wireless private network coverage to at least one site of an organization, and at least one computing device in a cloud provider network implementing at least one network function in an associated core network of said wireless private network.

[0152] Clause 6. The cellular network of clause 5, wherein the wireless private network provides a first quality of service level to a first application of the organization and a second quality of service level to a second application of the organization.

[0153] Clause 7. The cellular network of clauses 5-6, further comprising at least another computing device in a cloud provider substrate extension of said cloud provider network located on the premises of said organization, said at least another computing device implementing said at least one network function.

[0154] Clause 8. The cellular network of Clause 7, further comprising instructions executable on the at least one computing device, the instructions, when executed, causing the at least one computing device to at least: receive a request from the organization to transfer the at least one network function from the at least one computing device to the at least another computing device; and, in response to the request, transfer the at least one network function from the at least one computing device to the at least another computing device.

[0155] Clause 9. A cellular network as described in clauses 5 to 8, wherein at least one radio unit and at least one antenna implementing said at least one cell are operated by a cloud service provider that also operates said cloud provider network.

[0156] Clause 10. The cellular network of clauses 5-9, wherein usage metrics corresponding to usage of the wireless private network are tracked for the organization by a cloud service provider.

[0157] Clause 11. The cellular network described in Clauses 5-10, further comprising a wireless private network management service executable on the at least one computing device, the wireless private network management service, when executed, causing the at least one computing device to at least monitor at least one of usage or delay of the wireless private network, and, in response to determining that the at least one of the usage or delay meets a threshold criterion, pre-configure the equipment to implement additional cells in the wireless private network prior to shipping the equipment to the organization.

[0158] Clause 12. The cellular network of clause 11, wherein the wireless private network management service, when executed, further causes the at least one computing device to at least automatically identify additional frequency spectrum for the additional cell.

[0159] Clause 13. A cellular network as described in Clauses 11-12, wherein the wireless private network management service, when executed, further causes the at least one computing device to at least automatically allocate additional computing capacity of the at least one computing device to the at least one network function.

[0160] Clause 14. A method comprising: receiving, by at least one computing device, a request from an organization to a provider to provision a wireless private network and an associated core network to cover a site of the organization; determining a placement of one or more cells to cover the site; causing, by the at least one computing device, cell equipment to be pre-configured to implement the one or more cells prior to shipping the cell equipment to the organization; and provisioning, by the at least one computing device, at least a portion of the associated core network for the organization on a cloud provider network.

[0161] Clause 15. The method of clause 14, further comprising: determining, by the at least one computing device, to add an additional cell to the wireless private network; and having the at least one computing device pre-configure the additional cell equipment to implement the additional cell before shipping the additional cell equipment to the organization.

[0162] Clause 16. The method of clauses 14-15, further comprising scaling the quantity of computing resources allocated by the at least one computing device to one or more network functions of the associated core network within the cloud provider network.

[0163] Clause 17. The method of clauses 14-16, further comprising reserving, by said at least one computing device, a frequency spectrum for said one or more cells via a spectrum reservation service.

[0164] Clause 18. The method of clauses 14-17, further comprising probing, by the at least one computing device, an existing private network of the organization to determine at least one requirement for the wireless private network.

[0165] Clause 19. The method of clauses 14-18, further comprising implementing, by the at least one computing device, a first quality of service level over the wireless private network for a first device type, and implementing, by the at least one computing device, a second quality of service level over the wireless private network for a second device type.

[0166] Clause 20. The method of clauses 14 to 19, wherein the associated core network is provisioned within the cloud provider network.

[0167] It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations set forth for a clear understanding of the principles of the present disclosure. Many variations and modifications may be made to the above-described embodiment(s) without substantially departing from the spirit and principles of the present disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.

Claims

1. 1. A computer-implemented method comprising: receiving a request from an organization to a provider to provision a cellular network infrastructure for a wireless private network and an associated core network to cover a site of the organization with wireless signals; determining a layout of one or more cells covering the site of the organization; pre-configuring the cell equipment to implement the one or more cells prior to shipping the cell equipment to the organization; provisioning the cellular network infrastructure for the organization relative to the associated core network; A method comprising:

2. The method of claim 1 , wherein the cellular equipment comprises a computer server.

3. The method of claim 1 , wherein the associated core network is provisioned on the premises of the organization.

4. The method of claim 1 , wherein the associated core network is provisioned in a provider substrate extension of a cloud provider network.

5. determining to add an additional cell to the wireless private network; pre-configuring the additional cell equipment to implement the additional cell prior to shipping the additional cell equipment to the organization; The method of claim 1 further comprising:

6. The method of claim 1 , further comprising reserving a frequency spectrum for the one or more cells via a spectrum reservation service.

7. The method of claim 1 , further comprising probing an existing private network of the organization to determine at least one of bandwidth or latency requirements for the wireless private network.

8. Implementing a first quality of service level over the wireless private network for a first user equipment device type; implementing a second quality of service level over the wireless private network for a second user equipment device type; The method of claim 1 further comprising:

9. 1. A system including at least one computing device, the at least one computing device receiving a request from an organization for a provider to provision a cellular network infrastructure for a wireless private network and an associated core network, covering a site of the organization with wireless signals; pre-configuring the cell equipment to implement one or more cells before shipping the cell equipment to the organization; Provisioning the organization with the cellular network infrastructure for the associated core network. At least the system configured as follows:

10. The system of claim 9 , wherein the at least one computing device is further configured to determine an arrangement of the one or more cells to cover the site with the wireless signals.

11. 10. The system of claim 9, wherein the at least one computing device is further configured to probe at least an existing private network of the organization to determine at least one bandwidth or latency requirement of the wireless private network or the associated core network.

12. 10. The system of claim 9, wherein the at least one computing device is further configured to at least scale an amount of resources allocated to the associated core network based at least in part on at least one of a usage metric or a delay metric.

13. 10. The system of claim 9, wherein the at least one computing device is further configured to implement at least differentiated quality of service levels for individual user equipment devices in the wireless private network.

14. 1. A system including at least one computing device, the at least one computing device receiving a request from an organization to a provider for provisioning a cellular network infrastructure for a wireless private network, covering a site of the organization with wireless signals; pre-configuring the device to implement the wireless private network prior to shipping the device to the organization; A system configured to provision the cellular network infrastructure for the organization's core network.

15. The system of claim 14 , wherein the at least one computing device is further configured to at least reserve a spectrum allocation for the wireless private network using a spectrum reservation service.

16. The system of claim 14 , wherein the at least one computing device is further configured to at least determine an arrangement of equipment for covering the site with the wireless signals.

17. 15. The system of claim 14, wherein the at least one computing device is further configured to at least provision one or more network slices on the wireless private network to provide differentiated quality of service to individual user equipment devices.

18. The system of claim 14 , wherein at least a portion of the core network is provisioned in a cloud provider network.

19. The system of claim 14 , wherein at least a portion of the core network is provisioned on a server on the organization's premises.

20. 15. The system of claim 14, wherein the equipment comprises a computer server.

Images