Customer-defined capacity constraint planning for communication networks

Abstract

Various embodiments of client-defined capacity constraint planning in a communication network are disclosed. In one embodiment, a request for service from a radio-based network is received from a first client device. A determination is made that a network function in the radio-based network is capacity constrained. In response to determining that the network function in the radio-based network is capacity constrained and based at least in part on a set of rules specific to the radio-based network, service from the network function to a second client device is suspended. Access to the network function is provided to the first client device but not to the second client device.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of co-pending U.S. patent application No. 17 / 491,128, filed on September 30, 2021, and entitled “CUSTOMER-DEFINED CAPACITY LIMIT PLANS FOR COMMUNICATION NETWORKS,” which is incorporated by reference in its entirety as if set forth herein. Background Art

[0003] 5G is the fifth-generation technology standard for broadband cellular networks, slated to eventually replace the fourth-generation (4G) standard, Long Term Evolution (LTE). 5G technology promises significantly increased bandwidth, expanding the cellular market beyond smartphones and providing last-mile connectivity for desktops, set-top boxes, laptops, Internet of Things (IoT) devices, and more. Some 5G cells may utilize spectrum similar to 4G, while others may utilize spectrum in the millimeter wave (mmWave) band. Millimeter wave cells offer relatively small coverage areas but offer significantly higher throughput than 4G. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Many aspects of the present disclosure may be better understood with reference to the following drawings. The components in the drawings are not necessarily drawn to scale, but emphasis is placed on clearly illustrating the principles of the present disclosure. Additionally, in the drawings, like reference numerals designate corresponding parts throughout the various views.

[0005] Figure 1A is a diagram of an example of a communication network deployed and managed according to various embodiments of the present disclosure.

[0006] Figure 1B is an example of a private network used on an organizational campus with multiple buildings and deployed according to various embodiments of the present disclosure.

[0007] Figure 1C is a diagram of an example scenario illustrating an example implementation of capacity constraint planning according to various embodiments.

[0008] Figure 2A An example of a networking environment including a cloud provider network and also including various provider underlay extensions of the cloud provider network according to some embodiments of the present disclosure is shown, which may be in Figure 1A Used in various locations within a communications network.

[0009] Figure 2B depiction Figure 1A Example of cellularization and geographical distribution of communication networks.

[0010] Figure 3Showing some embodiments according to the present disclosure Figure 2A An example of a networking environment that includes geographically dispersed provider infrastructure extensions.

[0011] Figure 4 According to various embodiments of the present disclosure Figure 2A A schematic block diagram of a networking environment.

[0012] Figure 5 and Figure 6 is a diagram showing various embodiments according to the present disclosure as Figure 4 A flowchart of an example of the functionality of a portion of a network management service and network functions executed in a computing environment within a networked environment.

[0013] Figure 7 According to various embodiments of the present disclosure, Figure 4 A schematic block diagram of an example illustration of a computing environment employed in a networking environment. DETAILED DESCRIPTION

[0014] This disclosure relates to planning for customer-specified capacity constraints for communication networks. All networks have limited capacity in terms of network bandwidth, processing, and memory capabilities. In some cases, these capacities can be over-provisioned or designed to make reaching these limits unlikely. In other cases, these capacities can be designed to be reached frequently in order to fully utilize the provisioned capacity. For example, if all client devices are using the network, network bandwidth may be over-subscribed or under-provisioned relative to demand to reduce costs.

[0015] Typically, if capacity limits are reached in a network, the corresponding network equipment behaves in an unpredictable manner, dropping data packets or limiting client device access to the network. Network protocols, such as the Transmission Control Protocol (TCP), are designed to accommodate some transmission failures in best-effort networks without guarantees. In some cases, the network can enforce Quality of Service (QoS) guarantees for network slices or individual data flows, but the network may still handle constrained situations in unpredictable ways. For example, data flows may have the same QoS, but the flow may exceed bandwidth constraints, causing network equipment to drop data packets in an unspecified manner. Furthermore, standard QoS functionality does not allow for controlling client device access to networks with excess capacity.

[0016] Various embodiments of the present disclosure facilitate customers to specify network failure or capacity constraint plans for their access networks. Such networks may include radio-based networks, such as 4G and 5G radio access networks, and portions of the network may be provisioned using cloud provider network infrastructure. Such plans may define sets of rules that control how various network components handle exceeding capacity limits. Network components may include radio access network components and core network components, such as user plane functions (UPFs), admission control functions, and other functions. Capacity constraint plans may create relative priorities between different client devices or different types of client devices, as well as relative priorities between different types of network traffic. Depending on the configuration, these priorities may override configured network slices or QoS parameters in the event that capacity limits are exceeded. Thus, rather than causing the network to fail in an unpredictable manner, customer-defined capacity constraint plans enable components of the network to fail in such a way that certain client devices can still connect to the network and certain data flows can continue to be transmitted.

[0017] Previous radio-based network deployments relied on manual deployment and configuration at every step of the process. This proved to be extremely time-consuming and expensive. Furthermore, in previous generations, software was inherently tied to vendor-specific hardware, preventing customers from deploying alternative software. In contrast, with 5G, the hardware is decoupled from the software stack, which allows for greater flexibility and allows components of the radio-based network to execute on cloud provider infrastructure. Using a cloud distribution model for radio-based networks, such as 5G networks, can facilitate the processing of network traffic from hundreds of millions to billions of connected devices and compute-intensive applications, while delivering faster speeds, lower latency, and greater capacity than other types of networks.

[0018] Historically, businesses have had to choose between performance and price when evaluating their enterprise connectivity solutions. Cellular networks offer high performance, good indoor and outdoor coverage, and advanced Quality of Service (QoS) connectivity features, but dedicated cellular networks can be expensive and complex to manage. While Ethernet and Wi-Fi require less upfront investment and are easier to manage, businesses often find them less reliable, require significant effort to obtain optimal coverage, and don't offer QoS features such as guaranteed bit rate, latency, and reliability.

[0019] The disclosed radio-based private network service offers enterprises the best of both worlds—the performance, coverage, and QoS of carrier-grade cellular networks, with the deployment and operational simplicity and cost associated with Wi-Fi. The disclosed service provides suitable hardware in different form factors that enterprises can deploy at their sites, integrated with software that runs the entire network, from small cell sites to internet breakout points. Enterprises can freely deploy a variety of 5G devices and sensors across the enterprise (factory floors, warehouses, halls, and communication centers) and manage these devices, register users, and assign QoS through a management console. Using the disclosed technology, customers can allocate constant bit rate throughput to all their devices (such as cameras, sensors, or IoT devices), providing reliable, low-latency connectivity for devices operating on the factory floor and broadband connectivity for all handheld devices. The disclosed service manages all the software required to provide connectivity that meets specified constraints and requirements. This enables a whole new set of applications with strict QoS requirements or high IoT device density that traditionally could not run on Wi-Fi networks.

[0020] The exposed service supports multiple deployment scenarios. In a cloud-only deployment, the service provides a small radio cell that enterprise customers can place on-site, while the network functions and other network software run in the nearest cloud provider availability zone or edge location (or one of several nearest cloud provider availability zones or edge locations). For enterprises that prefer an on-premises deployment, the exposed service provides cloud provider hardware, such as the underlying extensions described herein. In this model, the network and applications remain on-premises, enabling enterprises to securely store and process data that needs to remain local (e.g., for regulatory compliance, security, etc.). Furthermore, the exposed service allows any compute and storage not used to run the radio-based network to be used to run any local workloads via the same APIs that customers use to run workloads in traditional cloud provider regions. Advantageously, because of this, enterprises don't have to worry about overscaling and wasting capacity, as the service will make any excess capacity available for local processing and will provision new hardware and software as network needs change. Furthermore, the exposed service provides application development APIs that expose and manage 5G capabilities such as QoS, enabling customers to build applications that can fully exploit the latency and bandwidth capabilities of their network without having to understand the network's details.

[0021] Additionally, the exposed services offer private zones for running local applications within the cloud provider's network. These private zones can be connected to and effectively become part of a broader regional zone, allowing customers to manage them using the same APIs and tools used within the cloud provider's network. Just like availability zones, virtual private network subnets can be assigned to private zones. APIs can be used to create and assign subnets to all zones a customer wishes to use, including private zones and existing zones. A management console provides a simplified process for creating private zones. Virtual machine instances and containers can be launched in private zones just as they would in regional zones. Customers can configure network gateways to define routes, assign IP addresses, set up network address translation (NAT), and more. Autoscaling can be used to scale the capacity of virtual machine instances or containers based on demand within the private zone. The same management and authentication APIs used within the cloud provider's network can be used within the private zone. In some cases, cloud services available in regional zones can be accessed remotely from the private zone over a secure connection, eliminating the need to upgrade or modify on-premises deployments.

[0022] Various embodiments of the present disclosure introduce methods that allow customers to order and deploy radio-based networks and associated core networks in an automated manner. Customers may include businesses and organizations that wish to establish radio-based networks for internal use (e.g., private 5G networks). Through various user interfaces or APIs, customers can specify their network plans or requirements (e.g., physical site layout and device / application types and quantities), and the various components required to implement the radio-based network for the customer can be automatically determined and provisioned. Hardware such as antennas, radio components, and computer servers can be pre-configured for the customer's radio-based network and shipped to the customer. The process of installing the pre-configured hardware is largely plug-and-play, and the radio-based network can be activated through a user interface or API. In addition to deploying radio-based networks (such as all or part of a new radio access network), various embodiments of the present disclosure can facilitate modification and management of radio-based networks, including the deployment of pre-configured equipment for additional cells and the allocation of QoS constraints for specific devices or applications on their radio-based networks.

[0023] Various embodiments of the present disclosure may also introduce the concepts of elasticity and utility computing from the cloud computing model to radio-based networks and associated core networks. For example, the disclosed technology may run core and radio access network functions and associated control plane management functions on cloud provider infrastructure, thereby creating a cloud-native core network and / or a cloud-native radio access network (RAN). In some implementations, such core and RAN network functions may be based on Third Generation Partnership Project (3GPP) specifications. By providing a cloud-native radio-based network, customers can dynamically scale their radio-based dedicated network based on utilization, latency requirements, and / or other factors. In some cases, the hardware sent to the customer includes sufficient capacity to run programs for operating and managing the radio-based network and the customer's other workloads (e.g., their applications), so that any capacity not used for the radio-based network is accessible to run workloads under the utility computing model. Advantageously, the customer's radio-based network can be scaled to this excess capacity as needed, for example, allowing the hardware usage requirements of the radio-based network to be increased even before new physical hardware is provisioned for the customer. Customers can also configure thresholds to receive alerts related to radio-based network usage and excess capacity usage of their provisioned infrastructure, allowing them to more effectively manage the provisioning of new infrastructure or de-provisioning of existing infrastructure based on their dynamic network and workload requirements.

[0024] Those skilled in the art will appreciate from this disclosure that certain embodiments may be able to achieve certain advantages, including some or all of the following: (1) improving the functionality of a computer network by allowing an administrator or operator to define customized rules that control the operation of network components when the network components are at capacity limits; (2) improving the functionality of a computer network by allowing high priority devices to connect to the network even when components of the computer network are at capacity limits; (3) improving the performance of a computer network by allowing high priority devices to receive high performance throughput by associating customized capacity limit rules with dynamically allocated network slices having high quality of service parameters; and so on.

[0025] One of the benefits of the present disclosure is the ability to deploy and link network functions together to deliver end-to-end services that meet specified constraints and requirements. According to the present disclosure, network functions organized into microservices work together to provide end-to-end connectivity. One set of network functions is part of the radio network, operating in cell towers and performing conversion of wireless signals to IP. Other network functions run in large data centers, executing subscriber-related business logic and routing IP traffic to the internet and back. For applications that use new 5G features (such as low-latency communications and reserved bandwidth), these two types of network functions need to work together to appropriately schedule and reserve wireless spectrum and perform real-time computation and data processing. The currently disclosed technology provides edge positioning hardware (as further described below) that integrates with network functions that operate across the entire network, from cell sites to internet breakout points, and orchestrates the network functions to meet the required quality of service (QoS) constraints. This enables a whole new set of applications with strict QoS requirements, from factory-based Internet of Things (IoT) to augmented reality (AR), virtual reality (VR), game streaming, and autonomous navigation support for connected vehicles, which were previously impossible to run on mobile networks.

[0026] The described "Elastic 5G" service provides and manages all the hardware, software, and network functions required to build a network. In some embodiments, the network functions may be developed and managed by a cloud service provider, however, the described control plane may manage network functions across a range of providers, allowing customers to use a single set of APIs to call and manage their choice of network functions on cloud infrastructure. The Elastic 5G service beneficially automates the creation of an end-to-end 5G network from hardware to network functions, thereby reducing the time to deploy the network and the operational costs of operating the network. By providing an API that exposes network capabilities, the exposed Elastic 5G service enables applications to simply specify the desired QoS as constraints, and then deploy and link network functions together to deliver an end-to-end service that meets the specified requirements, making it possible to easily build new applications.

[0027] This disclosure describes embodiments related to the creation and management of a cloud-native 5G core and / or cloud-native 5G RAN and associated control plane components. Cloud native refers to an approach to building and running applications that leverages the advantages of a cloud computing distribution model, such as dynamic scalability, distributed computing, and high availability (including geographic distribution, redundancy, and failover). Cloud native refers to how these applications are created and deployed to be suitable for deployment in a public cloud. While cloud-native applications can (and often do) run in a public cloud, they can also run in on-premises data centers. Some cloud-native applications can be containerized, i.e., packaging different parts, functions, or sub-units of an application in their own containers that can be dynamically orchestrated so that each part is actively scheduled and managed to optimize resource utilization. These containerized applications can be built using a microservices architecture to improve the overall agility and maintainability of the application.

[0028] In a microservices architecture, an application is arranged as a series of smaller sub-units ("microservices") that can be deployed and scaled independently of each other and can communicate with each other over a network. These microservices are typically fine-grained in that they have specific technical and functional granularity and typically implement lightweight communication protocols. The microservices of an application can perform different functions from each other, can be deployed independently, and can use different programming languages, databases, and hardware / software environments. Decomposing an application into smaller services can improve the modularity of the application, enable on-demand replacement of individual microservices, and parallelize development by enabling teams to develop, deploy, and maintain their microservices independently of each other. In some examples, microservices can be deployed using virtual machines, containers, or serverless functions. The disclosed core and RAN software can follow a microservices architecture such that the described radio-based network consists of independent sub-units that can be deployed and scaled on demand.

[0029] Now go to Figure 1A , shows an example of a communication network 100 deployed and managed according to various embodiments of the present disclosure. The communication network 100 includes an access network 103, which can correspond to a cellular network such as a fourth generation (4G) long term evolution (LTE) network, a fifth generation (5G) network, a 4G-5G hybrid core sixth generation (6G) network having both a 4G RAN and a 5G RAN, or another network that provides network access. The access network 103 can be operated by an enterprise, a non-profit organization, a school system, a government entity, a communications service provider, or a cloud service provider for another organization. In various embodiments, the access network 103 can use private network addresses or public network addresses.

[0030] Various deployments of access network 103 may include one or more of a core network and a RAN network, as well as a control plane for running the core network and / or RAN network on cloud provider infrastructure. As described above, these components can be developed in a cloud-native manner, for example using a microservices architecture, enabling efficient scaling of traffic and transactions using centralized control and distributed processing. These components can be based on 3GPP specifications in a manner that adheres to an application architecture (CUPS architecture) that separates control plane processing from user plane processing.

[0031] Access network 103 provides wireless network access to multiple client devices 106, which can be mobile devices or fixed-location devices. In various examples, client devices 106 can include smartphones, connected vehicles, IoT devices, sensors, machines (such as in manufacturing facilities), hotspots, and other devices. Client devices 106 are sometimes referred to as user equipment (UE) or customer premises equipment (CPE).

[0032] Access network 103 may include a radio access network (RAN) that provides wireless network access to multiple client devices 106 via multiple cells 109. Each of the cells 109 may be equipped with one or more antennas and one or more radios that transmit and receive wireless data signals to and from client devices 106. In some cases, based on the resources of the deployed hardware, cells 109 may have a limited device capacity (e.g., 64 devices, 128 devices, or other limits). Antennas may be configured for one or more frequency bands, and radios may also be frequency agile or tunable. Antennas may be associated with specific gains or beamwidths to focus signals in a specific direction or azimuth range, potentially allowing for frequency reuse in different directions. Antennas may also be horizontally, vertically, or circularly polarized. In some examples, radios may utilize multiple-input, multiple-output (MIMO) technology to transmit and receive signals. Thus, the RAN implements radio access technologies to enable radio connections with client devices 106 and provides connectivity to the core network of the radio-based dedicated network. The components of a RAN include the base stations and antennas that cover a given physical area, as well as the core network items required to manage connections to the RAN.

[0033] Data traffic is typically routed to the core network via a fiber optic transport network consisting of multiple hops of Layer 3 routers (e.g., at aggregation sites). The core network is typically housed in one or more data centers. The core network typically aggregates data traffic from terminal devices, authenticates subscribers and devices, applies personalized policies, and manages device mobility before routing traffic to operator services or the internet. For example, the 5G core may be decomposed into multiple microservice elements with control plane and user plane separation. The 5G core may include virtualized, software-based network functions (e.g., deployed as microservices) rather than physical network elements, and can therefore be instantiated in a multi-access edge computing (MEC) cloud infrastructure. The network functions of the core network may include a user plane function (UPF), an access and mobility management function (AMF), and a session management function (SMF), which will be described in more detail below. For data traffic destined for locations outside the communication network 100, the network functions typically include a firewall, through which traffic can enter or leave the communication network 100 to reach external networks (such as the internet or a cloud provider network). Note that in some embodiments, the communications network 100 may include facilities that allow traffic to enter or exit from sites further downstream in the core network (eg, at an aggregation site or access network 103).

[0034] The UPF provides an interconnection point between the mobile infrastructure and the data network (DN), namely the encapsulation and decapsulation of the General Packet Radio Service (GPRS) Tunneling Protocol for the User Plane (GTP-U). The UPF can also provide a session anchor point for providing mobility within the RAN, including sending one or more end marker packets to the RAN base station. The UPF can also handle packet routing and forwarding, including directing traffic to a specific data network based on traffic matching filters. Another feature of the UPF includes per-flow or per-application QoS processing, including transport-level packet marking for uplink (UL) and downlink (DL), and rate limiting. The UPF can be implemented as a cloud-native network function using a modern microservices approach, which can be deployed, for example, within a serverless framework (which abstracts the underlying infrastructure on which the code runs via managed services).

[0035] The AMF can receive connection and session information from the client device 106 or the RAN and can handle connection and mobility management tasks. For example, the AMF can manage handovers between base stations in the RAN. In some examples, the AMF can be considered an access point to the 5G core by terminating certain RAN control plane and client device 106 traffic. The AMF can also implement encryption and integrity protection algorithms.

[0036] The SMF handles session establishment or modification, such as by creating, updating, and deleting protocol data unit (PDU) sessions and managing session context within the UPF. The SMF also implements Dynamic Host Configuration Protocol (DHCP) and IP Address Management (IPAM). The SMF can be implemented as a cloud-native network function using a modern microservices approach.

[0037] The various network functions implementing access network 103 may be deployed in a distributed computing device 112, which may correspond to a general-purpose computing device configured to perform network functions. For example, distributed computing device 112 may execute one or more virtual machine instances, which in turn may be configured to execute one or more services that perform network functions. In one embodiment, distributed computing device 112 is a ruggedized machine deployed at each cell site.

[0038] In contrast, one or more centralized computing devices 115 can perform various network functions at a central site operated by a customer. For example, the centralized computing device 115 can be centrally located at the customer's premises in a well-equipped server room. The centralized computing device 115 can execute one or more virtual machine instances, which are in turn configured to execute one or more services that perform network functions.

[0039] In one or more embodiments, network traffic 103 from the access network is backhauled to one or more computing devices on the core network 118, which may be located in one or more data centers remote from the customer site. The core network 118 may also perform various network functions, including routing network traffic to and from the network 121, which may correspond to the Internet and / or other external public or private networks. The core network 118 may perform functionality related to the management of the communication network 100 (e.g., billing, mobility management, etc.) as well as transport functionality for relaying traffic between the communication network 100 and other networks.

[0040] Move to Figure 1B , shows an example of a radio-based private network 150 used on an organizational campus having multiple buildings 153 (such as a business, school, or other organizational premises such as a company) and deployed according to various embodiments of the present disclosure. Figure 1B An example with multiple buildings is depicted, but it should be understood that the disclosed technology is similarly applicable to any layout of a site, which may include one or more buildings and / or one or more outdoor spaces (such as a stadium or other outdoor venue).

[0041] The radio-based private network 150 in this non-limiting example includes four cells 156a, 156b, 156c, and 156d to completely cover the organizational campus. The cells 156 may overlap slightly to provide comprehensive coverage within each building 153. Adjacent or overlapping cells 156 are configured to operate at non-interfering frequencies. For example, cell 156a may use frequency A, cell 156b may use frequency B, and cell 156c may use frequency C, all of which are different frequencies when the coverage of the respective cells 156a, 156b, and 156c overlaps. However, cell 156d may use, for example, frequency A or B because the coverage of cell 156d does not overlap with cells 156a or 156b.

[0042] Note that cells 156 can be added to or removed from the radio-based private network 150 based on usage or other network metrics. In some cases, the signal strength to cells 156 can be increased to reduce the number of cells 156, or decreased to increase the number of cells 156, while allowing spectrum reuse between cells 156. Additionally, computing capacity can be added within the geographic area of an organization's campus or within a cloud provider's network to reduce latency, maintain security, and increase reliability of the radio-based private network 150 as needed. In some cases, the computing capacity of the software implementing the radio-based private network 150 can be provisioned largely or entirely within the cloud provider's network rather than at the customer's premises (such as in a cloud service provider's regional data center). This software can implement various network functions, such as UPF, AMF, SMF, etc., which can correspond to core network functions, central unit network functions, and distributed unit network functions. Some network functions, such as distributed unit network functions, can remain at the cell site.

[0043] Figure 1C is a diagram illustrating an example scenario of an example implementation of capacity constraint planning. At stage 160a, client devices 106a, 106b, and 106c are currently connected to communication network 100 ( Figure 1A ), where processing is performed in network function 163. In this example, it is assumed that network function 163 has a device capacity of three concurrent devices. Client device 106d then sends a service request 165 to access communication network 100, which involves processing using network function 163. Because network function 163 is already at capacity, network function 163 must determine how to handle incoming service request 165. The default rule may be to deny service request 165 because network function 163 is already at capacity. However, client device 106d may be a high-priority device, and the owner of communication network 100 may prefer that client device 106d have access.

[0044] Thus, the owner of communication network 100 can configure custom rules that override the default rules and provide service to high-priority client device 106d via network function 163. At stage 160b, network function 163 determines, based on rules from the capacity constraint plan, to select one or more specific client devices 106a, 106b, or 106c for disconnection. In this example, based on the rules, network function 163 selects client devices 106b and 106c for service suspension 167, while providing service to client device 106d via network function 163. Client devices 106b and 106c may be selected based on their relatively low priority compared to other client devices 106, based on a first-in, first-out method, based on a last-in, first-out method, based on a round-robin method, or based on other methods defined by the rules. In this example, multiple client devices 106b and 106c suspend their service to accommodate a single client device 106d. For example, client device 106d may have high bandwidth requirements for video, and client devices 106b and 106c may collectively utilize comparable bandwidth. In another example, a single client device 106 may suspend its service to accommodate a different client device 106 .

[0045] Figure 2A An example of a networked environment 200 including a cloud provider network 203 and also various provider underlay extensions of the cloud provider network according to some embodiments is shown, which can be used in conjunction with local customer deployments within the communication network 100 of Figure 1. The cloud provider network 203 (sometimes simply referred to as the "cloud") refers to a network-accessible pool of computing resources (such as computing resources, storage resources, and networking resources, applications, and services), which can be virtualized or bare metal. The cloud can provide convenient on-demand network access to a shared pool of configurable computing resources that can be programmatically provisioned and released in response to customer commands. These resources can be dynamically provisioned and reconfigured to adjust to variable loads. Therefore, cloud computing can be viewed as applications delivered as services over a publicly accessible network (e.g., the Internet, a cellular communication network) and the hardware and software in the cloud provider's data center that provides those services.

[0046] The cloud provider network 203 provides users with an on-demand, scalable computing platform over the network. For example, it allows users to have scalable "virtual computing devices" available for their use through their use of compute servers (which provide computing instances using one or both of a central processing unit (CPU) and a graphics processing unit (GPU), optionally in conjunction with local storage) and block storage servers (which provide virtualized persistent block storage for designated computing instances). These virtual computing devices have the attributes of a personal computing device, including hardware (various types of processors, local memory, random access memory (RAM), hard disk and / or solid-state drive (SSD) storage), operating system options, networking capabilities, and pre-loaded application software. Each virtual computing device can also virtualize its console input and output (e.g., keyboard, display, and mouse). This virtualization allows users to connect to their virtual computing devices using computer applications such as browsers, APIs, and software development kits (SDKs) to configure and use their virtual computing devices as if they were personal computing devices. Unlike personal computing devices, which have a fixed amount of hardware resources available to the user, the hardware associated with a virtual computing device can be scaled up or down depending on the resources the user requires.

[0047] As indicated above, users can use various interfaces 206 (e.g., APIs) via the intermediate network 212 to connect to virtualized computing devices and other cloud provider network 203 resources and services and configure and manage telecommunications networks such as 5G networks. An API refers to an interface and / or communication protocol between a client device 215 and a server such that if a client issues a request in a predefined format, the client should receive a response in a specific format or cause a defined action to be initiated. In the context of a cloud provider network, an API provides a gateway for clients to access the cloud infrastructure by allowing them to obtain data from the cloud provider network or cause actions within the cloud provider network, thereby enabling the development of applications that interact with resources and services hosted in the cloud provider network. The API can also enable different services of the cloud provider network to exchange data with each other. Users can choose to deploy their virtual computing systems to provide network-based services for their own use and / or for use by their customers or clients.

[0048] Cloud provider network 203 can include a physical network (e.g., sheet metal boxes, cables, rack hardware) called the underlay. The underlay can be thought of as the network fabric containing the physical hardware that runs the provider network's services. The underlay can be isolated from the rest of cloud provider network 203; for example, it may not be possible to route from an underlay network address to an address in the production network running the cloud provider service, or to a customer network hosting customer resources.

[0049] The cloud provider network 203 may also include an overlay network of virtualized computing resources running on the bottom layer. In at least some embodiments, a hypervisor or other device or process on the network bottom layer may use encapsulation protocol technology to encapsulate and route network packets (e.g., client IP packets) between client resource instances on different hosts within the provider network through the network bottom layer. Encapsulation protocol technology may be used on the network bottom layer to route encapsulated packets (also referred to as network bottom layer packets) between endpoints on the network bottom layer via an overlay network path or route. Encapsulation protocol technology may be viewed as providing a virtual network topology overlaid on the network bottom layer. Thus, network packets may be routed along the bottom layer network according to the structure in the overlay network (e.g., a virtual network that may be referred to as a virtual private cloud (VPC), a port / protocol firewall configuration that may be referred to as a security group). A mapping service (not shown) may coordinate the routing of these network packets. A mapping service may be a regional distributed lookup service that maps a combination of an overlay Internet Protocol (IP) and a network identifier to the bottom layer IP so that a distributed bottom layer computing device can find where to send the packet.

[0050] For illustration, each physical host device (e.g., a compute server, a block storage server, an object storage server, a control server) may have an IP address in the underlying network. Hardware virtualization technology enables multiple operating systems to run concurrently on a host computer, for example, as virtual machines (VMs) on a compute server. A hypervisor or virtual machine monitor (VMM) on the host allocates the host's hardware resources among the various VMs on the host and monitors the execution of the VMs. Each VM may be provided with one or more IP addresses in the overlay network, and the VMM on the host may know the IP addresses of the VMs on the host. The VMM (and / or other devices or processes on the network underlay) may use encapsulation protocol technology to encapsulate network packets (e.g., client IP packets) and route the network packets between virtualized resources on different hosts within the cloud provider network 203 through the network underlay. Encapsulation protocol technology may be used on the network underlay to route encapsulated packets between endpoints on the network underlay via overlay network paths or routes. Encapsulation protocol technology may be viewed as providing a virtual network topology overlaid on the network underlay. The encapsulation protocol technology may include a mapping service that maintains a mapping directory that maps IP overlay addresses (e.g., IP addresses visible to the customer) to underlying IP addresses (IP addresses not visible to the customer) that can be accessed by various processes on the cloud provider network 203 for routing packets between endpoints.

[0051] As shown in the figure, in various embodiments, the traffic and operations underlying the cloud provider network can be broadly divided into two categories: control plane traffic carried on the logical control plane 218 and data plane operations carried on the logical data plane 221. While the data plane 221 represents the movement of user data through the distributed computing system, the control plane 218 represents the movement of control signals through the distributed computing system. The control plane 218 typically includes one or more control plane components or services distributed across and implemented by one or more control servers. Control plane traffic typically includes management operations, such as establishing isolated virtual networks for various customers, monitoring resource usage and health, identifying specific hosts or servers to launch requested compute instances, provisioning additional hardware as needed, and so on. The data plane 221 includes customer resources (e.g., compute instances, containers, block storage volumes, databases, file storage) implemented on the cloud provider network. Data plane traffic typically includes non-management operations, such as transferring data to and from customer resources.

[0052] The control plane components are typically implemented on a set of servers separate from the data plane servers, and control plane traffic and data plane traffic can be sent over separate / different networks. In some embodiments, control plane traffic and data plane traffic can be supported by different protocols. In some embodiments, messages (e.g., packets) sent over the cloud provider network 203 include a flag indicating whether the traffic is control plane traffic or data plane traffic. In some embodiments, the payload of the traffic can be inspected to determine its type (e.g., control plane or data plane). Other techniques for distinguishing traffic types are possible.

[0053] As shown, the data plane 221 may include one or more computing servers, which may be bare metal (e.g., a single tenant) or may be virtualized by a hypervisor to run multiple VMs (sometimes referred to as "instances") or microVMs for one or more customers. These computing servers may support virtualized computing services (or "hardware virtualization services") for the cloud provider network. The virtualized computing service may be part of the control plane 218, allowing customers to issue commands via the interface 206 (e.g., an API) to launch and manage computing instances (e.g., VMs, containers) of their applications. The virtualized computing service may provide virtual computing instances with different computing and / or memory resources. In one embodiment, each of the virtual computing instances may correspond to one of several instance types. An instance type may be characterized by its hardware type, computing resources (e.g., the number, type, and configuration of CPUs or CPU cores), memory resources (e.g., the capacity, type, and configuration of local memory), storage resources (e.g., the capacity, type, and configuration of locally accessible storage devices), network resources (e.g., the characteristics of its network interface and / or network capabilities), and / or other suitable descriptive characteristics. Using instance type selection functionality, an instance type can be selected for a customer based, for example, at least in part, on input from the customer. For example, the customer can select an instance type from a set of predefined instance types. As another example, the customer can specify desired resources for the instance type and / or requirements for the workload the instance will run, and the instance type selection functionality can select an instance type based on such specifications.

[0054] The data plane 221 may also include one or more block storage servers, which may include persistent storage for storing customer data volumes and software for managing these volumes. These block storage servers may support the cloud provider's network's managed block storage service. The managed block storage service may be part of the control plane 218, allowing customers to issue commands via interface 206 (e.g., an API) to create and manage volumes for their applications running on compute instances. Block storage servers include one or more servers on which data is stored as blocks. A block is a sequence of bytes or bits, typically containing a certain integer number of records, with a maximum length of the block size. Block data is typically stored in a data buffer and read or written an entire block at a time. Generally, a volume may correspond to a logical collection of data, such as a set of data maintained on behalf of a user. A user volume can be viewed as an individual hard drive ranging in size, for example, from 1 GB to 1 terabyte (TB) or larger, consisting of one or more blocks stored on a block storage server. Although viewed as an individual hard drive, it should be understood that a volume may be stored as one or more virtualized devices implemented on one or more underlying physical host devices. A volume may be partitioned several times (e.g., up to 16 times), with each partition hosted by a different host. The data of a volume can be replicated between multiple devices within a cloud provider network to provide multiple copies of the volume (wherein such copies can collectively represent a volume on a computing system). Volume copies in a distributed computing system can advantageously provide automatic failover and recovery, for example, by allowing a user to access a primary copy of the volume or a secondary copy of the volume that is synchronized with the primary copy at a block level, so that a failure of the primary copy or the secondary copy does not prevent access to the volume information. The role of the primary copy can be to facilitate reads and writes (sometimes referred to as "input and output operations" or simply "I / O operations") on the volume and propagate any writes to the secondary copies (preferably synchronously in the I / O path, but asynchronous replication can also be used). The secondary copies can be updated synchronously with the primary copy and provide a seamless transition during a failover operation, whereby the secondary copy assumes the role of the primary copy and the previous primary copy is designated as a secondary copy or a new replacement secondary copy is provisioned. Although some examples in this document discuss primary and secondary copies, it should be understood that a logical volume may include multiple secondary copies. A computing instance can virtualize its I / O to a volume through a client. A client represents instructions that enable a compute instance to connect to a remote data volume (e.g., a data volume stored on a physically separate computing device accessed over a network) and perform I / O operations on the remote data volume. The client can be implemented on an offload card of a server that includes a processing unit (e.g., a CPU or GPU) of the compute instance.

[0055] The data plane 221 may also include one or more object storage servers, which represent another type of storage device within the cloud provider network. An object storage server includes one or more servers on which data is stored as objects within resources called buckets and can be used to support the managed object storage service of the cloud provider network. Each object typically includes the stored data, a variable amount of metadata that enables the object storage server to analyze various capabilities of the stored object, and a globally unique identifier or key that can be used to retrieve the object. Each bucket is associated with a given user account. Customers can store as many objects in their buckets as needed, can write, read, and delete objects in their buckets, and can control access to their buckets and the objects contained therein. In addition, in an embodiment with multiple different object storage servers distributed across different regions of the above-mentioned regions, users can select the region (or multiple regions) in which to store buckets, for example to optimize latency. Customers can use buckets to store various types of objects, including machine images that can be used to start VMs, and snapshots that represent point-in-time views of volume data.

[0056] The provider underlay extension 224 ("PSE") provides the resources and services of the cloud provider network 203 within a separate network (such as a telecommunications network), thereby extending the functionality of the cloud provider network 203 to new locations (e.g., for reasons related to latency in communicating with customer devices, legal compliance, security, etc.). In some implementations, the provider underlay extension 224 can be configured to provide the capacity of cloud-based workloads to run within the telecommunications network. In some implementations, the provider underlay extension 224 can be configured to provide core and / or RAN functionality of the telecommunications network and can be configured with additional hardware (e.g., radio access hardware). Some implementations can be configured to allow both, for example, by allowing unused capacity of the core and / or RAN functionality to be used to run cloud-based workloads.

[0057] As indicated, such provider underlay extensions 224 may include cloud provider network-managed provider underlay extensions 227 (e.g., formed of servers located in cloud provider-managed facilities separate from those associated with cloud provider network 203), communications service provider underlay extensions 230 (e.g., formed of servers associated with communications service provider facilities), customer-managed provider underlay extensions 233 (e.g., formed of servers located locally at a customer or partner facility), and other possible types of underlay extensions.

[0058] As shown in the exemplary provider underlay extension 224, the provider underlay extension 224 may similarly include a logical separation between a control plane 236 and a data plane 239, which extend the control plane 218 and data plane 221 of the cloud provider network 203, respectively. The provider underlay extension 224 may be pre-configured, for example, by the cloud provider network operator with an appropriate combination of hardware and software and / or firmware elements to support various types of computing-related resources, and to do so in a manner that reflects the experience of using the cloud provider network. For example, one or more provider underlay extension location servers may be provisioned by the cloud provider for deployment within the provider underlay extension 224. As described above, the cloud provider network 203 may offer a set of predefined instance types, each with a different type and quantity of underlying hardware resources. Each instance type may also be offered in a variety of sizes. To enable customers to continue using the same instance types and sizes they are used to in the region within the provider underlay extension 224, the servers may be heterogeneous servers. Heterogeneous servers may concurrently support multiple instance sizes of the same type and may also be reconfigured to host any instance type supported by their underlying hardware resources. Reconfiguration of heterogeneous servers can be done instantly using the available capacity of the servers, that is, while other VMs are still running and consuming other capacity of the servers at the provider underlay extension location. This can improve the utilization of computing resources within the edge location by allowing better packaging of running instances on the servers, and also provide a seamless experience regarding instance usage across the cloud provider network 203 and the provider underlay extension 227 managed by the cloud provider network.

[0059] The provider underlay extension server can host one or more compute instances. A compute instance can be a VM or a container that packages code and all its dependencies, allowing applications to run quickly and reliably across computing environments (e.g., including VMs and microVMs). Additionally, if the customer desires, the server can host one or more data volumes. Within a region of the cloud provider network 203, such volumes can be hosted on dedicated block storage servers. However, since the provider underlay extension 224 may have significantly less capacity than in the region, including such dedicated block storage servers may not provide an optimal utilization experience. Therefore, the block storage service can be virtualized within the provider underlay extension 224, with one of the VMs running the block storage software and storing the volume's data. Similar to the operation of the block storage service within a region of the cloud provider network 203, volumes within the provider underlay extension 224 can be replicated for persistence and availability. Volumes can be provisioned in their own isolated virtual network within the provider underlay extension 224. The compute instances and any volumes together constitute an extension of the provider network data plane 221 to the data plane 239 within the provider underlay extension 224.

[0060] In some implementations, servers within the provider underlay extension 224 may host certain local control plane components, such as those that enable the provider underlay extension 224 to continue operating in the event of a loss of connectivity back to the cloud provider network 203. Examples of these components include a migration manager that can move compute instances between provider underlay extension servers if necessary to maintain availability, and a key-value data store that indicates where volume replicas are located. However, the control plane 236 functionality for the provider underlay extension will typically remain within the cloud provider network 203 to allow customers to use as much of the provider underlay extension's resource capacity as possible.

[0061] The migration manager may have a centralized coordination component running in the region, and local controllers running on PSE servers (as well as servers in cloud provider data centers). When a migration is triggered, the centralized coordination component may identify the target edge location and / or target host, while the local controller may coordinate the data transfer between the source host and the target host. The described resource movement between hosts in different locations may take one of several forms of migration. Migration refers to moving virtual machine instances (and / or other resources) between hosts in a cloud computing network or between hosts outside a cloud computing network and hosts within the cloud. There are different types of migration, including live migration and restart migration. During a restart migration, the customer experiences an outage and effective power cycle of their virtual machine instances. For example, the control plane service may coordinate a restart migration workflow that involves tearing down the current domain on the original host and then creating a new domain for the virtual machine instance on the new host. The instance is restarted by shutting down on the original host and starting up again on the new host.

[0062] Live migration refers to the process of moving a running virtual machine or application between different physical machines without significantly interrupting the availability of the virtual machine (for example, the end user will not notice the downtime of the virtual machine). When the control plane executes the live migration workflow, it can create a new "passive" domain associated with the instance, while the instance's original domain continues to run as the "active" domain. The virtual machine's memory (including any in-memory state of running applications), storage, and network connectivity are transferred from the original host with the active domain to the destination host with the inactive domain. While the memory contents are transferred to the destination host, the virtual machine may be briefly paused to prevent state changes. The control plane can transition the inactive domain to become the active domain and demote the original active domain to the inactive domain (sometimes called "rollover"), after which the inactive domain can be discarded.

[0063] Techniques for various types of migrations involve managing a critical phase: the time that a virtual machine instance is unavailable to customers, which should be kept as short as possible. In currently disclosed migration techniques, this can be particularly challenging because resources are being moved between hosts in geographically separated locations that can be connected via one or more intermediate networks. For live migrations, the disclosed techniques can dynamically determine the amount of memory state data to pre-copy (e.g., while the instance is still running on the source host) and to post-copy (e.g., after the instance begins running on the destination host) based on, for example, latency between the locations, network bandwidth / usage patterns, and / or based on which memory pages the instance uses most frequently. In addition, the specific time to transfer memory state data can be dynamically determined based on network conditions between the locations. This analysis can be performed by a migration management component in the region, or by a migration management component running locally in the source edge location. If the instance has access to virtualized storage, both the source and target domains can be attached to the storage simultaneously to enable uninterrupted access to their data during the migration and in the event a rollback to the source domain is required.

[0064] The server software running on the provider underlay extension 224 can be designed by the cloud provider to run on the cloud provider underlay network, and this software can be able to run unmodified in the provider underlay extension 224 to create a dedicated copy of the underlay network ("shadow underlay") within the edge location by using a local network manager 242. The local network manager 242 can run on the provider underlay extension 224 server and bridge the shadow underlay with the provider underlay extension 224 network, for example, by acting as a virtual private network (VPN) endpoint or endpoint between the provider underlay extension 224 and the proxies 245, 248 in the cloud provider network 203 and by implementing a mapping service (for traffic encapsulation and decapsulation) to associate data plane traffic (from the data plane proxy 248) and control plane traffic (from the control plane proxy 245) with the appropriate servers. By implementing a local version of the provider network's underlay overlay mapping service, the local network manager 242 allows resources in the provider underlay extension 224 to communicate seamlessly with resources in the cloud provider network 203. In some implementations, a single local network manager 242 can perform these actions for all servers hosting compute instances in the provider underlay extension 224. In other implementations, each of the servers hosting the compute instances may have a dedicated local network manager 242. In multi-rack edge locations, inter-rack communications may go through the local network managers 242, where the local network managers maintain open tunnels between each other.

[0065] Provider underlay extension locations can utilize secure networking tunnels through the provider underlay extension 224 network to the cloud provider network 203, for example, to maintain the security of customer data while traversing the provider underlay extension 224 network and any other intermediary networks (possibly including the public internet). Within the cloud provider network 203, these tunnels consist of virtual infrastructure components including an isolated virtual network (e.g., in an overlay network), a control plane proxy 245, a data plane proxy 248, and an underlay network interface. These proxies 245 and 248 can be implemented as containers running on compute instances. In some embodiments, each server in a provider underlay extension 224 location hosting a compute instance can utilize at least two tunnels: one for control plane traffic (e.g., Constrained Application Protocol (CoAP) traffic) and one for encapsulated data plane traffic. A connectivity manager (not shown) within the cloud provider network 203 manages the cloud provider network-side lifecycle of these tunnels and their components, for example, by automatically provisioning them when needed and maintaining them in a healthy operational state. In some embodiments, direct connections between the provider underlay extension 224 location and the cloud provider network 203 can be used for control and data plane communications. Compared to VPNs that go through other networks, direct connections provide constant bandwidth and more consistent network performance because their network path is relatively fixed and stable.

[0066] A control plane (CP) agent 245 can be provisioned in the cloud provider network 203 to represent a specific host in an edge location. The CP agent 245 acts as an intermediary between the control plane 218 in the cloud provider network 203 and the control plane targets in the control plane 236 of the provider underlay extension 224. Specifically, the CP agent 245 provides the infrastructure for tunneling management API traffic destined for the provider underlay extension servers from the regional underlay to the provider underlay extension 224. For example, a virtualized compute service in the cloud provider network 203 can issue a command to the VMM of a server in the provider underlay extension 224 to launch a compute instance. The CP agent 245 maintains a tunnel (e.g., VPN) to the local network manager 242 of the provider underlay extension. The software implemented in the CP agent 245 ensures that only well-formed API traffic leaves and returns to the underlay. The CP agent 245 provides a mechanism to expose remote servers on the cloud provider underlay while still protecting the underlying security materials (e.g., encryption keys, security tokens) from leaving the cloud provider network 203. The unidirectional control plane traffic tunnel imposed by the CP proxy 245 also prevents any (potentially compromised) device from calling back to the underlay. The CP proxy 245 can be instantiated one-to-one with a server at the provider underlay extension 224, or can be capable of managing control plane traffic for multiple servers in the same provider underlay extension.

[0067] A data plane (DP) agent 248 may also be provisioned in the cloud provider network 203 to represent a specific server in the provider underlay extension 224. The DP agent 248 acts as a shadow or anchor for the server and can be used by services within the cloud provider network 203 to monitor the health of the host (including its availability, used / idle compute and capacity, used / idle storage and capacity, and network bandwidth utilization / availability). The DP agent 248 also allows isolated virtual networks to span the provider underlay extension 224 and the cloud provider network 203 by acting as a proxy for servers in the cloud provider network 203. Each DP agent 248 may be implemented as a packet forwarding compute instance or container. As shown, each DP agent 248 may maintain a VPN tunnel with a local network manager 242, which manages traffic to the server represented by the DP agent 248. This tunnel may be used to send data plane traffic between the provider underlay extension server and the cloud provider network 203. Data plane traffic flowing between the provider underlay extension 224 and the cloud provider network 203 may pass through the DP agent 248 associated with the provider underlay extension 224. For data plane traffic flowing from the provider underlay extension 224 to the cloud provider network 203, the DP agent 248 may receive the encapsulated data plane traffic, verify its correctness, and allow it to enter the cloud provider network 203. The DP agent 248 may forward the encapsulated traffic from the cloud provider network 203 directly to the provider underlay extension 224.

[0068] The local network manager 242 can provide secure network connectivity with the agents 245, 248 established in the cloud provider network 203. After the connection between the local network manager 242 and the agents 245, 248 has been established, the customer can issue commands via the interface 206 to instantiate a compute instance (and / or perform other operations using the compute instance) using the provider underlay extension resources in a manner similar to how such commands would be issued for a compute instance hosted within the cloud provider network 203. From the customer's perspective, the customer can now seamlessly use local resources within the provider underlay extension (as well as resources located in the cloud provider network 203, if desired). The compute instance located on the server at the provider underlay extension 224 can communicate with electronic devices located on the same network, as well as with other resources located in the cloud provider network 203 as needed. A local gateway 251 can be implemented to provide network connectivity between the provider underlay extension 224 and the network associated with the extension (e.g., the communication service provider network in the example of the communication service provider underlay extension 230).

[0069] There may be situations where data needs to be transferred between the object storage service and the provider underlay extension (PSE) 224. For example, the object storage service may store machine images used to boot VMs, as well as snapshots representing point-in-time backups of volumes. The Object Gateway, which can be provisioned on a PSE server or dedicated storage device, provides customers with configurable, per-bucket caching of the contents of their object storage buckets in the provider underlay extension 224 to minimize the impact of PSE regional latency on customer workloads. The Object Gateway may also temporarily store snapshot data from snapshots of volumes in the provider underlay extension 224, then synchronize it with the object servers in the region when possible. The Object Gateway may also store customer-specified machine images for use within the provider underlay extension 224 or on the customer's premises. In some implementations, data within the provider underlay extension 224 may be encrypted with a unique key, and for security reasons, the cloud provider may restrict key sharing from regions to the provider underlay extension 224. Consequently, data exchanged between the object storage server and the Object Gateway may utilize encryption, decryption, and / or re-encryption to preserve security boundaries regarding encryption keys or other sensitive data. The transformation agent may perform these operations and may create a PSE bucket (on the object storage server) using the PSE encryption key to store the snapshot data and the machine image data.

[0070] In the manner described above, the provider underlay extension 224 forms an edge location because it provides the resources and services of the cloud provider network 203 outside of a traditional cloud provider data center and closer to the customer's devices. Edge locations, as referred to herein, can be structured in a variety of ways. In some implementations, an edge location can be an extension of the cloud provider network underlay, including a limited amount of capacity provided outside of an availability zone (e.g., in a cloud provider's small data center or other facility located near customer workloads and potentially far from any availability zone). Such edge locations can be referred to as "far zones" (due to being far from other availability zones) or "near zones" (due to being close to customer workloads). Near zones can be connected to publicly accessible networks such as the Internet in various ways (e.g., directly, via another network, or via a dedicated connection to a zone). While typically near zones have more limited capacity than a zone, in some cases, near zones may have considerable capacity, such as thousands or more racks.

[0071] In some implementations, an edge location may be an extension of a cloud provider network underlay formed by one or more servers located locally at a customer or partner facility, where such servers communicate with a nearby availability zone or region of the cloud provider network over a network (e.g., a publicly accessible network such as the Internet). This type of underlay extension located outside of a cloud provider network data center may be referred to as an "outpost" of the cloud provider network. Some outposts may be integrated into a communications network, for example, as multi-access edge computing (MEC) sites whose physical infrastructure is distributed across telecom data centers, telecom aggregation sites, and / or telecom base stations within the telecom network. In a local example, the limited capacity of the outpost may be available only to the customer who owns the premises (and any other accounts permitted by the customer). In a telecom example, the limited capacity of the outpost may be shared among multiple applications (e.g., games, virtual reality applications, healthcare applications) that send data to users of the telecom network.

[0072] Edge locations may include data plane capacity that is at least partially controlled by the control plane of a nearby availability zone of the provider network. Thus, an availability zone group may include a "parent" availability zone and any "child" edge locations that are subordinate to the parent availability zone (e.g., controlled at least in part by its control plane). Certain limited control plane functionality (e.g., features required for low-latency communication with customer resources and / or features that enable the edge location to continue operating when disconnected from the parent availability zone) may also exist in some edge locations. Thus, in the above examples, an edge location refers to an extension of at least data plane capacity located at the edge of the cloud provider network, close to customer devices and / or workloads.

[0073] exist Figure 1A In the example of Figure 1A ), centralized computing device 115 ( Figure 1A ) and core network 118( Figure 1A ) can be implemented by a provider underlay extension 224 of the cloud provider network 203. The installation or location of the provider underlay extension 224 within the communication network 100 may vary depending on the specific network topology or architecture of the communication network 100. The provider underlay extension 224 can generally be connected to any location on the communication network 100 where packet-based traffic (e.g., IP-based traffic) can be interrupted. In addition, communications between a given provider underlay extension 224 and the cloud provider network 203 typically transit at least a portion of the communication network 100 securely (e.g., via a secure tunnel, a virtual private network, a direct connection, etc.).

[0074] In 5G wireless network development efforts, edge locations may be considered as a possible implementation of multi-access edge computing (MEC). Such edge locations may be connected to various points in the 5G network that provide interruption for data traffic as part of a user plane function (UPF). Older wireless networks may also contain edge locations. For example, in a 3G wireless network, an edge location may be connected to a packet-switched network portion of the communication network 100, such as to a serving general packet radio service support node (SGSN) or to a gateway general packet radio service support node (GGSN). In a 4G wireless network, an edge location may be connected to a serving gateway (SGW) or a packet data network gateway (PGW) as part of a core network or evolved packet core (EPC). In some embodiments, traffic between the provider underlay extension 224 and the cloud provider network 203 may be interrupted from the communication network 100 without being routed through the core network.

[0075] In some embodiments, the provider underlay extension 224 may be connected to more than one communication network associated with a respective customer. For example, when two communication networks of respective customers share or route traffic through a common point, the provider underlay extension 224 may be connected to both networks. For example, each customer may allocate a portion of its network address space to the provider underlay extension, and the provider underlay extension may include a router or gateway that can differentiate between traffic exchanged with each of the communication networks 100. For example, traffic destined for the provider underlay extension 224 from one network may have a different destination IP address, source IP address, and / or virtual local area network (VLAN) tag than traffic received from another network. Traffic originating from the provider underlay extension to a destination on one of the networks may be similarly encapsulated with the appropriate VLAN tag, source IP address (e.g., from a pool assigned to the provider underlay extension from the destination network address space), and destination IP address.

[0076] Figure 2B Depicting a communication network 100 for providing a highly available user plane function (UPF) Figure 1A ) of cellularization and geographical distribution. Figure 2BIn

[15] , user devices 254 communicate with request routers 255 to route requests to one of multiple control plane cells 257a and 257b. Each control plane cell 257 may include a network service API gateway 260, a network slice configuration 262, a network service monitoring function 264, site planning data 266 (including a description of the required layout, device type, number of devices, etc. for a customer site), a network service / function catalog 268, a network function orchestrator 270, and / or other components. Large control planes can be divided into multiple cells to reduce the likelihood of large-scale errors affecting a wide range of customers, for example by having one or more cells per customer, per network, or per independently operated region.

[0077] The network service / function catalog 268 is also referred to as the NF repository function (NRF). In a service-based architecture (SBA) 5G network, control plane functions and a common data repository may be delivered through a set of interconnected network functions built using a microservices architecture. The NRF may maintain a record of available NF instances and the services they support, allowing other NF instances to subscribe and be notified of registrations from NF instances of a given type. Thus, the NRF may support service discovery by receiving discovery requests from NF instances along with details of which NF instances support a particular service. The network function orchestrator 270 may perform NF lifecycle management including instantiation, scale-out / scale-in, performance measurement, event correlation, deployment capacity limit rule sets, and termination. The network function orchestrator 270 may also onboard new NFs, manage migration to new or updated versions of existing NFs, identify the set of NFs that are appropriate for a particular network slice or the larger network, and manage the lifecycle of the NFs across the constituent access networks 103( Figure 1A )’s different computing devices and sites orchestrate NFs.

[0078] The control plane cell 257 can communicate with one or more cell sites 272, one or more customer local data centers 274, one or more local regions 276, and one or more regional regions 278. The cell site 272 includes computing hardware 280 that executes one or more distributed unit (DU) network functions 282. The customer local data center 274 includes computing hardware 283 that executes one or more DU or central unit (CU) network functions 284, a network controller 285, a UPF 286, one or more edge applications 287 corresponding to customer workloads, and / or other components.

[0079] The local region 276 (which may be located in a data center operated by a cloud service provider) may execute one or more core network functions 288, such as the AMF, SMF, a network exposure function (NEF) that securely exposes the services and capabilities of other network functions, and a unified data management (UDM) function that manages subscriber data for authorization, registration, and mobility management. The local region 276 may also execute the UPF 286, a metrics processing service 289, and one or more edge applications 287.

[0080] A regional zone 278 (which may be located in a data center operated by a cloud service provider) may execute one or more core network functions 288; a UPF 286; an operations support system (OSS) 290 that supports network management systems, service delivery, service implementation, service assurance, and customer service; an Internet Protocol Multimedia Subsystem (IMS) 291; a business support system (BSS) 292 that supports product management, customer management, revenue management, and / or order management; one or more portal applications 293, and / or other components.

[0081] In this example, the communication network 100 employs a cellular architecture to reduce the blast radius of each component. At the top level, the control plane is located in multiple control plane cells 257 to prevent a single control plane failure from affecting the entire deployment.

[0082] Within each control plane cell 257, multiple redundant stacks can be provided, with the control plane shifting traffic to secondary stacks as needed. For example, a cell site 272 can be configured to utilize a nearby local region 276 as its default core network. In the event that the local region 276 experiences an outage, the control plane can redirect the cell site 272 to use a backup stack in a regional region 278. Traffic that would normally be routed from the Internet to the local region 276 can be shifted to endpoints in the regional region 278. Each control plane cell 257 can implement a "stateless" architecture that shares a common session database across multiple sites, such as across availability zones or edge sites.

[0083] Figure 3 Shown is an example of a geographically dispersed provider infrastructure extension 224 ( Figure 2A) (or "edge location 303"). As shown, the cloud provider network 203 can be formed into multiple regions 306, where a region is a separate geographic area where the cloud provider has one or more data centers 309. Each region 306 can include two or more availability zones (AZs) connected to each other via a dedicated high-speed network, such as, for example, a fiber optic communication connection. An availability zone is an isolated fault domain that includes one or more data center facilities with separate power, separate networking, and separate cooling from other availability zones. The cloud provider may strive to locate availability zones within a region far enough apart that a natural disaster, a large-scale power outage, or other unexpected event will not take more than one availability zone offline at the same time. Customers can connect to resources within an availability zone of the cloud provider network via a publicly accessible network (e.g., the Internet, a cellular communication network, a communication service provider network). A transit center (TC) is the primary backbone location that links customers to the cloud provider network and can be co-located at other network provider facilities (e.g., an Internet service provider, a telecommunications provider). Two or more TCs can be operated per region for redundancy. The regions 306 are connected to a global network that includes private networking infrastructure (e.g., fiber optic connections controlled by a cloud service provider) that connects each region 306 to at least one other region. Cloud provider network 203 can deliver content from points of presence (PoPs) outside of, but connected to, these regions 306 via edge locations 303 and regional edge cache servers. This partitioning and geographic distribution of computing hardware enables cloud provider network 203 to provide customers with low-latency access to resources worldwide with a high degree of fault tolerance and stability.

[0084] The number of edge locations 303 can be much higher than the number of regional data centers or availability zones. This widespread deployment of edge locations 303 can provide low-latency connectivity to the cloud for a much larger group of end-user devices (compared to those that happen to be very close to a regional data center). In some embodiments, each edge location 303 can be peered to a portion of the cloud provider network 203 (e.g., a parent availability zone or regional data center). This peering allows various components operating in the cloud provider network 203 to manage the computing resources of the edge locations 303. In some cases, multiple edge locations 303 can be located or installed in the same facility (e.g., separate racks of computer systems) and managed by different zones or data centers to provide additional redundancy. It should be noted that although edge locations 303 are generally depicted herein as being within the communication service provider network or access network 103 ( FIG. 1 ), in some cases, such as when the cloud provider network facilities are relatively close to the communication service provider facilities, the edge locations 303 can remain within the physical premises of the cloud provider network 203 while being connected to the communication service provider network via fiber or other network links.

[0085] Edge locations 303 can be structured in a variety of ways. In some implementations, edge locations 303 can be an extension of the cloud provider network infrastructure, including a limited amount of capacity provided outside of the availability zones (e.g., in a cloud provider's small data center or other facility located near customer workloads and potentially far from any availability zones). Such edge locations 303 can be referred to as local regions (due to being more local or closer to a group of users than traditional availability zones). Local regions can be connected to publicly accessible networks such as the Internet in various ways (such as directly, via another network, or via a dedicated connection to region 306). Although local regions typically have more limited capacity than regions 306, in some cases, local regions may have considerable capacity, such as thousands or more racks. Some local regions may use infrastructure similar to typical cloud provider data centers, rather than the edge location 303 infrastructure described herein.

[0086] As indicated herein, the cloud provider network 203 can be formed into a plurality of regions 306, wherein each region 306 represents a geographic region in which the cloud provider clusters data centers. Each region 306 can also include multiple (e.g., two or more) availability zones (AZs) connected to each other via a dedicated high-speed network, such as a fiber optic communication connection. An AZ can provide an isolated fault domain comprising one or more data center facilities having separate power, separate networking, and separate cooling relative to those in another AZ. Preferably, the AZs within a region 306 are sufficiently far apart from each other so that the same natural disaster (or other failure-inducing event) does not affect more than one AZ simultaneously or take more than one AZ offline. Customers can connect to the AZs of the cloud provider network via a publicly accessible network (e.g., the Internet, a cellular communication network).

[0087] The decision of a given edge location 303 as a parent for an AZ or region 306 in a cloud provider network 203 can be based on a number of factors. One such parenting factor is data sovereignty. For example, to keep data originating from a communications network in a certain country within that country, an edge location 303 deployed within that communications network can be parented to an AZ or region 306 in that country. Another factor is service availability. For example, some edge locations 303 may have different hardware configurations, such as the presence of components such as local non-volatile storage (e.g., solid-state drives) for customer data, graphics accelerators, and so on. Some AZs or regions 306 may lack services that utilize these additional resources; therefore, an edge location can be parented to an AZ or region 306 that supports these resources. Another factor is the latency between an AZ or region 306 and an edge location 303. While deploying an edge location 303 within a communications network has latency benefits, those benefits can be offset by parenting an edge location 303 to a distant AZ or region 306, which can introduce significant latency to traffic from the edge location 303 to the region. Thus, edge location 303 is typically the parent of a nearby (in terms of network latency) AZ or zone 306 .

[0088] refer to Figure 4 , shows a networked environment 400 according to various embodiments. The networked environment 400 includes a computing environment 403, one or more client devices 406, one or more pre-deployed devices 409, a spectrum reservation service 410, and one or more access networks 103, which are in data communication with each other via a network 412. The network 412 includes, for example, the Internet, an intranet, an extranet, a wide area network (WAN), a local area network (LAN), a wired network, a wireless network, a cable television network, a satellite network, or other suitable network, or any combination of two or more such networks.

[0089] The computing environment 403 may include, for example, a server computer or any other system that provides computing capacity. Alternatively, the computing environment 403 may employ a plurality of computing devices that may, for example, be arranged in one or more server groups or computer groups or other arrangements. Such computing devices may be located in a single device or may be distributed across many different geographical locations. For example, the computing environment 403 may include a plurality of computing devices that together may include hosted computing resources, grid computing resources, and / or any other distributed computing arrangement. In some cases, the computing environment 403 may correspond to an elastic computing resource, in which the allocated capacity of processing, network, storage, or other computing-related resources may vary over time. For example, the computing environment 403 may correspond to a cloud provider network 203 ( Figure 2A ), where customers are charged based on their computing resource usage based on a utility computing model.

[0090] In some embodiments, computing environment 403 may correspond to a virtualized private network within a physical network, including, for example, virtual machine instances executing on physical computing hardware via a hypervisor. The virtual machine instances and any containers running on these instances may obtain network connectivity through virtualized network components enabled by physical network components, such as routers and switches.

[0091] According to various embodiments, various applications and / or other functions may be executed in the computing environment 403. In addition, various data is stored in a data store 415 accessible to the computing environment 403. It will be appreciated that the data store 415 may represent a plurality of data stores 415. The data stored in the data store 415 is associated with, for example, the operation of the various applications and / or functional entities described below.

[0092] The computing environment 403, which is part of a cloud provider network that provides utility computing services, includes computing devices 418 and other types of computing devices. The computing devices 418 may correspond to different types of computing devices 418 and may have different computing architectures. The computing architecture may differ by using processors with different architectures, such as x86, x86_64, ARM, Scalable Processor Architecture (SPARC), PowerPC, etc. For example, some computing devices 418 may have x86 processors, while other computing devices 418 may have ARM processors. The computing devices 418 may also differ in available hardware resources, such as local storage, graphics processing units (GPUs), machine learning extensions, and other features.

[0093] Computing devices 418 may have various forms of assigned computing capacity 421, which may include virtual machine (VM) instances, containers, serverless functions, and the like. VM instances may be instantiated from VM images. To this end, a customer may specify that a virtual machine instance should be launched on a particular type of computing device 418, but not on other types of computing devices 418. In various examples, a VM instance may execute solely on a particular computing device 418, or multiple VM instances may execute on a particular computing device 418. Furthermore, a particular computing device 418 may execute different types of VM instances, which may provide different amounts of available resources via the computing device 418. For example, certain types of VM instances may provide more memory and processing power than other types of VM instances.

[0094] Components executing on computing environment 403 include, for example, network management API 423, network management service 424, and other applications, services, processes, systems, engines, or functionality not discussed in detail herein.

[0095] The network management API 423 provides an interface for creating, deploying, activating, managing, updating, deactivating, and deleting access networks 103. The network management API 423 may also include functions for creating, updating, and deleting network functions 163 ( Figure 1C ) and define priorities relative to client devices 106 or other user equipment. In various embodiments, the network management API 423 may implement interfaces for creating a site or location, describing a site, deleting a site, updating a site, creating a network plan, describing a network plan, updating a network plan, deleting a network plan, creating a network, describing a network, listing networks associated with a customer, deleting a network, activating a network, creating a data plan, describing a data plan, listing data plans for a network, attaching a SIM or eSIM to a data plan, deleting a data plan, listing SIMs or eSIMs associated with a network, configuring small cell or radio unit installations individually or in batches, and other functionality.

[0096] The network management service 424 is executed to manage, configure, and monitor the access networks 103 operated by the cloud service provider on behalf of customers. To this end, the network management service 424 may generate multiple user interfaces that allow customers to order new access networks 103, scale up or down existing access networks 103, modify the operation of existing access networks 103, apply capacity constraint planning to network functions 163, configure client devices 106 permitted to use the access networks 103, define priorities for client devices 106, provide statistics and metrics regarding the operation of the access networks 103, reserve spectrum for customers' private networks via the spectrum reservation service 410, and so on. For example, the network management service 424 may generate one or more network pages, such as web pages, that include the user interfaces. Furthermore, the network management service 424 may support this functionality through an API callable by client applications 436. The network management service 424 may use the network management API 423 to implement various actions. In addition to facilitating interaction with users, the network management service 424 also enables the orchestration of deployment and configuration changes of the access networks 103 and the ongoing monitoring of performance parameters. For a particular site 438 , network management service 424 may generate a network plan 439 for the customer based at least in part on specifications of the customer's location in site 438 , automated site surveys by unmanned aerial vehicles, and / or other input parameters.

[0097] The network management service 424 can also implement provisioning and configuration changes on the hardware that implements the access network 103. The hardware may include radios, antennas, VM instances or containers that perform network functions, routers, switches, fiber optic terminal equipment, and the like. For example, an antenna may be configured to operate at a specific frequency. A radio may be programmed to operate at a specific frequency, join a specific access network 103, and backhaul traffic to a specific VM instance or container.

[0098] In some scenarios, the network management service 424 is executed to reconfigure hardware that is already present in the access network 103. In other scenarios, the network management service 424 is executed to preconfigure a set of hardware to be deployed to an existing or new access network 103. To this end, the network management service 424 may implement configuration on one or more pre-deployment devices 409 that are temporarily connected to the network 412 to facilitate pre-configuration before the pre-deployment devices 409 are shipped to a customer for deployment in the access network 103.

[0099] The network management service 424 can also automate and arrange the deployment of hardware to implement the access network 103. Based on a network plan 439 submitted by or generated for a customer, the network management service 424 can arrange the procurement of hardware components necessary to implement the access network 103 from one or more suppliers associated with the supplier computing device in accordance with the network plan 439. This can involve automatically placing orders with suppliers for new equipment, reserving equipment already in the supplier's inventory, or reassigning equipment already at the customer site or at another customer site where the reassigned equipment is no longer in use. In this regard, the network management service 424 can send instructions to the customer to return equipment that is no longer in use, whereupon the equipment can be directly sent to another customer for use in another deployment. In another scenario, the network management service 424 can send instructions to the customer to move a piece of equipment from one site where it is no longer in use to another site where it will be used. The network management service 424 can manage the connection of equipment to the network 412 to pre-configure it as a pre-deployed device 409. Network management service 424 may also arrange for delivery of equipment to a customer location, including potentially multiple customer locations corresponding to various cell sites.

[0100] The data groups stored in the data storage area 415 include, for example, one or more sites 438, one or more network plans 439, one or more data plans 440, one or more cellular topologies 442, one or more spectrum allocations 445, device data 448, one or more priority groups 450, one or more capacity limitation rule sets 452, data describing one or more network slices 454, radio unit configuration data 457, network function configuration data 453, and possibly other data.

[0101] Site 438 represents the specification of the location where access network 103 will be deployed for a customer. In one implementation, site 438 is created by network management API 423 by specifying a site name and address. In other examples, latitude and longitude coordinates may be used. Network management API 423 may associate a unique site identifier with site 438.

[0102] The network plan 439 is a specification of the access network 103 to be deployed for the customer. In various implementations, the network plan 439 may include one or more identifiers of sites 438, the location, position, or geographic area to be covered, the number of cells, the maximum number of client devices 106 per cell, the number of client devices 106 or SIMs, device identification information and licensing, a desired measure of edge computing capacity in the access network 103, desired maximum network latency, desired bandwidth or network throughput for one or more classes of devices, one or more quality of service parameters for applications or services, network address ranges to be allocated to client devices 106, an identifier of the type of spectrum to be used (e.g., Citizens Broadband Radio Service, TV White Spaces, licensed spectrum, etc.), and / or other parameters that may be used to create the access network 103. The customer may manually specify one or more of these parameters via a user interface or API. One or more parameters may be pre-populated with default parameters. In some cases, the network plan 439 may be generated for the customer based at least in part on an automated site survey using an unmanned aerial vehicle. In some cases, network plan 439 may include thresholds and reference parameters determined at least in part based on automatic exploration of the customer's existing private network.

[0103] Data plan 440 may correspond to one or more plans for accessing access network 103 from client device 106. Data plan 440 may include a network identifier, a data plan name, an uplink speed, a downlink speed, a unique identifier for data plan 440, latency requirements, jitter requirements, and / or other data. The parameter values defining data plan 440 may be used as a basis for billing customers by the cloud service provider under a utility computing model. For example, in a service level agreement (SLA), customers may be charged more for lower latency targets and / or higher bandwidth targets, and customers may be charged per device, per cell, based on the geographic area served, based on spectrum availability, and the like.

[0104] Cellular topology 442 includes an arrangement of multiple cells for customers that takes into account spectrum reuse given the location of the cells. Cellular topology 442 can be automatically generated given a site survey. In some cases, the number of cells in cellular topology 442 can be automatically determined based on the desired geographic area to be covered, the availability of backhaul connectivity at each site, signal propagation, available spectrum, and / or other parameters. For access network 103, cellular topology 442 can be developed to cover one or more buildings on an organizational campus, one or more schools in a school district, one or more buildings in a university or university system, and other areas.

[0105] Spectrum allocation 445 includes spectrum available for assignment to access network 103 and spectrum currently assigned to access network 103. Spectrum can include spectrum that is publicly accessible without restriction, spectrum that is personally owned or leased by customers, spectrum that is owned or leased by providers, spectrum that is free to use but requires a subscription, and so on.

[0106] Device data 448 corresponds to data describing the client devices 106 that are permitted to connect to the access network 103. The client device 106 may be associated with a particular account in the cloud provider network 203. This device data 448 includes the corresponding user, account information, billing information, a data plan 440, permitted applications or uses, an indication of whether the client device 106 is mobile or fixed, a location, a current cell, a network address, a device identifier 464 (e.g., an International Mobile Equipment Identity (IMEI) number, an International Mobile Subscriber Identity (IMSI) number, a device serial number (ESN), a Media Access Control (MAC) address, a Subscriber Identity Module (SIM) number, an Embedded SIM (eSIM) number, etc.), a device priority 465 for capacity constraint planning purposes, and the like.

[0107] Priority groups 450 may define one or more groups or categories of client devices 106 that have a common priority for the purposes of capacity limit planning. In other words, capacity limit rule sets 452 may treat client devices 106 as identical when they are within the same priority group 450. For example, types of client devices 106 (e.g., security cameras) may be in the same priority group 450, while client devices 106 associated with user categories (e.g., corporate executives) may be in the same priority group 450.

[0108] The capacity limit rule set 452 includes one or more customized rules that control access to the network function 163 when a capacity limit in the network function 163 is reached. As used herein, a capacity limit can be an absolute limit, above which the service cannot be provided, or a capacity limit can constitute a threshold, above which the service is associated with unacceptable service characteristics (such as unacceptable latency or unacceptable reliability). The capacity limit rule set 452 can be customer-specific or associated with a specific account of the cloud provider network 203. Different network functions 163 can be associated with different capacity limit rule sets 452. The capacity limit rule set 452 can replace or substitute one or more default rules that would otherwise be applied when a capacity limit is reached in the network function 163. The capacity limit rule set 452 can control which client devices 106 are disconnected or what type of network traffic is dropped in order to provide network access to higher priority client devices 106 and / or higher priority network traffic.

[0109] In some scenarios, the capacity limit rule set 452 may provide conditions under which the network function 163 should be scaled up to increase the capacity limit or scaled down to reduce the capacity limit. Scaling may include adding computing resources (e.g., the number of machine instances, processor capacity, memory capacity, network bandwidth, etc.) to the network function 163. For example, when a high-priority device requests service, there may not be currently any lower-priority devices receiving service from the network function 163. The solution specified by the capacity limit rule set 452 may be to scale up the network function 163 to add additional capacity, rather than suspending service to any client device 106. Conversely, when current utilization is well below the capacity limit, the capacity limit rule set 452 may provide for scaling down the network function 163 to reduce capacity, which may correspondingly reduce costs.

[0110] A network slice 454 corresponds to a network traffic flow that has been specified for one or more specific quality of service requirements 466. The flow may correspond to a flow associated with a specific application executing on a specific client device 106, all network traffic from a specific client device 106, a flow from all client devices 106 to a specific destination, a flow from a specific client device 106 to a specific destination, and so on. In one example, a network slice 454 is identified by a source port, a source network address, a destination port, a destination network address, and / or other information. A network slice 454 may be valid for a specific time period or a specific amount of data, or it may be valid until it is canceled or released. In one example, a network slice 454 is assigned on demand for a specific application executing on a client device 106. In some scenarios, a network slice 454 has a specific, recurring validity period (e.g., every weekday evening from midnight to 5:00 AM), or the quality of service requirements 466 for a network slice 454 may change based on a recurring period, current cost levels, and / or other factors or events.

[0111] The quality of service requirements 466 may correspond to minimum or maximum bandwidth, minimum or maximum latency, minimum or maximum reliability metrics, minimum or maximum signal strength, and the like. The quality of service requirements 466 may be associated with corresponding cost levels, which may include fixed components, usage-based components, and / or congestion-based components. For example, the quality of service requirements 466 may be associated with a recurring monthly fixed cost, a per-session or per-megabyte cost, and / or a dynamic cost based on congestion at a cell site or a particular network link. In some cases, a customer may select a quality of service requirement 466 that provides a high level of service. However, in other cases, a customer may select a quality of service requirement 466 that provides a low cost level but reduces the quality of service during certain times or in certain aspects. For example, a customer may select a quality of service requirement 466 that allows high throughput overnight and other lower priority throughput to send backup data over the network at a low cost.

[0112] Radio unit configuration data 457 may correspond to configuration settings for radio units deployed in access network 103. Such settings may include the frequency to use, the protocol to use, modulation parameters, bandwidth, network routing and / or backhaul configuration, location and altitude of the radio unit, capacity limit rule set 452, and the like.

[0113] The network function configuration data 463 corresponds to configuration settings that configure the operation of various network functions 163 for the access network 103. For example, the network function configuration data 463 may include a capacity limit rule set 452 related to a particular network function 163. In various embodiments, the network function 163 may be deployed in a VM instance or container located in a computing device 418, which may be located at a cell site, at a customer aggregation site, or in a data center remote from the customer. Non-limiting examples of the network function 163 may include an access and mobility management function, a session management function, a user plane function, a policy control function, an authentication server function, a unified data management function, an application function, a network exposure function, a network function repository, a network slice selection function, and / or other functions.

[0114] Client devices 406 represent a plurality of client devices 406 that can be coupled to network 412. Client devices 406 may include, for example, processor-based systems, such as computer systems. Such computer systems may be embodied in the form of desktop computers, laptop computers, personal digital assistants, cellular phones, smartphones, set-top boxes, music players, internet pads, tablet computer systems, gaming consoles, e-book readers, smart watches, head-mounted displays, voice interface devices, or other devices. Client devices 406 may include displays comprising, for example, one or more devices such as liquid crystal displays (LCDs), gas plasma-based flat-panel displays, organic light-emitting diode (OLED) displays, electrophoretic ink (E-ink) displays, LCD projectors, or other types of display devices.

[0115] The client device 406 can be configured to execute various applications, such as a client application 436 and / or other applications. The client application 436 can be executed on the client device 406, for example, to access network content provided by the computing environment 403 and / or other servers, thereby presenting a user interface on a display. To this end, the client application 436 can include, for example, a browser, a dedicated application, etc., and the user interface can include a web page, an application screen, etc. The client device 406 can be configured to execute applications other than the client application 436, such as, for example, an email application, a social networking application, a word processor, a spreadsheet, and / or other applications.

[0116] In some embodiments, spectrum reservation service 410 provides spectrum reservation for a customer's private network. In one scenario, spectrum reservation service 410 is operated by an entity, such as a third party, to manage reservations and coexistence in publicly accessible spectrum. An example of such spectrum is Citizens Broadband Radio Service (CBRS). In another scenario, spectrum reservation service 410 is operated by a telecommunications service provider to sell or sublicense portions of spectrum owned or licensed by the provider.

[0117] Next reference Figure 5 , shows a flow chart of one example of operations for providing a portion of the network management service 424 according to various embodiments. It will be appreciated that Figure 5 The flowchart of FIG. 4 provides only an example of the many different types of functional arrangements that may be used to implement the operation of portions of the network management service 424 as described herein. Figure 5 The flowchart of can be viewed as depicting a process in a computing environment 403 ( Figure 4 ) is an example of an element that implements a method in .

[0118] Beginning at block 503, the network management service 424 operates or manages the access network 103 ( Figure 4 ) of the client receiving client device 106 ( Figure 1A ) and its corresponding device priority 465 ( Figure 4 ) specification. The specification may indicate the corresponding device identifier 464 ( Figure 4 ) and / or other information about the client device 106. For example, the specification may associate the client device 106 with the priority group 450 ( Figure 4 ) are associated, and the priority groups are in turn associated with specific priority levels. In one embodiment, priority is defined as a numerical value, where a larger numerical value corresponds to a larger relative priority.

[0119] In block 506, the network management service 424 receives a capacity limit rule set 452 ( Figure 4 For example, the capacity limit rule set 452 may define a specification for processing network function 163 ( Figure 1C) in a capacity limiting rule set 452, which may override or replace the default rules for handling capacity limiting in the network function 163. That is, instead of randomly failing or disconnecting, the capacity limiting rule set 452 is used to configure the network function 163 to fail or deny service in a predictable and reliable manner while still providing access to high-priority client devices 106. In addition, the capacity limiting rule set 452 may include customized algorithms or methods for selecting which client devices 106 will be allowed to connect or continue service. Such methods may include first-in, first-out (FIFO), last-in, first-out (LIFO), round-robin, random distribution, and / or other methods. These methods are particularly useful when client devices 106 have the same priority level.

[0120] In block 509, the network management service 424 configures the network function set (i.e., one or more network functions 163) to implement the corresponding rules in the capacity limit rule set 452 instead of the one or more default rules. For example, the network management service 424 may push a new configuration file implementing the capacity limit rule set 452 to the corresponding network function 163 via the control plane. The network management service 424 may also cause the network functions 163 to restart or otherwise reload their configuration settings. Thereafter, the operation of the portion of the network management service 424 ends.

[0121] Now go to Figure 6 , showing providing network functionality 163 ( Figure 1C ) is a flowchart of an example of a portion of the operation. It should be understood that Figure 6 The flowchart of FIG. 1 merely provides an example of many different types of functional arrangements that may be employed to implement the operation of portions of network functions 163 as described herein. As an alternative, Figure 6 The flowchart of can be viewed as depicting a process in a computing environment 403 ( Figure 4 ) is an example of an element that implements a method in .

[0122] Beginning at block 603, the network function 163 receives a request from the first client device 106 ( Figure 1A ) receives a service request for a service. In one example, the network function 163 is located in the radio access network of the access network 103. In another example, the network function 163 is located in the core network of the access network 103. The network function 163 may be hosted by the cloud provider network 203 ( Figure 2A) on resources managed or provided on behalf of the customer, or network functions 163 may be hosted on resources within the customer's premises. In some cases, the service request may correspond to an explicit connection request or a request for a network address. In other cases, the connection request may simply correspond to using access network 103 by client device 106 sending one or more data packets through access network 103.

[0123] In block 606, the network function 163 determines that the network function 163 is at capacity limit. For example, based on a measure of resources assigned to the network function 163, the network function 163 may have a strict limit (e.g., 64 or another number) on the number of connections or client devices 106 that can be served concurrently. In some cases, the limit may be based on licensing restrictions. In other examples, the limit may correspond to a bandwidth limit, a memory limit, a processor limit, etc. Alternatively, the network function 163 may determine that the network function 163 is approaching capacity limit within a defined threshold so as to take action to avoid reaching the capacity limit.

[0124] In block 609, the network function 163 determines the device priority 465 ( Figure 4 In some cases, the network function 163 may determine a priority group 450 ( Figure 4 ) associated with the priority. For example, the network function 163 may query the database for the device identifier 464 ( Figure 4 ) to determine the device priority 465. Alternatively, the device priority 465 may be included in the connection request or data packet sent by the first client device 106.

[0125] In block 612, the network function 163 may select one or more second client devices 106 currently using the network function 163 to interrupt access. In this example, the first client device 106 has a higher priority than the one or more second client devices 106. In other examples, the first client device 106 may have a lower priority and may be denied access. In the event that multiple second client devices 106 have a lower priority than the first client device 106, the network function 163 may select as few second client devices 106 as necessary to accommodate the first client device 106. Figure 4 ) to select the second client device 106, the capacity limitation rule set may specify random selection, round-robin selection, FIFO selection, LIFO selection, or another method.

[0126] In block 615, network function 163 suspends service to the selected second client device 106 based at least in part on capacity limitation rule set 452. For example, network function 163 may pre-pair service for first client device 106 but not pre-pair service for second client device 106. This may disconnect the selected second client device 106 from access network 103, or the selected second client device 106 may be accommodated by another instance of network function 163. Alternatively, functionality in access network 103 may be limited by suspending service from network function 163, but some network connectivity may still be available.

[0127] In block 618, the network function 163 provides services to the first client device 106, which may enable connectivity between the first client device 106 and the access network 103. The priority of the first client device 106 may continue to be considered across scheduling, admission control, and resource allocation algorithms. In block 621, the network function 163 may dynamically assign the network slice 454 ( Figure 4 ) can be assigned to the first client device 106. For example, a client device with a corresponding QoS requirement 466 ( Figure 4 ) is assigned to the first client device 106. However, it should be noted that the QoS requirement 466 can be separate from the priority. In one example, the client device 106 can have a relatively low priority but a relatively high QoS requirement 466, or the client device 106 can have a relatively high priority but a relatively low QoS requirement.

[0128] In block 624, network function 163 may dynamically adjust other network slices 454. For example, if the selected second client device 106 that interrupts access to network function 163 is assigned one or more network slices 454, resources assigned to network slices 454 in access network 103 may be released and reallocated to network slices 454 currently being used by client devices 106 currently connected via network function 163. In some cases, network function 163 may dynamically adjust QoS requirements 466 of network slices 454 based at least in part on priority. In one example, QoS requirements 466 associated with lower-priority client devices 106 are adjusted to meet QoS requirements 466 of higher-priority client devices 106. By utilizing device priority 465 and QoS requirements 466, these techniques will not only allow higher-priority client devices 106 to access access network 103, but will also ensure that higher-priority client devices 106 receive the resources 466 they require based on their QoS requirements.

[0129] In block 627, the capacity of the network function 163 may be scaled up or down based at least in part on the capacity limit rule set 452. For example, if the current capacity is insufficient to accommodate the network slice 454, then capacity may be increased. Similarly, if the relative priorities and the capacity limit rule set 452 do not allow for suspending service to existing client devices 106, then capacity may be increased. Conversely, the capacity limit rule set 452 may provide for scaling down capacity if utilization falls below a capacity limit threshold amount. When scaling the capacity of the network function 163, new machine instances may be launched and assigned to the network function 163 in the cloud provider network 203. Alternatively, existing compute capacity located at the edge or in the core network may be reassigned to the network function 163 instead of the customer workload or a different network function 163. Thereafter, operation of the portion of the network function 163 ends.

[0130] refer to Figure 7 , shows a schematic block diagram of a computing environment 403 according to an embodiment of the present disclosure. The computing environment 403 includes one or more computing devices 700. Each computing device 700 includes at least one processor circuit, for example, having a processor 703 and a memory 706, both of which are coupled to a local interface 709. To this end, each computing device 700 may include, for example, at least one server computer or similar device. The local interface 709 may include, for example, a data bus with an accompanying address / control bus or other bus structure as will be appreciated.

[0131] Data and several components executable by processor 703 are stored in memory 706. Specifically, network management API 423, network management service 424, and possibly other applications are stored in memory 706 and are executable by processor 703. Data storage area 415 and other data may also be stored in memory 706. In addition, an operating system may be stored in memory 706 and may be executed by processor 703.

[0132] It should be understood that there may be other applications stored in the memory 706 and executable by the processor 703, as is known. Where any component discussed herein is implemented in software, any of a variety of programming languages may be employed, such as, for example, C, C++, C#, Objective C, Perl, PHP, Visual Ruby, or other programming languages.

[0133] Many software components are stored in memory 706 and are executable by processor 703. In this regard, the term "executable" refers to a program file in a form that can ultimately be run by processor 703. Examples of executable programs can be, for example, compiled programs that can be translated into machine code in a format that can be loaded into a random access portion of memory 706 and executed by processor 703; source code that can be expressed in a suitable format, such as object code that can be loaded into a random access portion of memory 706 and executed by processor 703; or source code that can be interpreted by another executable program to generate instructions in a random access portion of memory 706 to be executed by processor 703, etc. The executable program can be stored in any part or component of memory 706, including, for example, random access memory (RAM), read-only memory (ROM), a hard drive, a solid-state drive, a USB flash drive, a memory card, an optical disk such as a compact disk (CD) or a digital versatile disk (DVD), a floppy disk, a magnetic tape, or other storage component.

[0134] Memory 706 is defined herein as including volatile and non-volatile memory and data storage components. Volatile components are those that do not retain data values when power is removed. Non-volatile components are those that retain data when power is removed. Thus, memory 706 may include, for example, random access memory (RAM), read-only memory (ROM), a hard drive, a solid-state drive, a USB flash drive, a memory card accessed via a memory card reader, a floppy disk accessed via an associated floppy disk drive, an optical disk accessed via an optical drive, a magnetic tape accessed via an appropriate tape drive, and / or other memory components, or a combination of any two or more of these memory components. In addition, RAM may include, for example, static random access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM), and other such devices. ROM may include, for example, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other similar memory devices.

[0135] Furthermore, the processor 703 may represent multiple processors 703 and / or multiple processor cores, and the memory 706 may represent multiple memories 706 each operating in parallel processing circuits. In this case, the local interface 709 may be a suitable network that facilitates communication between any two of the multiple processors 703, between any processor 703 and the memory 706, or between any two of the memories 706. The local interface 709 may include additional systems designed to coordinate this communication, including, for example, performing load balancing. The processor 703 may be electrical or some other usable structure.

[0136] While the network management API 423, network management services 424, and various other systems described herein may be embodied in software or code executed by general-purpose hardware as discussed above, they may alternatively be embodied in dedicated hardware or a combination of software / general-purpose hardware and dedicated hardware. If embodied in dedicated hardware, each may be implemented as a circuit or state machine using any one or a combination of a variety of technologies. These technologies may include, but are not limited to, discrete logic circuits having logic gates for implementing various logic functions when one or more data signals are applied, application-specific integrated circuits (ASICs) having appropriate logic gates, field-programmable gate arrays (FPGAs), or other components. Such technologies are generally well known to those skilled in the art and, therefore, will not be described in detail herein.

[0137] Figures 5 and 6 The flowcharts illustrate the functionality and operation of an implementation of portions of network management service 424 and network functions 163. If embodied in software, each block may represent a module, segment, or portion of code that includes program instructions for implementing a specified logical function. The program instructions may be embodied in the form of source code, which includes human-readable statements written in a programming language, or in the form of machine code, which includes digital instructions recognizable by a suitable execution system, such as processor 703 in a computer system or other system. Machine code may be converted from source code, etc. If embodied in hardware, each block may represent a circuit or multiple interconnected circuits to implement a specified logical function.

[0138] although Figures 5 and 6 The flowcharts of the embodiment of the present invention show a specific order of execution, but it should be understood that the order of execution may be different from that depicted. For example, the order of execution of two or more blocks may be scrambled relative to the order shown. In addition, Figures 5 and 6 Two or more blocks shown in succession in FIG may be executed concurrently or partially concurrently. Furthermore, in some embodiments, Figures 5 and 6 One or more of the blocks shown may be skipped or omitted. In addition, any number of counters, state variables, warning semaphores, or messages may be added to the logic flow described herein for purposes such as enhancing utility, accounting, performance measurement, or providing troubleshooting assistance. It should be understood that all such variations are within the scope of this disclosure.

[0139] Furthermore, any logic or application comprising software or code described herein (including network management API 423 and network management service 424) may be embodied in any non-transitory computer-readable medium for use by or in conjunction with an instruction execution system (such as, for example, processor 703 in a computer system or other system). In this sense, logic may include, for example, statements comprising instructions and declarations that can be retrieved from a computer-readable medium and executed by an instruction execution system. In the context of the present disclosure, a "computer-readable medium" may be any medium that can contain, store, or maintain the logic or application described herein for use by or in conjunction with an instruction execution system.

[0140] Computer-readable media can include any of a number of physical media, such as, for example, magnetic, optical, or semiconductor media. More specific examples of suitable computer-readable media would include, but are not limited to, magnetic tape, magnetic floppy disks, magnetic hard drives, memory cards, solid-state drives, USB flash drives, or optical disks. Additionally, the computer-readable medium can be a random access memory (RAM), including, for example, static random access memory (SRAM) and dynamic random access memory (DRAM), or a magnetic random access memory (MRAM). Additionally, the computer-readable medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other types of memory devices.

[0141] Furthermore, any logic or application described herein (including network management API 423 and network management service 424) can be implemented and structured in a variety of ways. For example, one or more of the applications described herein can be implemented as modules or components of a single application. Furthermore, one or more of the applications described herein can be executed on shared or separate computing devices, or a combination thereof. For example, multiple applications described herein can be executed on the same computing device 700, or on multiple computing devices 700 in the same computing environment 403.

[0142] Unless specifically stated otherwise, disjunctive language such as the phrase "at least one of X, Y, or Z" should be understood in context as generally used to present that an item, term, etc. can be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended to, and should not, imply that certain embodiments require the presence of at least one of X, at least one of Y, or at least one of Z, respectively.

[0143] Embodiments of the present disclosure may be described at least in terms of:

[0144] Clause 1. A system comprising: a radio-based network operated by a cloud provider network on behalf of a customer, the radio-based network including a plurality of network functions; and at least one computing device in the cloud provider network, the at least one computing device configured to at least perform the following operations: receive a request from the customer to, in response to at least one network function in a network function set of the plurality of network functions being at a capacity limit, provision a service from the network function set to a first client device rather than provisioning the service to a second client device in accordance with at least one rule defined by the customer, the capacity limit being an absolute capacity limit or a threshold above which the service is associated with unacceptable characteristics; and configure the network function set in the radio-based network to implement the at least one rule rather than a default rule for handling the capacity limit.

[0145] Clause 2. The system of clause 1, wherein the default rule comprises at least one of a round-robin algorithm or a first-in-first-out algorithm.

[0146] Clause 3. The system of any of clauses 1 to 2, wherein the request associates a first priority level with the first client device and a second priority level with the second client device, and the first priority level is higher than the second priority level.

[0147] Clause 4. The system of Clause 3, wherein the first priority level applies to a first class of client devices and the second priority level applies to a second class of client devices.

[0148] Clause 5. A system as described in any of clauses 1 to 4, wherein the network function set is configured to implement the at least one rule rather than the default rule for handling the capacity limitation, causing the network function set to provide network service to the first client device and suspend the network service to the second client device in accordance with the at least one rule, rather than denying the network service to the first client device in accordance with the default rule.

[0149] Clause 6. A system as described in any of clauses 1 to 5, wherein configuring the network function set to implement the at least one rule rather than the default rule for handling the capacity limitation causes the network function set to dynamically allocate a network slice with a quality of service requirement to the first client device.

[0150] Clause 7. A computer-implemented method comprising: receiving a request for service from a radio-based network from a first client device; determining that a network function in the radio-based network is at capacity limitation; suspending the service from the network function to a second client device in response to determining that the network function in the radio-based network is at capacity limitation and based at least in part on a set of rules specific to the radio-based network; and providing access to the network function to the first client device instead of the second client device.

[0151] Clause 8. The computer-implemented method of Clause 7, further comprising: determining a priority associated with the first client device; and determining, based at least in part on the priority, to provide the service from the network function to the first client device rather than the second client device.

[0152] Clause 9. The computer-implemented method of Clause 8, further comprising receiving a specification of the priority associated with the first client device from an operator of the radio-based network.

[0153] Clause 10. A computer-implemented method as described in any of clauses 7 to 9, further comprising: dynamically assigning the first client device to a network slice in the radio-based network based at least in part on the rule set, the network slice being associated with a quality of service requirement.

[0154] Clause 11. The computer-implemented method of any of Clauses 7 to 10, further comprising selecting the second client device from a plurality of client devices currently using the network function based at least in part on the rule set.

[0155] Clause 12. The computer-implemented method of Clause 11, wherein the rule set defines rules for handling the capacity limitation in the network function that are different from default rules for the network function.

[0156] Clause 13. The computer-implemented method of Clause 12, wherein the default rule denies the first client device the service from the network function.

[0157] Clause 14. A computer-implemented method as described in any of clauses 7 to 13, further comprising: scaling computing resources assigned to the network function based at least in part on the rule set and in response to determining that the network function in the radio-based network is at the capacity limit.

[0158] Clause 15. The computer-implemented method of any of clauses 7 to 14, wherein the radio-based network is provisioned for a customer by a cloud provider network, and the rule set is configured by the customer.

[0159] Clause 16. The computer-implemented method of any of Clauses 7 to 15, wherein network traffic from the second client device is associated with a higher quality of service parameter than network traffic from the first client device.

[0160] Clause 17. A non-transitory computer-readable medium storing instructions capable of being executed in at least one computing device, wherein the instructions, when executed, cause the at least one computing device to at least perform the following operations: receive a capacity constraint plan from an operator of a radio-based network, the capacity constraint plan including operator-specified rules that prioritize service from a network function to a first client device over a second client device in response to the network function being capacity constrained; and configure the set of network functions in the radio-based network to implement the operator-specified rules rather than default rules for handling the capacity constraint.

[0161] Clause 18. The non-transitory computer-readable medium of Clause 17, wherein the operator-specified rule defines a method of selecting a second client device from a plurality of second client devices receiving the service from the network function to suspend the service from the network function.

[0162] Clause 19. A non-transitory computer-readable medium as described in any of clauses 17 to 18, wherein the instructions, when executed, further cause the at least one computing device to perform at least the following operations: configure the network function to dynamically allocate a network slice to the first client device based at least in part on rules specified by the operator.

[0163] Clause 20. The non-transitory computer-readable medium of any one of clauses 17 to 19, wherein the radio-based network comprises a radio access network, the network function is implemented in the radio access network, and the capacity limit corresponds to a maximum number of client devices concurrently served by the network function.

[0164] It should be emphasized that the above-described embodiments of the present disclosure are merely examples of possible implementations for a clear understanding of the principles of the present disclosure. Many variations and modifications may be made to the above-described embodiments without departing substantially from the spirit and principles of the present disclosure. All such modifications and variations are intended to be included within the scope of the present disclosure and are protected by the appended claims.

Claims

1. A system comprising: a radio-based network operated by a cloud provider network on behalf of a customer, the radio-based network comprising a plurality of network functions; as well as At least one computing device in the cloud provider network, the at least one computing device being configured to perform at least the following operations: providing a service from the set of network functions to the first client device and denying the service to the second client device in accordance with at least one rule defined by the customer that establishes a priority for the first client device relative to a second client device, in response to at least one network function in the set of network functions being at a client device capacity limit, the client device capacity limit being an absolute capacity limit of the service beyond which the service cannot be provided to additional client devices; as well as A set of network functions in the radio-based network is configured to implement the at least one rule defined by the customer instead of a default rule for handling the absolute capacity limitation. 2 . The system of claim 1 , wherein the default rule comprises at least one of a round-robin algorithm or a first-in-first-out algorithm. 3 . The system of claim 1 , wherein a first priority level is associated with the first client device, a second priority level is associated with the second client device, and the first priority level is higher than the second priority level.

4. The system of claim 3, wherein the first priority level applies to a first class of client devices and the second priority level applies to a second class of client devices.

5. The system of claim 1 , wherein the network functionality set is configured to implement the at least one rule instead of a default rule for handling the client device capacity limitation, such that the network functionality set provides network service to the first client device and suspends network service to the second client device in accordance with the at least one rule instead of denying network service to the first client device in accordance with the default rule.

6. The system of claim 1 , wherein the network function set is configured to implement the at least one rule instead of the default rule for handling the absolute capacity limit, causing the network function set to dynamically allocate a network slice with a quality of service requirement to the first client device.

7. The system of claim 1, wherein the radio-based network comprises a radio access network in which the network functions are implemented, and the absolute capacity limit corresponds to a maximum number of client devices concurrently served by the service.

8. The system of claim 1, wherein network traffic from the first client device is associated with a higher quality of service parameter than network traffic from the second client device.

9. A computer-implemented method comprising: operating a radio-based network on behalf of a customer by a cloud provider network, the radio-based network comprising a plurality of network functions; providing a service from the set of network functions to the first client device and denying provision of the service to the second client device in accordance with at least one rule defined by the customer that establishes a priority for the first client device relative to a second client device, in response to at least one network function in the set of network functions being at a client device capacity limit, the client device capacity limit being an absolute capacity limit of the service beyond which the service cannot be provided to additional client devices; as well as A set of network functions in the radio-based network is configured to implement the at least one rule defined by the customer instead of a default rule for handling the absolute capacity limitation.

10. The computer-implemented method of claim 9, wherein the default rule comprises at least one of a round-robin algorithm or a first-in-first-out algorithm.

11. The computer-implemented method of claim 9, wherein a first priority level is associated with the first client device and a second priority level is associated with the second client device, and the first priority level is higher than the second priority level.

12. The computer-implemented method of claim 11, wherein the first priority level applies to a first class of client devices and the second priority level applies to a second class of client devices.

13. The computer-implemented method of claim 9 , wherein the network functionality set is configured to implement the at least one rule instead of a default rule for handling the client device capacity limitation, causing the network functionality set to provide network service to the first client device and suspend network service to the second client device in accordance with the at least one rule instead of denying network service to the first client device in accordance with the default rule.

14. A computer-implemented method as described in claim 9, wherein configuring the network function set to implement the at least one rule instead of the default rule for handling the absolute capacity limit causes the network function set to dynamically allocate a network slice with a quality of service requirement to the first client device.

15. A non-transitory computer-readable medium storing instructions executable in at least one computing device, wherein the instructions, when executed, cause the at least one computing device to perform at least the following operations: providing a service from a plurality of network functions in a radio-based network to the first client device and denying provision of the service to the second client device in response to at least one network function in the set of network functions being at a client device capacity limit, according to at least one rule defined by a customer that establishes a priority for the first client device relative to a second client device, the client device capacity limit being an absolute capacity limit for the service beyond which the service cannot be provided to additional client devices, the radio-based network being operated by a cloud provider network on behalf of the customer; A set of network functions in the radio-based network is configured to implement the at least one rule defined by the customer instead of a default rule for handling the absolute capacity limitation. 16 . The non-transitory computer-readable medium of claim 15 , wherein the default rule comprises at least one of a round-robin algorithm or a first-in-first-out algorithm. 17 . The non-transitory computer-readable medium of claim 15 , wherein a first priority level is associated with the first client device, a second priority level is associated with the second client device, and the first priority level is higher than the second priority level.

18. The non-transitory computer-readable medium of claim 17, wherein the first priority level applies to a first class of client devices and the second priority level applies to a second class of client devices.

19. The non-transitory computer-readable medium of claim 15 , wherein the network functionality set is configured to implement the at least one rule instead of a default rule for handling the client device capacity limitation, causing the network functionality set to provide network service to the first client device and suspend network service to the second client device in accordance with the at least one rule instead of denying network service to the first client device in accordance with the default rule.

20. The non-transitory computer-readable medium of claim 15, wherein the network function set is configured to implement the at least one rule instead of the default rule for handling the absolute capacity limit, causing the network function set to dynamically allocate a network slice having a quality of service requirement to the first client device.

Images