Customer-defined capacity limit plans for communications networks

Abstract

Disclosed are various embodiments for a customer-defined capacity limiting plan in a communications network. In one embodiment, a request for service from a wireless network is received from a first client device. A network function within the wireless network is determined to be capacity limited. Service to a second client device from the network function to the network function is suspended in response to determining that the network function within the wireless network is capacity limited and based at least in part on a rule set specific to the wireless network. The first client device is provided access to the network function on behalf of the second client device.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims priority and the benefit thereof to co - pending U.S. Patent Application No. 17 / 491,128, entitled "CUSTOMER - DEFINED CAPACITY LIMIT PLANS FOR COMMUNICATION NETWORKS," filed on September 30, 2021, which is hereby incorporated by reference in its entirety as shown herein.

Background Art

[0002] 5G is the fifth - generation technical standard for broadband cellular networks and is ultimately planned to replace the fourth - generation (4G) standard of Long - Term Evolution (LTE). 5G technology has come to provide a significantly increased bandwidth, thereby expanding the cellular market beyond smartphones to provide last - mile connections to desktops, set - top boxes, laptops, Internet of Things (IoT) devices, etc. Some 5G cells can use the same frequency spectrum as 4G, while there are also 5G cells that can use the frequency spectrum in the millimeter - wave band. The service area of millimeter - wave band cells is relatively small, but they come to provide much higher throughput than 4G.

Summary of the Invention

[0003] Many aspects of the present disclosure can be better understood with reference to the following drawings. The components of the drawings are not necessarily to scale and instead focus on clearly illustrating the principles of the present disclosure. Further, in the drawings, like reference numerals designate corresponding parts throughout several of the drawings.

Brief Description of the Drawings

[0004]

Figure 1A

Figure 1B

Figure 1C

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

DETAILED DESCRIPTION OF THE INVENTION

[0005] The present disclosure relates to an ability limitation plan for a communication network defined by a customer. All networks are limited in ability in terms of network bandwidth, processing ability, and memory ability. In some cases, those abilities can be over-provisioned or designed so that they are unlikely to reach the limit. In other cases, those abilities can be designed to reach frequently so as to fully utilize what is provisioned. For example, if all client devices are using the network to reduce costs, the network bandwidth can be over-subscribed or under-provisioned relative to the demand.

[0006] Typically, when reaching the ability limitation in a network, each network device behaves in an unpredictable way, such as dropping data packets or restricting access of client devices to the network. Network protocols such as the Transmission Control Protocol (TCP) are designed to adapt to some transmission failures in a best-effort without guarantee. In some cases, the network can enforce a Quality of Service (QoS) guarantee for network slices or individual data flows, but the network can still handle constrained situations in an unpredictable way. For example, data flows can have the same QoS, but the data flows can exceed the bandwidth constraint and the network devices can leave the packets dropped in an unpredictable way. Moreover, standard QoS features do not enable controlling the admission of client devices to a network with excessive ability.

[0007] Various embodiments of the present disclosure facilitate customer specifications for network impairments or capacity limitation plans for their access networks. Such networks may include wireless networks such as 4G and 5G wireless access networks, and portions of the network may be provisioned using cloud provider network infrastructure. Such plans can define a set of rules that control how individual network components handle capacity limitations that they exceed. Network components can include wireless access network components and core network components such as user plane functions (UPF), admission control functions, and other functions. The capacity limitation plan can create relative priorities among different client devices or different types of client devices according to the relative priorities within different types of network traffic. As configured, those priorities can be prioritized over network slices or QoS parameters configured in the event of exceeding capacity limitations. Thus, rather than causing the network to fail in an unpredictable manner, the customer-defined capacity limitation plan enables network components to fail in such a way that a particular client device can still connect to the network and a particular data flow can be transmitted.

[0008] Previous deployments of wireless networks relied on manual deployment and configuration at each step of the process, which has proven to require a significant amount of time and cost. Additionally, in previous generations, software was essentially tied to vendor-specific hardware, preventing customers from deploying alternative software. In contrast, 5G decouples hardware from the software stack, increasing flexibility and enabling the components of wireless networks to be run on the infrastructure of cloud providers. Using a cloud delivery model for wireless networks such as 5G networks can make it easier to process network traffic from hundreds to billions of connected devices and compute-intensive applications while delivering faster, lower latency, and greater capacity than other types of networks.

[0009] Previously, enterprises had to choose between performance and price when evaluating enterprise connectivity solutions. Cellular networks can provide high performance, excellent indoor and outdoor coverage, and advanced Quality of Service (QoS) connectivity features, but private cellular networks can be expensive and complex to manage. Ethernet and Wi-Fi require less initial investment and are easier to manage, but in many cases, enterprises realize that they are less reliable, require a lot of work to obtain optimal coverage, and do not provide QoS features such as guaranteed bitrate, latency, and reliability.

[0010] The disclosed private wireless network service can provide enterprises with the advantages of both the performance, coverage, and QoS of a carrier-grade cellular network and the ease and cost of Wi-Fi-related deployment and operation. The disclosed service can provide appropriate hardware in various form factors that enterprises can deploy at sites, integrated with the software that runs the entire network from small cell sites to Internet breakout. Enterprises can freely deploy various 5G devices and sensors across the enterprise (factory floors, warehouses, lobbies, and communication centers) and manage these devices, register users, and assign QoS from a management console. Using the disclosed technology, customers can assign a throughput of a certain bitrate to all devices (such as cameras, sensors, or IoT devices), assign a highly reliable low-latency connection to devices operating on the factory floor, and assign a broadband connection to all handheld devices. The disclosed service can manage all the software necessary to provide connections that meet specified constraints and requirements. This enables a whole new series of applications with strict QoS requirements or high IoT device density requirements that could not conventionally be run on Wi-Fi networks.

[0011] The disclosed service supports multiple deployment scenarios. In a cloud-only deployment, the service can provide small wireless cells that enterprise customers can deploy on-site, while network functions and other network software are run in the availability zones or edge locations of the nearest (or nearest several) cloud providers. For enterprises that prefer on-premises deployment, the disclosed service provides cloud provider hardware such as the substrate extensions described herein. In this mode, the network and applications remain on the enterprise's premises, and the enterprise can securely store and process data that needs to be kept local (e.g., regulatory compliance, security concerns, etc.). Further, the disclosed service enables any computing and storage not used for running the wireless network to be used for running any local workloads via the same APIs that customers can use to run workloads in the traditional cloud provider realm. Thus, as an advantage, this service allows any excess capacity to be used for local processing, and new hardware and software are provisioned in response to changes in network needs, so enterprises no longer need to worry about overscaling and wasting capacity. Additionally, the disclosed service can provide application development APIs that expose and manage 5G features such as QoS, enabling customers to build applications that can fully utilize the latency and bandwidth capabilities of the network without having to understand the network details.

[0012] In addition, the disclosed service can provide a private zone for running local applications within the cloud provider network. This private zone can be connected to a broader regional zone and effectively become a part of it, enabling customers to manage this private zone using the same APIs and tools as those used in the cloud provider network. Similar to the availability zone, virtual private network subnets can be assigned to the private zone. Using the API, subnets can be created and assigned to all zones that the customer wants to use, including the private zone and other existing zones. The management console may provide a simplified process for creating a private zone. Virtual machine instances and containers can be launched in the private zone in exactly the same way as in the regional zone. Customers can configure a network gateway to define routes, assign IP addresses, set up network address translation (NAT), etc. Using auto-scaling, the capabilities of virtual machine instances or containers can be scaled as needed in the private zone. The same management APIs and authentication APIs as those of the cloud provider network can be used within the private zone. In some cases, cloud services available in the regional zone can be remotely accessed from the private zone via a secure connection, so that these cloud services can be accessed without the need to upgrade or change the local deployment.

[0013] Various embodiments of the present disclosure introduce an approach that enables customers to order and deploy wireless networks and associated core networks in an automated manner. Customers can include enterprises and organizations that desire to set up a wireless network (e.g., a private 5G network) for internal use. Through various user interfaces or APIs, customers can specify network plans or requirements (e.g., the layout of a physical site, device / application types and quantities), and various components necessary to implement the wireless network for the customer can be automatically determined and provisioned. Hardware such as antennas, radios, and computer servers can be preconfigured for the customer's wireless network. The process of installing the preconfigured hardware is mainly plug-and-play, and the wireless network can be activated through a user interface or API. In addition to deploying a wireless network, such as all or part of a new wireless access network, various embodiments of the present disclosure can facilitate the modification and management of the wireless network, including the deployment of preconfigured equipment for additional cells and the assignment of QoS constraints for specific devices or applications on the wireless network.

[0014] Various embodiments of the present disclosure can also incorporate the concepts of elasticity from the cloud computing model and utility computing into wireless networks and related core networks. For example, the disclosed techniques can execute core and radio access network functions, as well as related control plane management functions, on cloud provider infrastructure to create a cloud-native core network and / or a cloud-native radio access network (RAN). Such core and RAN network functions can be based on Third Generation Partnership Project (3GPP®) specifications, depending on the implementation. By providing a cloud-native wireless network, customers can dynamically scale a wireless private network based on usage, latency requirements, and / or other factors. In some cases, the hardware shipped to the customer includes sufficient capabilities to execute both a program for the operation and management of the wireless network and the customer's other workloads (e.g., its applications), and any capabilities not used for the wireless network may be made accessible for executing workloads under a utility computing model. As an advantage, the customer's wireless network can scale up to this excess capacity as needed, enabling, for example, an increase in the hardware usage requirements of the wireless network even before new physical hardware is provisioned for the customer. Also, the customer may configure thresholds to receive alerts regarding wireless network usage and excess capacity usage of the provisioned infrastructure in order to more effectively manage the provisioning of new infrastructure or the de-provisioning of existing infrastructure based on dynamic networking and workload requirements.

[0015] Those skilled in the art will appreciate that, in light of the present disclosure, certain embodiments may have the ability to achieve certain advantages, including, but not limited to: (1) improving the functionality of a computer network by enabling an administrator and an operator to define custom rules to control the operation of network components when the network components are capacity - limited; (2) improving the functionality of a computer network by enabling high - priority devices to connect to the network even when the components of the computer network are capacity - limited; (3) improving the performance of a computer network by enabling high - priority devices to receive high - performance throughput by associating custom capacity - limiting rules with dynamically - allocated network slices having high service - quality parameters, and so on.

[0016] Among the advantages of the present disclosure is the ability to deploy and chain network functions to deliver end-to-end services that meet specified constraints and requirements. According to the present disclosure, network functions composed of microservices cooperate to provide an end-to-end connection. One set of network functions is part of a wireless network, operates within a base station, and performs the conversion from wireless signal to IP. Other network functions execute subscriber-related business logic and are executed in a large data center that routes IP traffic to and from the Internet. For an application to use new 5G features such as low-latency communication and reserved bandwidth, both of these types of network functions need to cooperate to properly schedule and reserve the wireless spectrum and perform real-time computing and data processing. The techniques disclosed herein provide edge location hardware (further described below) integrated with network functions that are executed across the entire network, from the cell site to the Internet breakout, and orchestrate the network functions to meet the required quality of service (QoS) constraints. This enables a whole new series of applications with strict QoS requirements that were previously not possible to operate on mobile networks, ranging from factory-based Internet of Things (IoT) to augmented reality (AR), virtual reality (VR), game streaming, and autonomous navigation support for connected vehicles.

[0017] The described "Elastic 5G" service provides and manages all of the hardware, software, and network functions necessary for network construction. In some embodiments, the network functions may be developed and managed by a cloud service provider, but the described control plane enables a customer to manage network functions across various providers such that the customer can use a single set of APIs to call and manage the network functions selected on the cloud infrastructure. The Elastic 5G service has the advantage of automating the creation of an end-to-end 5G network from hardware to network functions, thereby reducing the deployment time and the operational costs associated with network operation. By providing APIs that expose network functions, the disclosed Elastic 5G service enables applications to specify the desired QoS as a constraint and then simply deploy and chain network functions to deliver an end-to-end service that meets the specified requirements, thus making it possible to easily build new applications.

[0018] The present disclosure describes embodiments related to the creation and management of cloud-native 5G cores and / or cloud-native 5G RANs, and related control plane components. Cloud-native refers to an approach for building and running applications that leverage the benefits of cloud computing delivery models such as dynamic scalability, distributed computing, and high availability (including geographic distribution, redundancy, and failover). Cloud-native refers to the way in which these applications are created and deployed such that they are suitable for deployment in a public cloud. Cloud-native applications can (and often are) run in a public cloud, but they can also be run in an on-premises data center. Some cloud-native applications can be containerized, for example, different parts, functions, or subunits of an application can be packaged into their own containers and dynamically orchestrated such that each part is actively scheduled and managed to optimize resource utilization. These containerized applications can be built using a microservices architecture to improve the overall agility and maintainability of the application.

[0019] In a microservices architecture, an application is arranged as a collection of smaller sub-units ("microservices") that can be deployed and scaled independently of each other and communicate with each other over a network. These microservices typically have a specific technical and functional granularity and often implement lightweight communication protocols, so they have a fine granularity. The microservices of an application can perform different functions from each other, can be deployed independently, and can use different programming languages, databases, and hardware / software environments. Decomposing an application into smaller services beneficially improves the modularity of the application, makes it possible to replace individual microservices as needed, and parallelizes development by enabling teams to develop, deploy, and maintain their microservices independently of each other. Microservices can, in some examples, be deployed using virtual machines, containers, or serverless functions. The disclosed core and RAN software can conform to a microservices architecture in which the described wireless network is composed of independent sub-units that can be deployed on demand and scaled.

[0020] Referring now to FIG. 1A, an example of a deployed and managed communication network 100 according to various embodiments of the present disclosure is shown. The communication network 100 may correspond to an access network 103 for a cellular network such as a fourth generation (4G) Long-Term Evolution (LTE) network, a fifth generation (5G) network, a 4G-5G hybrid core including both 4G and 5G RANs, a sixth generation (6G) network, a wired network, or another network that provides network access. The access network 103 may be operated by a cloud service provider for an enterprise, non-profit organization, school organization, government agency, government entity, communication service provider, or other organization. The access network 103 may use a private network address or a public network address in various embodiments.

[0021] Various deployments of the access network 103 can include one or more of a core network and a RAN network, as well as a control plane for executing the core network and / or the RAN network on cloud provider infrastructure. As described above, these components can be developed in a cloud-native manner using, for example, a microservices architecture such that centralized control and distributed processing are used to efficiently scale traffic and transactions. These components can be based on 3GPP (registered trademark) specifications by following an application architecture (CUPS architecture) in which the processing of the control plane and the user plane are separated.

[0022] Access network 103 provides wireless network access to a plurality of client devices 106 that can be mobile devices or fixed location devices. In various examples, client devices 106 can include smartphones, connected vehicles, IoT devices, sensors, machinery (such as in a manufacturing facility), hotspots, and other devices. Client devices 106 can be referred to as user equipment (UE) or customer premise equipment (CPE).

[0023] Access network 103 can include a radio access network (RAN) that provides wireless network access to a plurality of client devices 106 through a plurality of cells 109. Each of the cells 109 can be equipped with one or more antennas and one or more radio units for transmitting and receiving wireless data signals to and from client devices 106. In some cases, the cells 109 can have limited device capabilities (e.g., 64 devices, 128 devices, or another limit) based on the deployed hardware resources. The antennas can be configured for one or more frequency bands, and the radio units can also be frequency agile or frequency tunable. To concentrate the signal in a specific direction or azimuth range, a specific gain or beamwidth can be associated with the antennas, which can enable frequency reuse in different directions. Further, the antennas can be horizontally polarized, vertically polarized, or circularly polarized. In some examples, the radio units can utilize multiple-input multiple-output (MIMO) technology to transmit and receive signals. Thus, the RAN implements wireless access technology to enable a wireless connection with client devices 106 and provides a connection to the core network of a wireless private network. The components of the RAN include not only the base stations and antennas that cover a given physical area but also the necessary core network items for managing the connection to the RAN.

[0024] Data traffic is often routed to the core network via a fiber transport network composed of multiple hops of layer 3 routers (e.g., at an aggregation site). The core network is typically housed in one or more data centers. Usually, the core network aggregates data traffic from end devices, authenticates subscribers and devices, applies personalized policies, manages device mobility, and then routes the traffic to operator services or the Internet. For example, the 5G core can be separated into a control plane and a user plane and decomposed into several microservice elements. Since the 5G core can include virtualized software-based network functions (e.g., deployed as microservices) rather than physical network elements, it can be instantiated within a multi-access edge computing (MEC) cloud infrastructure. The network functions of the core network can include user plane functions (UPF), access and mobility management functions (AMF), and session management functions (SMF), which will be described in more detail below. In the case of data traffic destined for locations external to the communication network 100, the network functions typically include a firewall for an external network such as the Internet or a cloud provider network through which traffic can enter and leave the communication network 100. Note that in some embodiments, the communication network 100 can include facilities that allow traffic to enter and leave sites further downstream from the core network (e.g., an aggregation site or access network 103).

[0025] The UPF provides an interconnection point between the mobile infrastructure and the data network (DN), that is, it provides encapsulation and decapsulation of the General Packet Radio Service (GPRS) Tunneling Protocol (GTP-U) for the user plane. The UPF may also provide a session anchor point for providing mobility within the RAN, such as sending one or more end marker packets to the RAN base station. The UPF may handle packet routing and forwarding, including directing the flow to a specific data network based on traffic matching filters. Another function of the UPF includes QoS processing for each flow or application, including transport-level packet marking and rate limiting for the uplink (UL) and downlink (DL). The UPF can be implemented as a cloud-native network function using the latest microservices approach, for example, it can be deployed within a serverless framework (which abstracts the underlying infrastructure where the code is executed via a managed service).

[0026] The AMF can receive connection and session information from the client device 106 or the RAN and can process connection and mobility management tasks. For example, the AMF can manage handovers between base stations within the RAN. In some examples, the AMF can be regarded as an access point to the 5G core by terminating the traffic of a specific RAN control plane and the client device 106. The AMF may also implement encryption and integrity protection algorithms.

[0027] The SMF can handle session establishment or change, for example, by creating, updating, and deleting Protocol Data Unit (PDU) sessions and managing session contexts within the UPF. The SMF may also implement the Dynamic Host Configuration Protocol (DHCP) and IP Address Management (IPAM). The SMF may be implemented as a cloud-native network function using the latest microservices approach.

[0028] The various network functions for implementing the access network 103 can be deployed within a distributed computing device 112 that can correspond to a general-purpose computing device configured to execute the network functions. For example, the distributed computing device 112 can execute one or more virtual machine instances that are sequentially configured to execute one or more services that implement the network functions. In one embodiment, the distributed computing device 112 is a highly durable machine deployed at each cell site.

[0029] In contrast, one or more centralized computing devices 115 can execute various network functions at a central site operated by a customer. For example, the centralized computing device 115 may be intensively placed within the customer's premises in an adjusted server room. The centralized computing device 115 can execute one or more virtual machine instances that are sequentially configured to execute one or more services that implement the network functions.

[0030] In one or more embodiments, network traffic from the access network 103 is backhauled to one or more computing devices on a core network 118 that can be located in one or more data centers remote from the customer's site. The core network 118 can also perform various network functions, including routing network traffic to and from a network 121 that can correspond to the Internet and / or other external public or private networks. The core network 118 can perform functions related to the management of the communication network 100 (e.g., billing, mobility management, etc.) and a transport function for relaying traffic between the communication network 100 and other networks.

[0031] Moving on to FIG. 1B, an example of a wireless private network 150 deployed according to various embodiments of the present disclosure and used on the premises of an organization (such as a campus of a business, school, or other organization, etc.) having a plurality of buildings 153 is shown. Although FIG. 1B shows an example with a plurality of buildings, it will be understood that the disclosed techniques can be similarly applied to any layout of a site that may include one or more buildings and / or one or more outdoor spaces (such as a stadium or other outdoor venue, etc.).

[0032] The wireless private network 150 in this non-limiting example includes four cells 156a, 156b, 156c, and 156d to fully cover the organization's premises. The cells 156 may somewhat overlap in order to comprehensively cover within each building 153. Adjacent or overlapping cells 156 are configured to operate on non-interfering frequencies. For example, cell 156a may use frequency A, cell 156b may use frequency B, cell 156c may use frequency C, and since the coverage of each of the cells 156a, 156b, and 156c overlaps, these are all distinct frequencies. However, since the coverage of cell 156d does not overlap with cell 156a or cell 156b, cell 156d can use, for example, frequency A or frequency B.

[0033] Note that cell 156 can be added to or removed from the wireless private network 150 depending on usage or other network metrics. In some cases, the signal strength to cell 156 can be increased to reduce the number of cells 156 while allowing for spectrum reuse between cells 156, or decreased to increase the number of cells 156. Further, computing capabilities can be added within the geographic area within the organization's premises or within the cloud provider network to reduce the latency of the wireless private network 150, maintain security, and increase reliability as desired. In some cases, the computing capabilities of the software implementing the wireless private network 150 can be mostly or entirely provisioned within the cloud provider network, such as in a regional data center of a cloud service provider, rather than within the customer's premises. This software can implement various network functions such as UPF, AMF, SMF that can support core network functions, central unit network functions, and distributed unit network functions. Some network functions, such as distributed unit network functions, can remain at the cell site.

[0034] FIG. 1C is a diagram of an exemplary scenario showing an exemplary implementation of a capacity-limiting plan. At stage 160a, client devices 106a, 106b, and 106c are currently connected to communication network 100 (FIG. 1A), and the processing is executed in network function 163. In this example, it is desired to assume that network function 163 has the device capabilities of three simultaneous devices. Client device 106d then sends a service request 165 to access communication network 100, which involves processing using network function 163. Since network function 163 is already capacity-limited, network function 163 needs to determine how to handle the incoming service request 165. Due to the fact that network function 163 is already capacity-limited, the default rule may be to reject service request 165. However, client device 106d could be a high-priority device, and the owner of communication network 100 may prefer that client device 106d have access.

[0035] Accordingly, the owner of the communication network 100 may configure a custom rule that overrides the default rules and provides services to the high-priority client device 106d via the network function 163. In stage 160b, the network function 163 determines to select one or more specific client devices 106a, 106b, or 106c that will be disconnected according to the rules from the capacity-limited plan. In this example, according to the rules, the network function 163 selects the client devices 106b and 106c for service reservation 167, and the client device 106d is provided with services via the network function 163. The client devices 106b and 106c may be selected with relatively low priority compared to the other client devices 106 by a first-in first-out method, a last-in last-out method, a round-robin method, or another method defined by the rules. In this example, multiple client devices 106b and 106c may have their services reserved to accommodate a single client device 106d. For example, the client device 106d may have high bandwidth requirements for video, and both client devices 106b and 106c may utilize comparable bandwidth. In another example, a single client device 106 may have its service reserved to accommodate one different client device 106.

[0036] Figure 2A shows an example of a network environment 200 that includes a cloud provider network 203 according to some embodiments, and further includes various provider substrate extensions of the cloud provider network that can be used in combination with an on-premises customer deployment within the communication network 100 of FIG. 1. The cloud provider network 203 (which may also be simply referred to as the "cloud") refers to a pool of network-accessible computing resources (such as computing, storage, and networking resources, applications, and services), which can be virtualized or bare metal. The cloud can provide convenient on-demand network access to a shared pool of configurable computing resources that can be programmatically provisioned and released in response to customer commands. These resources can be dynamically provisioned and reconfigured to adjust to varying loads. Thus, cloud computing can be regarded as both applications provided as services via a publicly accessible network (such as the Internet, a cellular communication network, etc.) and the hardware and software within the data centers of cloud providers that provide those services.

[0037] The cloud provider network 203 can provide users with an on-demand scalable computing platform via a network. For example, users can freely have a scalable "virtual computing device" through the use of a computing server that provides computing instances (optionally using local storage via the use of one or both of a central processing unit (CPU) and a graphics processing unit (GPU)) and a block storage server that provides persistent block storage virtualized to a specified computing instance. These virtual computing devices have the attributes of a personal computing device, including hardware (various types of processors, local memory, random access memory (RAM), hard disks, and / or solid-state drive (SSD) storage), a selected operating system, network functions, and preloaded application software. Each virtual computing device may also virtualize its console input / output (e.g., keyboard, display, and mouse). This virtualization enables users to configure and use their virtual computing devices as if they were personal computing devices by connecting to their virtual computing devices using computer applications such as browsers, APIs, and software development kits (SDKs). Different from a personal computing device that owns a fixed amount of hardware resources available to the user, the hardware associated with a virtual computing device can scale up or down according to the resources required by the user.

[0038] As described above, the user can connect to the resources and services of the virtual computing device and other cloud provider networks 203 via the intermediate network(s) 212 using various interfaces 206 (e.g., APIs), and can configure and manage a telecommunications network such as a 5G network. The API refers to the interface and / or communication protocol between the client device 215 and the server such that when the client makes a request in a predefined format, the client needs to receive a response in a specific format or initiate a defined action. In the context of a cloud provider network, the API provides a gateway for the customer to access the cloud infrastructure and enables the development of applications that interact with the resources and services hosted in the cloud provider network by allowing the customer to retrieve data from the cloud provider network and execute actions within the cloud provider network. The API can also enable various services in the cloud provider network to exchange data with each other. The user can choose to deploy their own virtual computing system and provide network-based services for their own use and / or for use by their customers or clients.

[0039] The cloud provider network 203 can include a physical network (e.g., sheet metal boxes, cables, rack hardware) referred to as a substrate. The substrate can be regarded as a network fabric that includes the physical hardware for executing the services of the provider network. The substrate can be isolated from the rest of the cloud provider network 203, for example, it may not be possible to route from the substrate network address to an address within the production network that executes the services of the cloud provider or to a customer network that hosts customer resources.

[0040] The cloud provider network 203 may include an overlay network of virtualized computing resources running on a substrate. In at least some embodiments, a hypervisor or other device or process on the network substrate may use encapsulation protocol techniques to encapsulate and route network packets (e.g., client IP packets) across the network substrate between client resource instances on different hosts within the provider network. Encapsulation protocol techniques may be used on the network substrate to route encapsulated packets (also referred to as network substrate packets) between endpoints on the network substrate over an overlay network path or route. Encapsulation protocol techniques may be considered to provide a virtual network topology overlayed on the network substrate. Thus, network packets may be routed along the substrate network according to constructs within the overlay network (e.g., a virtual network that may be referred to as a virtual private cloud (VPC), a port / protocol firewall configuration that may be referred to as a security group). A mapping service (not shown) may coordinate the routing of these network packets. The mapping service may be a geographically distributed lookup service that maps a combination of an overlay Internet protocol (IP) and a network identifier to a substrate IP, enabling distributed substrate computing devices to locate the destination of a packet.

[0041] For example, each physical host device (e.g., a computing server, a block storage server, an object storage server, a control server) may have an IP address within the substrate network. Using hardware virtualization technology, it may be possible to simultaneously execute multiple operating systems on a host computer, e.g., as virtual machines (VMs) on a computing server. The hypervisor or virtual machine monitor (VMM) on the host allocates the host's hardware resources to various VMs on the host and monitors the execution of the VMs. Each VM may be given one or more IP addresses within the overlay network, and the VMM on the host may recognize the IP addresses of the VMs on the host. The VMM (and / or other devices or processes on the network substrate) may use encapsulation protocol technology to encapsulate and route network packets (e.g., client IP packets) across the network substrate between virtualized resources on different hosts within the cloud provider network 203. Encapsulation protocol technology may be used on the network substrate to route encapsulated packets between endpoints on the network substrate via an overlay network path or route. Encapsulation protocol technology may be regarded as providing a virtual network topology overlayed on the network substrate. Encapsulation protocol technology may include a mapping service that maintains a mapping directory that maps IP overlay addresses (e.g., IP addresses visible to a customer) to substrate IP addresses (IP addresses not visible to a customer), and this mapping directory may be accessed by various processes on the cloud provider network 203 to route packets between endpoints.

[0042] As illustrated, the traffic and operations of the cloud provider network substrate can be broadly subdivided into two categories in various embodiments: control plane traffic carried on the logical control plane 218 and data plane operations carried on the logical data plane 221. The data plane 221 represents the movement of user data through the distributed computing system, and the control plane 218 represents the movement of control signals through the distributed computing system. The control plane 218 generally includes one or more control plane components or services that are distributed across and implemented by one or more control servers. Control plane traffic generally includes administrative operations such as the establishment of various customer-segregated virtual networks, monitoring of resource usage and health, identification of the specific host or server on which a requested compute instance is launched, and provisioning of additional hardware as needed. The data plane 221 includes customer resources (e.g., compute instances, containers, block storage volumes, databases, file storage) implemented on the cloud provider network. Data plane traffic generally includes non-administrative operations such as data transfers to and from customer resources.

[0043] Control plane components are typically implemented on a separate set of servers from the data plane servers, and control plane traffic and data plane traffic can be sent via separate / different networks. In some embodiments, control plane traffic and data plane traffic can be supported by different protocols. In some embodiments, messages (e.g., packets) sent via the cloud provider network 203 include a flag indicating whether the traffic is control plane traffic or data plane traffic. In some embodiments, the payload of the traffic can be inspected to determine its type (e.g., whether it is control plane or data plane). Other techniques for distinguishing traffic types are also possible.

[0044] As illustrated, data plane 221 may be bare metal (e.g., single tenant), or may include one or more computing servers that are virtualized by a hypervisor to run multiple VMs (sometimes referred to as "instances") or micro-VMs for one or more customers. These computing servers can support virtualized computing services (or "hardware virtualization services") of a cloud provider network. The virtualized computing service may be part of control plane 218, enabling a customer to issue commands via interface 206 (e.g., an API) to launch and manage computing instances (e.g., VMs, containers) for an application. The virtualized computing service may provide virtual computing instances with various computing resources and / or memory resources. In one embodiment, each of the virtual computing instances may correspond to one of several instance types. An instance type may be characterized by its hardware type, computing resources (e.g., number, type, and configuration of CPUs or CPU cores), memory resources (e.g., capacity, type, and configuration of local memory), storage resources (e.g., capacity, type, and configuration of locally accessible storage), network resources (e.g., characteristics of its network interface and / or network functions), and / or other suitable descriptive characteristics. An instance type selection function may be used to select an instance type for a customer, for example, (at least in part) based on input from the customer. For example, a customer may be able to select an instance type from a set of pre-defined instance types. As another example, a customer may specify desired resources of an instance type and / or requirements of a workload that the instance runs, and the instance type selection function may select an instance type based on such specifications.

[0045] The data plane 221 may also include one or more block storage servers, which may include persistent storage for storing volumes of customer data and software for managing these volumes. Such block storage servers may support a managed block storage service of the cloud provider network. The managed block storage service may be part of the control plane 218 and enables a customer to issue commands via an interface 206 (e.g., an API) to create and manage volumes of applications running on computing instances. The block storage server includes one or more servers where data is stored as blocks. A block is a sequence of bytes or bits and typically contains an integral number of records with a maximum length of the block size. Blocked data is typically stored in a data buffer and the entire block is read and written at once. Generally, a volume can correspond to a logical collection of data, such as a set of data maintained on behalf of a user. For example, a user volume that can be treated as an individual hard drive ranging in size from 1 GB to over 1 terabyte (TB) is composed of one or more blocks stored in the block storage server. Although treated as an individual hard drive, it will be understood that a volume can be stored as one or more virtualized devices implemented on one or more underlying physical host devices. A volume may be partitioned a small number of times (e.g., up to 16 times) and each partition may be hosted by a different host. The data of a volume may be replicated among multiple devices within the cloud provider network to provide multiple replicas of the volume (such replicas may collectively represent a volume on a computing system).Replicas of a volume in a distributed computing system can, beneficially, provide automatic failover and recovery, for example, by enabling a user to access either a primary replica of the volume or a secondary replica of the volume synchronized with the primary replica at the block level, so that access to the volume's information is not blocked by a failure of either the primary or secondary replica. The role of the primary replica can be to facilitate reads and writes on the volume (also sometimes referred to as "input / output operations" or simply "I / O operations") and to reflect any writes secondarily (which may use asynchronous replication, but preferably synchronously on the I / O path). The secondary replica is updated in synchronization with the primary replica and can provide a seamless transition during a failover operation, whereby the secondary replica assumes the role of the primary replica and the previous primary is designated as secondary or a new alternative secondary replica is provisioned. In a particular example herein, the primary and secondary replicas are described, but it will be understood that a logical volume can include multiple secondary replicas. A computing instance can virtualize its I / O to the volume via a client. The client can be implemented on an offload card of a server that includes a processing unit (e.g., a CPU or GPU) of the computing instance.

[0046] The data plane 221 may include one or more object storage servers corresponding to another type of storage within the cloud provider network. An object storage server includes one or more servers in which data is stored as objects within a resource called a bucket and may be used to support a managed object storage service of the cloud provider network. Each object typically includes the stored data, a variable amount of metadata that enables various functions of the object storage server related to the analysis of the stored object, and a globally unique identifier or key that can be used to retrieve the object. Each bucket is associated with a given user account. A customer can store a desired number of objects within their bucket, write, read, and delete objects within their bucket, and control access to their bucket and the objects contained therein. Further, in embodiments having different object storage servers distributed across different regions of the regions described above, a user can select, for example, the region (or regions) in which a bucket is stored to optimize latency. A customer can use a bucket to store various types of objects, such as a machine image that can be used to boot a VM or a snapshot representing a point-in-time view of the data of a volume.

[0047] The Provider Substrate Extension 224 (the "PSE") provides the resources and services of the cloud provider network 203 within a separate network such as a telecommunications network, thereby extending the functionality of the cloud provider network 203 to a new location (e.g., for reasons related to latency associated with customer devices, regulatory compliance, security, etc.). In some embodiments, the Provider Substrate Extension (PROVIDER SUBSTRATE EXTENSION) 224 can be configured to provide capabilities to cloud-based workloads running within a telecommunications network. In some implementations, the Provider Substrate Extension 224 can be configured to provide the core and / or RAN functions of a telecommunications network and can be composed of additional hardware (e.g., radio access hardware). Some embodiments can be configured to enable both, for example, by making unused capabilities available for execution of cloud-based workloads by the core and / or RAN functions.

[0048] As shown, such a Provider Substrate Extension 224 can include, among other possible types of substrate extensions, a Cloud Provider Network Managed Provider Substrate Extension 227 (e.g., formed by a server located within a cloud provider management facility separate from that associated with the cloud provider network 203), a Communications Service Provider Managed Provider Substrate Extension 230 (e.g., formed by a server associated with a communications service provider facility), and a Customer Managed Provider Substrate Extension 233 (e.g., formed by a server located on-premises at a customer or partner facility).

[0049] As illustrated by the exemplary provider substrate extension 224, the provider substrate extension 224 can similarly include a logical separation between a control plane 236 and a data plane 239 that extends the control plane 218 and the data plane 221 of the cloud provider network 203, respectively. The provider substrate extension 224 can be preconfigured by, for example, a cloud provider network operator to reflect the experience of using the cloud provider network to support various types of computing-related resources using an appropriate combination of hardware with software and / or firmware elements. For example, one or more provider substrate extension location servers can be provisioned by a cloud provider for deployment within the provider substrate extension 224. As described above, the cloud provider network 203 can provide a set of predefined instance types, each having various types and amounts of underlying hardware resources. Each instance type may also be provided in various sizes. The server can be a heterogeneous server to enable a customer to continue to use within the provider substrate extension 224 the same instance type and size that the customer uses in the region. A heterogeneous server can support multiple instance sizes of the same type and can be reconfigured to host any instance type supported by its underlying hardware resources. The reconfiguration of the heterogeneous server can be performed on the fly using the available capabilities of the server, i.e., while other VMs are still running and consuming other capabilities of the provider substrate extension location server.This enables better packing of the instances running on the server, thereby improving the utilization of computing resources within the edge location and also providing a seamless experience regarding the use of instances across the cloud provider network 203 and the cloud provider network managed provider substrate extension 227.

[0050] The provider substrate extension server may host one or more computing instances. The computing instance may be a VM that packages the code and all its dependencies, or a container, enabling the application to run quickly and reliably across the computing environment (including, for example, VMs and micro-VMs). Additionally, if requested by the customer, the server may host one or more data volumes. Within the area of the cloud provider network 203, such volumes may be hosted on dedicated block store servers. However, due to the possibility of significantly lower capabilities in the provider substrate extension 224 compared to that area, if the provider substrate extension 224 includes such dedicated block store servers, an optimal utilization experience may not be provided. Therefore, the block storage service may be virtualized within the provider substrate extension 224 such that one of the VMs runs block store software to store the data of the volume. Similar to the operation of the block storage service within the area of the cloud provider network 203, the volumes within the provider substrate extension 224 may be replicated for durability and availability. The volumes may be provisioned within a separate virtual network unique to the provider substrate extension 224. The computing instances and any volumes collectively constitute the data plane 239 extension of the provider network data plane 221 within the provider substrate extension 224.

[0051] In some embodiments, the servers within the provider substrate extension 224 may host certain local control plane components, such as components that enable the provider substrate extension 224 to continue to function in the event that the connection back to the cloud provider network 203 is severed. Examples of these components include a migration manager that can move compute instances between provider substrate extension servers as needed to maintain availability, and a key-value data store that indicates the location of volume replicas. However, generally, the control plane 236 functionality of the provider substrate extension remains within the cloud provider network 203 in order to allow the customer to use as much of the resource capacity of the provider substrate extension as possible.

[0052] The migration manager may have a local controller that runs on a PSE server (and servers within the data center of the cloud provider) together with the centralized correlation components that run within the region. The centralized correlation components can identify the target edge location and / or target host when a migration is triggered, and the local controller can coordinate the data transfer between the source host and the target host. The movement of the described resources between hosts at different locations can take one of several migration forms. Migration refers to moving virtual machine instances (and / or other resources) between hosts within a cloud computing network or between a host outside the cloud computing network and a host within the cloud. There are various types of migrations, such as live migration and restart migration. During a restart migration, the customer experiences the stopping and effective restart of the virtual machine instance. For example, the control plane service may coordinate a restart migration workflow that includes discarding the current domain on the original host and then creating a new domain for the virtual machine instance on the new host. The instance is restarted by shutting down on the original host and restarting on the new host.

[0053] Live migration refers to the process of moving a running virtual machine or application between different physical machines without significantly degrading the availability of the virtual machine (e.g., the downtime of the virtual machine is not noticed by the end user). When the control plane executes the live migration workflow, it can create a new "inactive" domain associated with the instance, while the original domain of the instance continues to be executed as the "active" domain. The memory of the virtual machine (including any in-memory state of the running application), storage, and network connectivity are transferred from the original host with the active domain to the destination host with the inactive domain. The virtual machine can be briefly paused to prevent state changes while transferring the contents of the memory to the destination host. The control plane can migrate the inactive domain to become the active domain, demote the original active domain to become the inactive domain (also called "flipping"), and then discard the inactive domain.

[0054] Various types of migration techniques involve management of a critical phase (the time during which a customer cannot use a virtual machine instance), and this phase needs to be made as short as possible. In currently disclosed migration techniques, this management can be particularly difficult because resources are moved between hosts at geographically separated locations that can be connected via one or more intermediate networks. In the case of live migration, the disclosed techniques can dynamically determine, for example, the amount of memory state data to copy prior (e.g., while the instance is still running on the source host) and after copying (e.g., after the instance has started execution on the destination host) based on, for example, the latency between locations, the network bandwidth / usage pattern, and / or which memory pages are most frequently used by the instance. Further, the particular time at which the memory state data is transferred can be dynamically determined based on the state of the network between locations. This analysis can be performed by a migration management component within the region or by a migration management component that executes locally within the source edge location. If an instance has access to virtualized storage, both the source domain and the target domain can be simultaneously attached to the storage to allow uninterrupted access to the data during migration and in the event of a rollback to the source domain being necessary.

[0055] The server software executed by the provider substrate extension 224 may be designed by the cloud provider to execute on the cloud provider substrate network, and this software can be made to execute without modification within the provider substrate extension 224 by creating a private replica of the substrate network (a "shadow substrate") within the edge location using the local network manager(s) 242. The local network manager(s) 242 can function as an endpoint, for example, between a virtual private network (VPN) endpoint or the provider substrate extension 224 within the cloud provider network 203 and the proxies 245, 248, and by implementing a mapping service (for traffic encapsulation and decapsulation) to associate data plane traffic (from the data plane proxy 248) and control plane traffic (from the control plane proxy 245) with the appropriate server(s), serve the shadow substrate with the network of the provider substrate extension 224, and bridge the shadow substrate with the network of the provider substrate extension 224 so that it can be executed on the provider substrate extension 224. By implementing a local version of the provider network's substrate overlay mapping service, the local network manager(s) 242 enables the resources within the provider substrate extension 224 to communicate seamlessly with the resources within the cloud provider network 203. In some implementations, a single local network manager 242 can perform those actions for all servers hosting compute instances within the provider substrate extension 224. In other embodiments, each server hosting a compute instance may have a dedicated local network manager 242.In a multi-rack edge location, the local network manager maintains open tunnels with each other, and inter-rack communication can pass through the local network manager 242.

[0056] The provider substrate extension location can utilize a secure network tunnel to reach the cloud provider network 203 via the provider substrate extension 224 network to maintain the security of customer data when passing through, for example, the provider substrate extension 224 network and other intermediate networks (which can include the public Internet). Within the cloud provider network 203, those tunnels are composed of virtual infrastructure components including an isolated virtual network (e.g., within an overlay network), a control plane proxy 245, a data plane proxy 248, and a substrate network interface. Such proxies 245, 248 can be implemented as containers running on a compute instance. In some embodiments, each server within the location of the provider substrate extension 224 that hosts a compute instance can utilize at least two tunnels: one for control plane traffic (e.g., Constrained Application Protocol (CoAP) traffic) and one for encapsulated data plane traffic. A connectivity manager (not shown) within the cloud provider network 203 manages the cloud provider network side life cycle of those tunnels and their components, for example, by automatically provisioning them when needed and managing them in a healthy operating state. In some embodiments, a direct connection between the provider substrate extension 224 location and the cloud provider network 203 can be used for control plane communication and data plane communication. Compared to a VPN over other networks, the direct connection can provide a certain bandwidth and more consistent network performance because its network path is relatively fixed and stable.

[0057] The control plane (CP) proxy 245 may be provisioned within the cloud provider network 203 to represent a particular host(s) at an edge location. The CP proxy 245 serves as an intermediary between the control plane 218 within the cloud provider network 203 and the control plane target within the control plane 236 of the provider substrate extension 224. That is, the CP proxy 245 provides infrastructure for tunneling management API traffic destined for the provider substrate extension servers from outside the regional substrate and to the provider substrate extension 224. For example, the virtual computing service of the cloud provider network 203 may issue commands to the VMM of the servers of the provider substrate extension 224 to start computing instances. The CP proxy 245 maintains a tunnel (e.g., VPN) to the local network manager 242 of the provider substrate extension. The software implemented within the CP proxy 245 ensures that only eligible API traffic exits and returns to the substrate. The CP proxy 245 publishes remote servers on the cloud provider substrate and provides a mechanism to protect that substrate security material (e.g., encryption keys, security tokens) from remaining within the cloud provider network 203. The one-way control plane traffic tunnel imposed by the CP proxy 245 also prevents any (potentially compromised) device from calling back into the substrate. The CP proxy 245 may be instantiated on a one-to-one basis with the servers of the provider substrate extension 224 or may manage the control plane traffic of multiple servers within the same provider substrate extension.

[0058] The data plane (DP) proxy 248 may also be provisioned within the cloud provider network 203 to represent a particular server(s) within the provider substrate extension 224. The DP proxy 248 functions as a shadow or anchor for the server(s), and may be used by services within the cloud provider network 203 to monitor the health of the host (including availability, used / free computing and capacity, used / free storage and capacity, and usage / availability of network bandwidth). The DP proxy 248 also functions as a proxy for the server(s) within the cloud provider network 203, enabling a separate virtual network to span the provider substrate extension 224 and the cloud provider network 203. Each DP proxy 248 may be implemented as a packet forwarding compute instance or container. As illustrated, each DP proxy 248 can maintain a VPN tunnel with a local network manager 242 that manages traffic to the server(s) that the DP proxy 248 represents. This tunnel can be used to transmit data plane traffic between the provider substrate extension server(s) and the cloud provider network 203. Data plane traffic flowing between the provider substrate extension 224 and the cloud provider network 203 can pass through the DP proxy 248 associated with that provider substrate extension 224. In the case of data plane traffic flowing from the provider substrate extension 224 to the cloud provider network 203, the DP proxy 248 may receive the encapsulated data plane traffic, verify its accuracy, and permit it to enter the cloud provider network 203. The DP proxy 248 can forward the encapsulated traffic directly from the cloud provider network 203 to the provider substrate extension 224.

[0059] The local network manager(s) 242 can provide secure network connectivity with the proxies 245, 248 established within the cloud provider network 203. After the connectivity between the local network manager 242 and the proxies 245, 248 is established, the customer can issue commands via the interface 206 using the provider substrate extension resources in the same way as commands are issued regarding the compute instances hosted within the cloud provider network 203 to instantiate (and / or perform other operations using) the compute instances. From the customer's perspective, the customer can now seamlessly use the local resources within the provider substrate extension (and, if desired, resources in the cloud provider network 203). Compute instances set up on the server in the provider substrate extension 224 can communicate with both electronic devices within the same network and other resources set up within the cloud provider network 203, as desired. A local gateway 251 may be implemented to provide network connectivity between the provider substrate extension 224 and the network to which the extension is coupled (e.g., the communication service provider network in the example of the communication service provider substrate extension 230).

[0060] Depending on the situation, it may be necessary to transfer data between the object storage service and the provider substrate extension (PSE) 224. For example, the object storage service can store the machine image used to start a VM and the snapshot representing the backup of a volume at a specific point in time. The object gateway can be provided on a PSE server or storage device and can provide customers with per-bucket caching of the object storage bucket contents within those provider substrate extensions 224 to minimize the impact of PSE-region latency on the customer's workload. The object gateway can also temporarily store snapshot data from a volume snapshot in the provider substrate extension 224 and then synchronize with the object server in the region when possible. The object gateway can also store the machine image specified by the customer for use within the provider substrate extension 224 or on the customer's premises. In some implementations, the data within the provider substrate extension 224 can be encrypted with a unique key, and the cloud provider can restrict the sharing of the key from the region to the provider substrate extension 224 for security reasons. Therefore, the data exchanged between the object store server and the object gateway can utilize encryption, decryption, and / or re-encryption to maintain the security boundary regarding the encryption key or other sensitive data. The conversion intermediary can perform those operations, and PSE buckets can be created (on the object store server) to store snapshot data and machine image data using the PSE encryption key.

[0061] In the manner described above, the provider substrate extension 224 forms an edge location in that it approaches the customer device outside of a conventional cloud provider data center to provide the resources and services of the cloud provider network 203. The edge locations referred to herein can be structured in several ways. In some embodiments, the edge location can be an extension of the cloud provider network substrate that includes a limited amount of capacity provided outside of an availability zone (e.g., within a small data center or other facility of the cloud provider that is located near the customer's workload and can be away from any availability zone). Such edge locations may be referred to as "far zones" (due to being far from other availability zones) or "near zones" (due to being close to the customer's workload). A near zone can be connected to a publicly accessible network such as the Internet in various ways, for example, directly, via another network, or via a private connection to the region. Typically, a near zone has limited capacity compared to a region, although in some cases, a near zone can have a significant amount of capacity such as thousands of racks or more.

[0062] In some implementations, the edge location can be an extension of the cloud provider network substrate formed by one or more servers located on-premises within a customer or partner facility, and such server(s) communicate via an availability zone near the cloud provider network and a network (e.g., a publicly accessible network such as the Internet). This type of substrate extension located outside of the cloud provider network data center may be referred to as an "outpost" of the cloud provider network. Some outposts can be integrated into the communication network, for example, as multi-access edge computing (MEC) sites having physical infrastructure that spans over telecommunications data centers, telecommunications aggregation sites, and / or telecommunications base stations within a telecommunications network. In an on-premises example, the limited capabilities of the outpost can be used only by the customer who owns the facility (and any other accounts permitted by the customer). In a telecommunications example, the limited capabilities of the outpost can be shared among some applications (e.g., games, virtual reality applications, healthcare applications) that want to send data to users of the telecommunications network.

[0063] Edge location can include data plane capabilities that are at least partially controlled by the control plane of an availability zone near the provider network. Thus, an availability zone group can include a "parent" availability zone and any "child" edge locations that are based on the parent availability zone (e.g., at least partially controlled by its control plane). Certain limited control plane functions (e.g., functions that require low-latency communication with customer resources and / or functions that enable an edge location to continue to function when disconnected from the parent availability zone) may also exist in some edge locations. Thus, in the example above, edge location refers to at least an extension of data plane capabilities located at the edge of the cloud provider network, close to customer devices and / or workloads.

[0064] In the example of FIG. 1A, the distributed computing device 112 (FIG. 1A), the centralized computing device 115 (FIG. 1A), and the core computing device 118 (FIG. 1A) can be implemented as a provider substrate extension 224 of the cloud provider network 203. The installation or placement of the provider substrate extension 224 within the communication network 100 can vary depending on the particular network topology or architecture of the communication network 100. The provider substrate extension 224 can be connected generally anywhere where the communication network 100 can generate packetized traffic (e.g., IP traffic). Further, the communication between a given provider substrate extension 224 and the cloud provider network 203 typically passes securely through at least a portion of the communication network 100 (e.g., via a secure tunnel, virtual private network, direct connection, etc.).

[0065] In 5G wireless network development efforts, edge locations may be considered as a possibility for the implementation of multi-access edge computing (MEC). Such edge locations can connect to various points within the 5G network that provide breakout of data traffic as part of the user plane function (UPF). Even in older wireless networks, edge locations can be incorporated. In 3G wireless networks, for example, an edge location can connect to the packet switched network portion of the communication network 100, such as a serving general packet radio service support node (SGSN) or a gateway general packet radio service support node (GGSN). In 4G wireless networks, an edge location can connect to a serving gateway (SGW) or a packet data network gateway (PGW) as part of the core network or the evolved packet core (EPC). In some embodiments, traffic between the provider substrate extension 224 and the cloud provider network 203 can be separated from the communication network 100 without routing through the core network.

[0066] In some embodiments, the provider substrate extension 224 can connect to multiple communication networks associated with each customer. For example, when two communication networks of each customer share or route traffic through a common point, the provider substrate extension 224 can connect to both networks. For example, each customer may allocate a portion of its network address space to the provider substrate extension, and the provider substrate extension may include a distinguishable router or gateway for traffic exchanged with each of the communication networks 100. For example, traffic addressed to the provider substrate extension 224 from one network can have a different destination IP address, source IP address, and / or virtual local area network (VLAN) tag than traffic received from another network. Similarly, traffic going from the provider substrate extension to a destination in one of the networks can be encapsulated to have an appropriate VLAN tag, a source IP address (e.g., from a pool allocated to the provider substrate extension from the destination network address space), and a destination IP address.

[0067] Figure 2B represents an example 253 of the cellularization and geographical distribution of a communication network 100 (Figure 1A) for providing a high availability user plane function (UPF). In Figure 2B, user device 254 communicates with request router 255 to route requests to one of a plurality of control plane cells 257a and 257b. Each control plane cell 257 may include a network service API gateway 260, a network slice configuration 262, a network service monitoring function 264, site planning data 266 (including layout, device type, number of devices, etc. that describe the customer's site requirements), a network service / function catalog 268, a network function orchestrator 270, and / or other components. To reduce the likelihood that a large-scale error affects a wide range of customers, the large control plane can be divided into cells by providing one or more cells that operate independently, for example, per customer, per network, or per region.

[0068] The network service / function catalog 268 is also referred to as the NF repository function (NRF). In a service-based architecture (SBA) 5G network, the control plane functions and the common data repository can be delivered via a set of interconnected network functions built using a microservices architecture. The NRF can maintain a record of the available NF instances and the services they support, enabling other NF instances to subscribe and be notified of registrations from a given type of NF instance. Thus, the NRF supports service discovery by receiving discovery requests from NF instances and can identify which NF instances support a particular service. The network function orchestrator 270 can perform NF lifecycle management, including instantiation, scale-out / in, performance measurement, event correlation, deployment of a set of capacity-limiting rules, and termination. The network function orchestrator 270 can also on-board new NFs, manage the migration of existing NFs to new or updated versions, identify the set of NFs appropriate for a particular network slice or large-scale network, and orchestrate NFs across different computing devices and sites that make up the access network 103 (FIG. 1A).

[0069] The control plane cell 257 can communicate with one or more cell sites 272, one or more customer local data centers 274, one or more local zones 276, and one or more regional zones 278. The cell site 272 includes computing hardware 280 that executes one or more distributed unit (DU) network functions 282. The customer local data center 274 includes computing hardware 283 that executes one or more DU or central unit (CU) network functions 284, a network controller 285, a UPF 286, one or more edge applications 287 corresponding to the customer's workload, and / or other components.

[0070] The local zone 276 may be within a data center operated by a cloud service provider and may execute one or more core network functions 288 such as an AMF, an SMF, a network exposure function (NEF) that securely exposes services and capabilities of other network functions, and an integrated data management (UDM) function that manages subscriber data for authentication, registration, and mobility management. The local zone 276 may also execute a UPF 286, a metric processing service 289, and one or more edge applications 287.

[0071] The regional zone 278 may be within a data center operated by a cloud service provider and may execute one or more core network functions 288; a UPF 286; an operations support system (OSS) 290 that supports network management systems, service delivery, service fulfillment, service assurance, and customer care; an Internet protocol multimedia subsystem (IMS) 291; a business support system (BSS) 292 that supports product management, customer management, revenue management, and / or order management; one or more portal applications 293, and / or other components.

[0072] In this example, the communication network 100 adopts a cellular architecture to reduce the blast radius of individual components. At the highest level, the control plane is within a plurality of control plane cells 257 to prevent failures of individual control planes from affecting all deployments.

[0073] Within each control plane cell 257, a plurality of redundant stacks with a control plane that shifts traffic to a secondary stack as needed may be provided. For example, the cell site 272 may be configured to utilize a nearby local zone 276 as its default core network. If the local zone 276 stops, the control plane can redirect the cell site 272 to use a backup stack within the regional zone 278. Typically, traffic routed from the Internet to the local zone 276 can be shifted to an endpoint in the regional zone 278. Each control plane cell 257 may implement a "stateless" architecture that shares a common session database across multiple sites (such as across availability zones or edge sites).

[0074] Figure 3 shows an exemplary cloud provider network 203 that includes a geographically distributed provider substrate extension 224 (Figure 2A) (or "edge location 303") according to some embodiments. As illustrated, the cloud provider network 203 may be formed as a plurality of regions 306, where a region is a distinct geographic area in which the cloud provider has one or more data centers 309. Each region 306 may include two or more availability zones (AZs) connected to each other via a private high-speed network such as a fiber communication connection. An availability zone refers to a separate failure domain that includes one or more data center facilities with separate power, separate network, and separate cooling from other availability zones. The cloud provider may strive to place the availability zones within a region far enough apart from each other so that multiple availability zones do not go offline simultaneously due to natural disasters, large-scale power outages, or other unexpected events. Customers can connect to resources within the availability zones of the cloud provider network via a publicly accessible network (e.g., the Internet, a cellular communication network, a communication service provider network). A transit center (TC) is a major backbone location that links customers to the cloud provider network and may be co-located with other network provider facilities (e.g., an Internet service provider, a telecommunications provider). Each region can operate two or more TCs for redundancy. Region 306 is connected to a global network that includes a private network infrastructure (e.g., a fiber connection controlled by a cloud service provider) that connects each region 306 to at least one other region. The cloud provider network 203 can deliver content from points of presence ("PoPs") that are external to these regions 306 but networked with these regions 306 via edge locations 303 and regional edge cache servers.Due to this partitioning and geographic dispersion of computing hardware, the cloud provider network 203 can provide customers with global low-latency resource access with a high degree of fault tolerance and stability.

[0075] Compared to the number of regional data centers or availability zones, the number of edge locations 303 can be much larger. By deploying edge locations 303 extensively in this way, low-latency connections to the cloud can be provided to a much larger group of end-user devices (compared to end-user devices that happen to be very close to a regional data center). In some embodiments, each edge location 303 can peer with a part of the cloud provider network 203 (e.g., a parent availability zone or regional data center). Such peering enables various components operating within the cloud provider network 203 to manage the computing resources of the edge location 303. In some cases, multiple edge locations 303 may be placed or installed in the same facility (e.g., separate racks of a computer system) and managed by different zones or data centers to provide additional redundancy. Note that in this specification, edge locations 303 are typically represented as being within the communication service provider network or access network 103 (FIG. 1A), but in some cases, such as when the cloud provider network facility is relatively close to the communication service provider facility, edge locations 303 may remain within the physical footprint of the cloud provider network 203 while being connected to the communication service provider network via a fiber or other network link.

[0076] Edge location 303 can be structured in several ways. In some embodiments, edge location 303 is an extension of the cloud provider network substrate that includes a limited amount of capacity provided outside of an availability zone (e.g., within a small data center located near a customer's workload and able to be away from any availability zone, or within another facility of the cloud provider). Such an edge location 303 may be referred to as a local zone (because it is closer to the local or a group of users than a conventional availability zone). A local zone can be connected to a publicly accessible network such as the Internet in various ways, e.g., directly, via another network, or via a private connection to region 306. Typically, a local zone has limited capacity compared to region 306, although in some cases a local zone can have a significant amount of capacity such as thousands of racks or more. Some local zones may use an infrastructure similar to a typical cloud provider data center instead of the edge location 303 infrastructure described herein.

[0077] As shown herein, cloud provider network 203 can be formed into several regions 306, each region 306 representing a geographic area where the cloud provider clusters data centers. Each region may further include a plurality of (e.g., two or more) availability zones (AZs) connected to each other via a private high-speed network, e.g., a fiber communication connection. An AZ can provide a separate failure domain that includes one or more data center facilities with separate power, separate network, and separate cooling from another AZ. The AZs within a region 306 are preferably located far enough apart from each other so that the same natural disaster (or other event causing a disruption) does not affect them or take multiple AZs offline simultaneously. A customer can connect to an AZ of the cloud provider network via a publicly accessible network (such as the Internet, a cellular communication network, etc.).

[0078] The pairing of a given edge location 303 to an AZ or region 306 of the cloud provider network 203 can be based on several factors. One such pairing factor is data sovereignty. For example, in order to keep data originating from one domestic communication network within that domestic country, an edge location 303 deployed within the communication network can be paired to an AZ or region 306 within that domestic country. Another factor is service availability. For example, some edge locations 303 can have different hardware configurations, such as the presence or absence of components such as local non-volatile storage for customer data (e.g., solid state drives), graphics accelerators, etc. Since some AZs or regions 306 may lack services to utilize those additional resources, the edge location can be paired to an AZ or region 306 that supports the use of those resources. Another factor is the latency between the AZ or region 306 and the edge location 303. The deployment of edge locations 303 within the communication network has latency advantages, and those advantages are offset by pairing the edge location 303 to a distant AZ or region 306 that introduces significant latency to the edge location 303 for regional traffic. Thus, edge locations 303 are often paired to nearby AZs or regions 306 (from a network latency perspective).

[0079] Referring to FIG. 4, a network environment 400 according to various embodiments is shown. The network environment 400 includes a computing environment 403, one or more client devices 406, one or more pre-deployed devices 409, a spectrum reservation service 410, and one or more access networks 103 that communicate data with each other via a network 412. The network 412 includes, for example, the Internet, an intranet, an extranet, a wide area network (WAN), a local area network (LAN), a wired network, a wireless network, a cable network, a satellite network, or other suitable network, etc., or any combination of two or more such networks.

[0080] The computing environment 403 can include, for example, a server computer or any other system that provides computing capabilities. Alternatively, the computing environment 403 can employ, for example, one or more server banks or computer banks or multiple computing devices that can be arranged in other configurations. Such computing devices can be located in a single facility or can be distributed among many different geographical locations. For example, the computing environment 403 can include multiple computing devices that together can provide host computing resources, grid computing resources, and / or any other distributed computing arrangement. In some cases, the computing environment 403 can be adapted to elastic computing resources where the allocation capabilities of processing, network, storage, or other computing-related resources can change over time. For example, the computing environment 403 can correspond to a cloud provider network 203 (FIG. 2A) where customers are billed based on their use of those computing resources according to a utility computing model.

[0081] In some embodiments, the computing environment 403 may correspond to a virtualized private network within a physical network that includes virtual machine instances, for example, executed on physical computing hardware by a hypervisor. The virtual machine instances and the containers running on these instances may be provided network connectivity via virtualized network components that are made available by physical network components such as routers and switches.

[0082] Various applications and / or other functions may be executed in the computing environment 403 according to various embodiments. Also, various data is stored in a data store 415 that is accessible to the computing environment 403. The data store 415 may represent multiple data stores 415 as recognizable. The data stored in the data store 415 is associated with, for example, the operation of various applications and / or functional entities described below.

[0083] The computing environment 403 as part of a cloud provider network that provides utility computing services includes a computing device 418 and other types of computing devices. The computing device 418 may correspond to different types of computing devices 418 and may have different computing architectures. The computing architecture may differ by utilizing processors having different architectures such as x86, x86_64, ARM, Scalable Processor Architecture (SPARC), PowerPC, and the like. For example, some computing devices 418 may have an x86 processor, while other computing devices 418 may have an ARM processor. The computing device 418 may also differ in available hardware resources such as local storage, a graphics processing unit (GPU), machine learning extensions, and other characteristics.

[0084] Computing device 418 can have various forms of allocated computing capabilities 421, which can include virtual machine (VM) instances, containers, serverless functions, and the like. A VM instance can be instantiated from a VM image. For this purpose, a customer can specify that the virtual machine instance should be launched within a particular type of computing device 418 rather than other types of computing devices 418. In various examples, one VM instance may be executed alone on a particular computing device 418, or multiple VM instances may be executed on a particular computing device 418. Also, a particular computing device 418 can execute different types of VM instances, which can provide different amounts of resources available via the computing device 418. For example, one type of VM instance can provide more memory and processing power than another type of VM instance.

[0085] Components executed on computing environment 403 include, for example, network management API 423, network management service 424, and other applications, services, processes, systems, engines, or functions not described in detail herein.

[0086] The network management API 423 provides an interface for creating, deploying, activating, managing, updating, deactivating, and deleting an access network 103. The network management API 423 may also include an interface for creating, updating, and deleting a capacity limit plan for a network function 163 (FIG. 1C) and for defining priorities for client device 106 or other user equipment. In various embodiments, the network management API 423 creates, describes, deletes, and updates a site or location, creates, describes, updates, and deletes a network plan, creates, describes, lists networks associated with a customer, deletes, activates a network, creates, describes, lists a data plan for a network, attaches a SIM or eSIM to a data plan, deletes a data plan, lists SIMs or eSIMs associated with a network, implements functions for configuring small cell or radio unit installations either singly or in batch, and other functions.

[0087] The network management service 424 is executed to manage, configure, and monitor the access network 103 operated by a cloud service provider on behalf of a customer. For this purpose, the network management service 424 enables a customer to order a new access network, scale up or scale down an existing access network 103, change the operation of an existing access network 103, apply a capacity limit plan to a network function 163, configure client devices 106 permitted to use the access network 103, define priorities for client devices 106, provide statistics and metrics regarding the operation of the access network 103, reserve the frequency spectrum of a customer's private network via the spectrum reservation service 410, etc. For example, the network management service 424 may generate one or more network pages such as a web page including a user interface. Also, the network management service 424 may support this functionality via an API that can be called by the client application 436. The network management service 424 may use the network management API 423 to implement various actions. In addition to facilitating interaction with the user, the network management service 424 also implements the orchestration of the deployment and configuration changes of the access network 103 and the continuous monitoring of performance parameters. For a specific site 438, the network management service 424 may generate a network plan 439 for the customer based at least in part on the specifications of the customer's locations within the site 438, an automated site survey by a drone, and / or other input parameters.

[0088] The network management service 424 may also implement provisioning and configuration changes on the hardware implementing the access network 103. This may include wireless units, antennas, VM instances or containers executing network functions, routers, switches, fiber termination devices, etc. For example, an antenna may be configured to operate at a specific frequency. A wireless unit may be programmed to operate at a specific frequency, participate in a specific access network 103, and backhaul traffic to a specific VM instance or container.

[0089] In some scenarios, the network management service 424 is executed to reconfigure the hardware already existing within the access network 103. In other scenarios, the network management service 424 is executed to preconfigure a set of hardware that is to be deployed to an existing or new access network 103. For this purpose, the network management service 424 may implement the configuration on one or more pre-deployed devices 409 that are temporarily connected to the network 412 to facilitate preconfiguration before the pre-deployed devices 409 that are to be deployed to the access network 103 are shipped to the customer.

[0090] The network management service 424 can also automate and arrange the hardware deployment to implement the access network 103. Based on the network plan 439 submitted by the customer or the network plan 439 generated for the customer, the network management service 424 can arrange for the procurement of hardware components from one or more vendors associated with the vendor computing devices required to implement the access network 103 according to the network plan 439. This can include automatically ordering new equipment from the vendor, reserving equipment already in the provider's inventory, or reassigning equipment that is no longer in use at the customer's site or another customer's site if available. In this regard, the network management service 424 may send instructions to the customer to return the unused equipment, and the equipment can be directly sent to other customers for use in other deployments. In another scenario, the network management service 424 can send instructions to the customer to move the equipment from a site where the equipment is no longer in use to another site where the equipment will be used. The network management service 424 can manage the connection of the equipment to the network 412 for pre-configuration as the pre-deployed device 409. The network management service 424 can also arrange for the shipment of the equipment to the customer's locations, including potentially multiple locations of the customer corresponding to each cell site.

[0091] The data stored in the data store 415 includes, for example, one or more sites 438, one or more network plans 439, one or more data plans 440, one or more cellular topologies 442, one or more spectrum allocations 445, device data 448, one or more priority groups 450, one or more sets of capacity limit rules 452, data 454 describing one or more network slices, radio unit configuration data 457, network function configuration data 453, and potentially other data.

[0092] Site 438 represents the specifications of the location where access network 103 will be deployed for the customer. In one embodiment, Site 438 is created by network management API 423 through site name and address specifications. In other examples, latitude and longitude coordinates may be used. Network management API 423 may associate a unique site identifier with Site 438.

[0093] Network plan 439 is the specification of access network 103 that will be deployed for the customer. In various embodiments, network plan 439 includes one or more identifiers of the site 438, premise, location, or geographical area to be covered, the number of cells, the maximum number of client devices 106 per cell, the number of client devices 106 or SIMs, device identification information and permissions, the desired measurement of edge computing capabilities within access network 103, the desired maximum network latency, the desired bandwidth or network throughput for one or more classes of devices, one or more service quality parameters for an application or service, the range of network addresses to be assigned to client devices 106, an identifier of the type of spectrum to be used (e.g., Citizens Broadband Radio Service, television white space, licensed spectrum, etc.), and / or other parameters that can be used to create access network 103. The customer can manually specify one or more of these parameters via a user interface or API. One or more of the parameters may be pre-set as default parameters. In some cases, network plan 439 may be generated for the customer based at least in part on an automated site survey using a drone. In some cases, network plan 439 may incorporate at least determined thresholds and reference parameters for an automated probe of the customer's existing private network.

[0094] The data plan 440 may correspond to one or more plans for accessing the access network 103 from the client device 106. The data plan 440 may include a network identifier, a data plan name, an uplink speed, a downlink speed, a unique identifier of the data plan 440, a latency requirement, a jitter requirement, and / or other data. The values of the parameters defining the data plan 440 may be used as a basis for the cloud service provider to bill the customer under the utility computing model. For example, the customer may be billed a high amount for low latency targets and / or high bandwidth targets in a service level agreement (SLA), and the customer may be billed on a per-device basis, on a per-cell basis, etc., based on the geographical area served, based on spectrum availability, etc.

[0095] The cellular topology 442 includes the arrangement of a plurality of cells for the customer, taking into account the location of the cells and, where possible, the reuse of the frequency spectrum. The cellular topology 442 can be automatically generated by performing a site survey. In some cases, the number of cells within the cellular topology 442 can be automatically determined based on the desired geographical area to be covered, the availability of backhaul connections at various sites, signal propagation, the available frequency spectrum, and / or other parameters. For the access network 103, the cellular topology 442 can be developed to cover one or more buildings within an organization, one or more schools within a school district, one or more buildings within a university or university system, and other areas.

[0096] The spectrum allocation 445 includes the frequency spectrum currently allocated to the access network 103, along with the available frequency spectrum that can be allocated to the access network 103. The frequency spectrum can include, without limitation, publicly accessible spectrum, spectrum personally owned or leased by the customer, spectrum owned or leased by the provider, spectrum that can be used for free but requires a reservation, etc.

[0097] Device data 448 corresponds to data that describes client device 106 for which access to access network 103 is permitted. Client device 106 may be associated with a specific account with cloud provider network 203. This device data 448 includes corresponding user, account information, billing information, data plan 440, permitted applications or usage, indication of whether client device 106 is mobile or stationary, location, current cell, network address, device identifier 464 (e.g., International Mobile Equipment Identity (IMEI) number, International Mobile Subscriber Identity (IMSI) number, Equipment Serial Number (ESN), Media Access Control (MAC) address, Subscriber Identity Module (SIM) number, embedded SIM (eSIM) number, etc.), and device priority 465 for capacity-limited planning, etc.

[0098] Priority group 450 may define one or more groups or classes of client devices 106 that have a common priority for capacity-limited planning. In other words, capacity-limited rule set 452 may treat client device 106 in the same way as when it is within the same priority group 450. For example, the type of client device 106 (e.g., security video camera) may be in the same priority group 450, and client device 106 associated with the user's class (e.g., corporate officer) may be in the same priority group 450.

[0099] The capacity limit rule set 452 includes one or more customized rules that control access to the network function 163 when the capacity limit of the network function 163 is reached. As used herein, the capacity limit can be an absolute limit beyond which a service cannot be provided, or the capacity limit can constitute a threshold associated with unacceptable service characteristics, such as unacceptable latency or unacceptable reliability, beyond which the service is associated. The capacity limit rule set 452 can be customer-specific or can be associated with a particular account with the cloud provider network 203. Different network functions 163 can be associated with different capacity limit rule sets 452. The capacity limit rule set 452 can complement or replace one or more default rules that would otherwise be applied when the capacity limit is reached within the network function 163. The capacity limit rule set 452 can control which client devices 106 are connected or which types of network traffic are dropped in order to provide network access to higher-priority client devices 106 and / or higher-priority network traffic.

[0100] In some scenarios, the capacity limit rule set 452 may provide conditions under which the network function 163 should be scaled up to extend the capacity limit or scaled down to reduce the capacity limit. Scaling may include adding computing resources (e.g., number of machine instances, processor capacity, memory capacity, network bandwidth, etc.) to the network function 163. For example, when a high-priority device requests a service, there may be no low-priority devices currently receiving service from the network function 163. The solution defined by the capacity limit rule set 452 may be to scale up the network function 163 to add additional capacity rather than interrupting service to any client device 106. Conversely, when current utilization is far below the capacity limit, the capacity limit rule set 452 may provide to scale down the network function 163 to reduce capacity, and accordingly, reduce costs.

[0101] Network slice 454 corresponds to a flow of network traffic designated for one or more specific service quality requirements 466. The flow can correspond to a flow associated with a specific application running on a specific client device 106, all network traffic from a specific client device 106, a flow from all client devices 106 to a specific destination, a flow from a specific client device 106 to a specific destination, etc. In one example, network slice 454 is identified by a source port, source network address, destination port, destination network address, and / or other information. Network slice 454 can be valid for a specific period of time or for a specific amount of data, or network slice 454 can be valid until cancelled or released. In one example, network slice 454 is allocated on demand for a specific application running on client device 106. In some scenarios, network slice 454 has a specific recurring validity period (e.g., from midnight to 5:00 p.m. on weekdays every week), or the service quality requirements 466 for network slice 454 can change based on a recurring period, the current cost level, and / or other factors or events.

[0102] Service quality requirement 466 can correspond to a minimum or maximum bandwidth, a minimum or maximum delay, a minimum or maximum reliability measure, a minimum or maximum signal strength, etc. Service quality requirement 466 can be associated with a corresponding cost level, which can include fixed components, usage-based components, and / or congestion-based components. For example, service quality requirement 466 can be associated with a recurring monthly fixed cost, a per-session or per-megabyte cost, and / or a dynamic cost based on congestion at a cell site or a particular network link. In some cases, a customer can select service quality requirement 466 that provides a high service level. In other cases, however, a customer can select service quality requirement 466 that provides a low cost level but degrades service quality during a particular time or in a particular manner. For example, a customer can select service quality requirement 466 that enables high throughput at night to transmit backup data over the network at low cost and select a lower-priority throughput otherwise.

[0103] Wireless unit configuration data 457 can correspond to the configuration settings of wireless units deployed in access network 103. Such settings can include the frequencies to be used, the protocols to be used, modulation parameters, bandwidth, network routing, and / or backhaul configuration, location, and height for the wireless units, a set of capacity limit rules 452, etc.

[0104] The network function configuration data 463 corresponds to the configuration settings that configure the operations of various network functions 163 for the access network 103. For example, the network function configuration data 463 may include an ability limitation rule set 452 for a specific network function 163. In various embodiments, the network function 163 may be deployed in a VM instance or a container located in a computing device 418 at a cell site, a customer aggregation site, or a data center located remotely from the customer. Non-limiting examples of the network function 163 may include an access and mobility management function, a session management function, a user plane function, a policy control function, an authentication server function, an integrated data management function, an application function, a network exposure function, a network function repository, a network slice selection function, and / or others.

[0105] The client device 406 represents a plurality of client devices 406 that can be coupled to the network 412. The client device 406 may include, for example, a processor-based system such as a computer system. Such a computer system may be embodied in the form of a desktop computer, a laptop computer, a personal digital assistant, a mobile phone, a smartphone, a set-top box, a music player, a web pad, a tablet computer system, a game console, an e-book reader, a smartwatch, a head-mounted display, an audio interface device, or other devices. The client device 406 includes a display including one or more devices such as, for example, a liquid crystal display (LCD) display, a gas plasma-based flat panel display, an organic light emitting diode (OLED) display, an electronic ink (E-ink) display, an LCD projector, or other types of display devices.

[0106] The client device 406 may be configured to execute various applications such as the client application 436 and / or other applications. By executing the client application 436 within the client device 406 and accessing, for example, the network content provided by the computing environment 403 and / or other servers, the user interface on the display can be rendered. For this purpose, the client application 436 may include, for example, a browser, a dedicated application, etc., and the user interface may include a network page, an application screen, etc. In addition to the client application 436, the client device 406 may also be configured to execute applications such as, for example, an e-mail application, a social networking application, a word processor, a spreadsheet, and / or other applications.

[0107] In some embodiments, the spectrum reservation service 410 provides a reservation of the frequency spectrum for a customer's private network. In one scenario, the spectrum reservation service 410 is operated by an entity such as a third party to manage reservations and coexistence in the publicly accessible spectrum. An example of such a spectrum is the Citizens Broadband Radio Service (CBRS). In another scenario, the spectrum reservation service 410 is operated by a telecommunications service provider to sell or sublicense portions of the spectrum owned or licensed by the provider.

[0108] Next, referring to FIG. 5, a flowchart is shown that provides an example of the operation of a portion of the network management service 424 according to various embodiments. It will be understood that the flowchart of FIG. 5 merely provides an example of many different types of functional arrangements that may be employed to implement the operation of the portion of the network management service 424 as described herein. Alternatively, the flowchart of FIG. 5 may be considered to illustrate an example of elements of a method implemented within the computing environment 403 (FIG. 4) according to one or more embodiments.

[0109] Starting from box 503, the network management service 424 receives from a customer who operates or manages the access network 103 (FIG. 4) the specifications of the client devices 106 (FIG. 1A) and their respective device priorities 465 (FIG. 4). The specifications may indicate the corresponding device identifiers 464 (FIG. 4) and / or other information for the client devices 106. For example, the specifications may associate the client devices 106 with a priority group 450 (FIG. 4), which in turn is associated with a particular priority level. In one embodiment, the priority is defined as a numerical value, and a larger numerical value corresponds to a higher relative priority.

[0110] In box 506, the network management service 424 receives the specification of the capacity limit rule set 452 (FIG. 4) from a customer who operates or manages the access network 103. For example, the capacity limit rule set 452 can override or replace the default rules in the network function 163 for dealing with capacity limits, or can define one or more rules for dealing with capacity limits in the network function 163 (FIG. 1C). That is, instead of a randomized failure or disconnection, the capacity limit rule set 452 is used to configure the network function 163 to fail or deny services in a predictable and reliable manner while still providing access to high-priority client devices 106. Further, the capacity limit rule set 452 can include a customized algorithm or method for selecting which client devices 106 are permitted to connect to or continue a service. Such methods can include first-in first-out (FIFO), last-in first-out (LIFO), round-robin or random distribution, and / or other approaches. These approaches can be employed especially when the client devices 106 are client devices of the same priority level.

[0111] In box 509, the network management service 424 configures the network function set (i.e., one or more network functions 163) to implement each rule of the capacity limit rule set 452 instead of one or more default rules. For example, the network management service 424 can push a new configuration file implementing the capacity limit rule set 452 to each network function 163 via the control plane. The network management service 424 can also restart the network functions 163 or, otherwise, reload their configuration settings. Thereafter, the operation of the portion of the network management service 424 ends.

[0112] Referring now to FIG. 6, there is shown a flowchart providing an example of the operation of a portion of network function 163 (FIG. 1C) according to various embodiments. It will be understood that the flowchart of FIG. 6 provides only an example of many different types of functional arrangements that may be employed to implement the operation of a portion of network function 163 as described herein. Alternatively, the flowchart of FIG. 6 may be considered to represent an example of elements of a method implemented within computing environment 403 (FIG. 4) according to one or more embodiments.

[0113] Beginning at block 603, network function 163 receives a service request for a service from a first client device 106 (FIG. 1A). In one example, network function 163 is located in a wireless access network of access network 103. In another example, network function 163 is located in a core network of access network 103. Network function 163 may be hosted on resources that are managed or provided on behalf of a customer by cloud provider network 203 (FIG. 2A), or network function 163 may be hosted on resources within a customer's premises. In some cases, the service request may correspond to an explicit connection request or a request for a network address. In other cases, the request to connect may simply correspond to the use of access network 103 by client device 106 to transmit one or more data packets over access network 103.

[0114] In box 606, the network function 163 determines that the network function 163 has capacity limitations. For example, the network function 163 may have strict limitations on the number of connections or client devices 106 that can be served simultaneously (e.g., 64 or another number) based on the measurement of the resources allocated to the network function 163. In some cases, the limitations may be based on authorization limitations. In other examples, the limitations may correspond to bandwidth limitations, memory limitations, and processor limitations, etc. Alternatively, the network function 163 may determine that the network function 163 is approaching capacity limitations within a defined threshold to take actions to avoid reaching the capacity limitations.

[0115] In box 609, the network function 163 determines the device priority 465 (FIG. 4) of the first client device 106. In some cases, the network function 163 may determine the priority associated with the priority group 450 (FIG. 4) to which the first client device 106 is assigned. For example, the network function 163 may query the database for the device identifier 464 (FIG. 4) to determine the device priority 465. Alternatively, the device priority 465 may be included in a connection request or data packet transmitted by the first client device 106.

[0116] In box 612, the network function 163 may select one or more second client devices 106 that are currently using the network function 163 to disconnect the access. In this example, the first client device 106 has a higher priority than one or more second client devices 106. In other examples, the first client device 106 may have a lower priority and may be denied access. If multiple second client devices 106 have a lower priority than the first client device 106, the network function 163 may select a number of second client devices 106 to the extent necessary to accommodate the first client device 106. The second client device 106 is selected by the capacity limit rule set 452 (Figure 4), and the capacity limit rule set 452 may define a random selection, a round-robin selection, a FIFO selection, a LIFO selection, or another approach.

[0117] In box 615, the network function 163 interrupts the service to the selected second client device(s) 106 based at least in part on the capacity limit rule set 452. For example, the network function 163 may provision the service to the first client device 106 instead of provisioning the service to the second client device(s) 106. This may disconnect the selected second client device(s) 106 from the access network 103, or the selected second client device(s) 106 may be accommodated by other instances of the network function 163. Alternatively, the functionality in the access network 103 may be restricted thanks to interrupting the service from the network function 163, but some network connectivity may still be available.

[0118] In box 618, the network function 163 can provide services to the first client device 106, which can enable the connectivity between the first client device 106 and the access network 103. The priority of the first client device 106 can continue to be taken into account across scheduling, admission control, and resource allocation algorithms. In box 621, the network function 163 can dynamically allocate the network slice 454 (Figure 4) to the first client device 106. For example, the network slice 454 having the corresponding QoS requirement 466 (Figure 4) can be dynamically allocated to the first client device 106 based at least in part on the priority of the first client device 106. However, note that the QoS requirement 466 can be separate from the priority. In one example, the client device 106 can have a relatively low priority but relatively high QoS requirements 466, or the client device 106 can have a relatively high priority but relatively low QoS requirements.

[0119] In box 624, network function 163 may dynamically adjust other network slices 454. For example, if a selected second client device 106 to which access to network function 163 has been disconnected is assigned one or more network slices 454, the resources assigned to network slices 454 within access network 103 may be released and re - assigned for use by client devices 106 connected simultaneously via network function 163. In some cases, network function 163 may dynamically adjust the QoS requirements 466 of network slices 454, at least in part based on priority. In one example, the QoS requirements 466 associated with lower - priority client devices 106 are adjusted to meet the QoS requirements 466 of higher - priority client devices 106. By using device priority 465 along with QoS requirements 466, these techniques not only enable client devices 106 with higher priority to have access to access network 103, but also ensure that client devices 106 with higher priority obtain the resources they need based on QoS requirements 466.

[0120] In box 627, the capabilities of network function 163 can be scaled up or down, at least in part, based on the set of capacity limit rules 452. For example, if the current capabilities are not sufficient to accommodate network slice 454, the capabilities can be increased. Similarly, if the relative priorities and the set of capacity limit rules 452 are such that services to existing client devices 106 are not likely to be disrupted, the capabilities can be scaled up. Conversely, if utilization falls below the threshold amount of capacity limits, the set of capacity limit rules 452 provides the capabilities that are to be scaled down. When scaling the capabilities of network function 163, new machine instances can be launched and assigned to network function 163 within cloud provider network 203. Alternatively, existing computing capabilities can be reassigned to network function 163, instead of customer workloads, or to a different network function 163, either at the edge or within the core network. Thereafter, the operation of the portion of network function 163 ends.

[0121] Referring to FIG. 7, a schematic block diagram of a computing environment 403 according to an embodiment of the present disclosure is shown. Computing environment 403 includes one or more computing devices 700. Each computing device 700 includes at least one processor circuit having, for example, a processor 703 and a memory 706, both of which are coupled to a local interface 709. For this purpose, each computing device 700 can include, for example, at least one server computer or similar device. Local interface 709 can include, for example, a data bus or other bus structure including an associated address / control bus so as to be recognizable.

[0122] Stored in the memory 706 are both data and several components executable by the processor 703. In particular, stored in the memory 706 and executable by the processor 703 are the network management API 423, the network management service 424, and potentially other applications. Also, stored in the memory 706 may be the data store 415 and other data. Additionally, the operating system may be stored in the memory 706 and executable by the processor 703.

[0123] It will be understood that there may be other applications stored in the memory 706 and executable by the processor 703 so as to be recognizable. If any component described herein is implemented in the form of software, it may employ any one of several programming languages such as C, C++, C#, Objective C, Java®, JavaScript®, Perl, PHP, Visual Basic®, Python®, Ruby, Flash®, or other programming languages.

[0124] Several software components are stored in memory 706 and are executable by processor 703. In this regard, the term "executable" means a program file in a form that can ultimately be launched by processor 703. Examples of executable programs can be, for example, a compiled program that can be loaded into the random access portion of memory 706 and can be converted into machine code in a format that can be launched by processor 703, source code that can be represented in a suitable format such as object code that can be loaded into the random access portion of memory 706 and can be launched by processor 703, or source code that can be interpreted by another executable program that generates instructions in the random access portion of memory 706 so as to be executed by processor 703. The executable program can be stored in any part or component of memory 706, including, for example, random access memory (RAM), read only memory (ROM), hard drive, solid state drive, USB flash drive, memory card, optical disk such as compact disk (CD) or digital versatile disk (DVD), floppy disk, magnetic tape, or other memory components.

[0125] Memory 706 is defined herein as including both volatile and non-volatile memory and data storage components. Volatile components do not retain data values upon power loss. Non-volatile components retain data upon power loss. Thus, memory 706 can include, for example, random access memory (RAM), read-only memory (ROM), hard disk drives, solid state drives, USB flash drives, memory cards accessed via a memory card reader, floppy disks accessed via an associated floppy disk drive, optical disks accessed via an optical disk drive, magnetic tapes accessed via a suitable tape drive, and / or other memory components, or any combination of two or more of these memory components. Additionally, RAM can include, for example, static random access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM), and other such devices. ROM can include, for example, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other similar memory devices.

[0126] Also, processor 703 can represent multiple processors 703 and / or multiple processor cores, and memory 706 can represent multiple memories 706 operating in parallel processing circuits. In such a case, local interface 709 can be a suitable network that facilitates communication between any two of a number of processors 703, between any processor 703 and any one of memories 706, or between any two of memories 706, etc. Local interface 709 can include additional systems designed to coordinate this communication, including, for example, performing load balancing. Processor 703 can have an electrical structure or some other available structure.

[0127] The network management API 423, network management service 424, and various other systems described herein may be embodied in software or code executed by general-purpose hardware as discussed above, but alternatively, similar ones may also be embodied in dedicated hardware or a combination of software / general-purpose hardware and dedicated hardware. When embodied in dedicated hardware, each may be implemented as a circuit or state machine using any one or combination of a number of techniques. These techniques may include, but are not limited to, discrete logic circuits having logic gates for implementing various logical functions upon application of one or more data signals, application-specific integrated circuits (ASICs) having appropriate logic gates, field programmable gate arrays (FPGAs), or other components. Such techniques are generally well-known to those skilled in the art and thus are not described in detail herein.

[0128] The flowcharts of FIGS. 5-6 illustrate the functionality and operation of an implementation aspect of the network management service 424 and a portion of the network function 163. When embodied in software, each block may represent a module, segment, or portion of code that includes program instructions for implementing a particular logical function(s). The program instructions may be embodied in source code that includes human-readable instruction statements written in a programming language, or in machine code that includes numerical instructions recognizable by an appropriate execution system such as a processor 703 in a computer system or other system. The machine code may be converted from, for example, source code. When embodied in hardware, each block may represent a circuit or a number of interconnected circuits for implementing a particular logical function(s).

[0129] The flowcharts of FIGS. 5-6 show a specific order of execution, but it is understood that the order of execution may be different from the order shown. For example, the order of execution of two or more blocks may be swapped from the order shown. Also, two or more blocks shown consecutively in FIGS. 5-6 may be executed simultaneously, or partially simultaneously. Further, in some embodiments, one or more blocks shown in FIGS. 5-6 may be skipped or omitted. Additionally, for purposes such as improving usefulness, explanation, performance measurement, or providing a clue to problem solving, any number of counters, state variables, warning semaphores, or messages may be added to the logical flow described herein. It is understood that all such variations are within the scope of the present disclosure.

[0130] Also, any logic or application described herein, including a network management API 423 and a network management service 424, including software or code, can be embodied in any non-transitory computer-readable medium used by, or associated with, an instruction execution system such as a processor 703 in a computer system or other system. In this sense, the logic can include, for example, instruction statements that are obtainable from a computer-readable medium and include instructions and declarations executable by an instruction execution system. In the context of the present disclosure, a "computer-readable medium" can be any medium that can include, store, or hold the logic or application described herein used by, or associated with, an instruction execution system.

[0131] A computer-readable medium can include any one of a number of physical media, such as, for example, magnetic, optical, or semiconductor media. More specific examples of suitable computer-readable media would include, but are not limited to, magnetic tape, magnetic floppy disk, magnetic hard drive, memory card, solid state drive, USB flash drive, or optical disk. Also, a computer-readable medium can be a random access memory (RAM) including, for example, static random access memory (SRAM) and dynamic random access memory (DRAM), or magnetic random access memory (MRAM). Additionally, a computer-readable medium can be a read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or other types of memory devices.

[0132] Furthermore, any of the logics or applications described herein that include network management API 423 and network management service 424 can be implemented and constructed in various ways. For example, one or more of the applications described can be implemented as modules or components of a single application. Additionally, one or more of the applications described herein can be executed on a shared computing device or separate computing devices, or combinations thereof. For example, multiple applications described herein can be executed on the same computing device 700 or on multiple computing devices 700 in the same computing environment 403.

[0133] Unless otherwise specified, disjunctive language such as the phrase "at least one of X, Y or Z" is generally understood in context to mean that the item, term, etc. may be either X, Y or Z, or any combination thereof (e.g., X, Y and / or Z). Thus, such disjunctive language is not generally intended, and should not be taken, to imply that a particular embodiment requires the presence of at least one of X, at least one of Y, or at least one of Z.

[0134] Embodiments of the present disclosure may be described by at least the following clauses.

[0135] Clause 1. A system comprising: a wireless network operated by a cloud provider network on behalf of a customer, the wireless network comprising a plurality of network functions; and at least one computing device within the cloud provider network, the at least one computing device receiving from the customer a request to provision the service from the network function set of the plurality of network functions to a first client device instead of provisioning the service to a second client device in response to at least one network function in the network function set having a capacity limit, the capacity limit being an absolute capacity limit with respect to a threshold associated with a characteristic for which the service or beyond the service is not acceptable, the wireless network being configured to configure the network function set within the wireless network to implement the at least one rule instead of a default rule for dealing with the capacity limit.

[0136] Clause 2. The system of Clause 1, wherein the default rule comprises at least one of a round robin algorithm or a first in first out algorithm.

[0137] Clause 3. The system according to any one of Clauses 1 to 2, wherein the request associates a first priority level with the first client device and a second priority level with the second client device, and the first priority level is higher than the second priority level.

[0138] Clause 4. The system according to Clause 3, wherein the first priority level is applied to a first class of client devices and the second priority level is applied to a second class of client devices.

[0139] Clause 5. Instead of the default rule for dealing with the capacity limitation, configuring the network function set to implement the at least one rule causes the network function set to provide the network service to the first client device under the at least one rule, rather than denying the network service to the first client device under the default rule, and interrupting the network service to the second client device. The system according to any one of Clauses 1 to 4.

[0140] Clause 6. Instead of the default rule for dealing with the capacity limitation, configuring the network function set to implement the at least one rule causes the network function set to dynamically allocate a network slice with service quality requirements to the first client device. The system according to any one of Clauses 1 to 5.

[0141] Clause 7. A computer-implemented method, comprising receiving, from a first client device, a request for a service from a wireless network; determining that a network function in the wireless network has a capacity limitation; in response to determining that the network function in the wireless network has the capacity limitation and at least partially based on a rule set specific to the wireless network, interrupting the service from the network function to a second client device; and providing access to the network function to the first client device instead of the second client device.

[0142] Clause 8. The computer-implemented method according to clause 7, further comprising determining a priority associated with the first client device; and determining to provide the service from the network function to the first client device instead of the second client device at least partially based on the priority.

[0143] Clause 9. The computer-implemented method according to clause 8, further comprising receiving, from an operator of the wireless network, a specification of the priority associated with the first client device.

[0144] Clause 10. The computer-implemented method according to any one of clauses 7 to 9, further comprising dynamically allocating the first client device to a network slice in the wireless network at least partially based on the rule set, wherein the network slice is associated with service quality requirements.

[0145] Clause 11. The computer-implemented method according to any one of clauses 7 to 10, further comprising selecting the second client device from a plurality of client devices currently using the network function at least partially based on the rule set.

[0146] Clause 12. The computer-implemented method according to clause 11, wherein the rule set defines rules for dealing with the capacity limitations within the network function, which are different from the default rules of the network function.

[0147] Clause 13. The computer-implemented method according to clause 12, wherein the default rule rejects the service from the network function to the first client device.

[0148] Clause 14. The computer-implemented method according to any one of clauses 7 to 13, further including scaling the computing resources allocated to the network function based at least in part on the rule set and in response to determining that the network function within the wireless network is subject to the capacity limitations.

[0149] Clause 15. The computer-implemented method according to any one of clauses 7 to 14, wherein the wireless network is provisioned for a customer by a cloud provider network, and the rule set is configured by the customer.

[0150] Clause 16. The computer-implemented method according to any one of clauses 7 to 15, wherein the network traffic from the second client device is associated with a higher quality of service parameter than the network traffic from the first client device.

[0151] A non-transitory computer-readable medium storing instructions that, when executed on at least one computing device, cause the at least one computing device to receive, at least, a capacity limitation plan from an operator of a wireless network, the capacity limitation plan including rules defined by the operator that prioritize services from a network function to a first client device over a second client device in response to the network function being under capacity limitation, and configure the network function within the wireless network to implement the rules defined by the operator instead of default rules for dealing with the capacity limitation.

[0152] The non-transitory computer-readable medium of clause 17, wherein the rules defined by the operator define an approach for selecting a second client device from among a plurality of second client devices receiving the service from the network function to interrupt the service from the network function.

[0153] The non-transitory computer-readable medium according to any one of clauses 17-18, wherein when executed, the instructions further cause the at least one computing device to configure the network function to dynamically allocate a network slice to the first client device based at least in part on the rules defined by the operator.

[0154] The non-transitory computer-readable medium according to any one of clauses 17-19, wherein the wireless network includes a radio access network, the network function is implemented within the radio access network, and the capacity limitation corresponds to a maximum number of client devices that can be served simultaneously by the network function.

[0155] It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations described for the purpose of clearly understanding the principles of the present disclosure. Many variations and modifications may be added to the above-described embodiment(s) without substantially departing from the spirit and principles of the present disclosure. It is intended that all such modifications and variations be included herein within the scope of the present disclosure and be protected by the following claims.

Claims

1. 1. A system comprising: a wireless network operated by a cloud provider network on behalf of a customer, the wireless network being equipped with a number of network functions; at least one computing device in the cloud provider network, the at least one computing device comprising at least providing a service to a first client device from a network function set of the plurality of network functions according to at least one rule defined by the customer that establishes a priority of the first client device relative to a second client device; and denying the service to the second client device in response to at least one network function in the network function set being at a capacity limit of a client device, the capacity limit of the client device being an absolute capacity limit for the service beyond which the service cannot be provided to additional client devices; configuring the network feature set in the wireless network to implement the at least one rule defined by the customer in place of a default rule for addressing the absolute capacity limitation; It is configured as follows: The system.

2. The system of claim 1 , wherein the default rules include at least one of a round robin algorithm or a first-in-first-out algorithm.

3. 2. The system of claim 1, wherein a first priority level is associated with the first client device and a second priority level is associated with the second client device, the first priority level being higher than the second priority level.

4. 4. The system of claim 3, wherein the first priority level applies to a first class of client devices and the second priority level applies to a second class of client devices.

5. 2. The system of claim 1, wherein configuring the network capability set to implement the at least one rule in place of the default rule for addressing a capacity limitation of the client device causes the network capability set to provide the network service to the first client device and interrupt the network service to the second client device under the at least one rule, rather than denying network service to the first client device under the default rule.

6. 2. The system of claim 1, wherein configuring the network capability set to implement the at least one rule in place of the default rule for addressing the absolute capacity limitation causes the network capability set to dynamically allocate a network slice having quality of service requirements to the first client device.

7. 2. The system of claim 1, wherein the wireless network includes a radio access network, the network functions are implemented within the radio access network, and the absolute capacity limit corresponds to a maximum number of client devices simultaneously served by the service.

8. The system of claim 1 , wherein network traffic from the first client device is associated with a higher quality of service parameter than network traffic from the second client device.

9. 1. A computer-implemented method, comprising: Operating a wireless network having a plurality of network functions by a cloud provider network on behalf of a customer; providing a service to a first client device from a network function set of the plurality of network functions, in accordance with at least one rule defined by the customer that establishes a priority of the first client device relative to a second client device; and in response to at least one network function in the network function set being at a capacity limit of a client device, denying the service to the second client device, the capacity limit of the client device being an absolute capacity limit for the service beyond which the service cannot be provided to additional client devices; configuring the network feature set in the wireless network to implement the at least one rule defined by the customer in place of a default rule for addressing the absolute capacity limitation; The computer-implemented method comprising:

10. 10. The computer-implemented method of claim 9, wherein the default rules include at least one of a round robin algorithm or a first-in-first-out algorithm.

11. 10. The computer-implemented method of claim 9, wherein a first priority level is associated with the first client device and a second priority level is associated with the second client device, the first priority level being higher than the second priority level.

12. 12. The computer-implemented method of claim 11, wherein the first priority level applies to a first class of client devices and the second priority level applies to a second class of client devices.

13. 10. The computer-implemented method of claim 9, wherein configuring the network capability set to implement the at least one rule in place of the default rule for addressing a capacity limitation of the client device causes the network capability set to provide network service to the first client device and interrupt the network service to the second client device under the at least one rule rather than denying network service to the first client device under the default rule.

14. 10. The computer-implemented method of claim 9, wherein configuring the network capability set to implement the at least one rule in place of the default rule for addressing the absolute capacity limitation causes the network capability set to dynamically allocate a network slice having quality of service requirements to the first client device.

15. A non-transitory computer-readable medium having stored thereon instructions for execution on at least one computing device, the instructions, when executed, causing the at least one computing device to perform at least: providing a service to a first client device from a network function set of a plurality of network functions in a wireless network, in accordance with at least one rule defined by a customer that establishes a priority of the first client device relative to a second client device, denying the service to the second client device in response to at least one network function in the network function set being at a capacity limit of the client device, the capacity limit of the client device being an absolute capacity limit for the service beyond which the service cannot be provided to additional client devices, the wireless network being operated by a cloud provider network on behalf of the customer; configuring said network functions in said wireless network to implement said at least one rule defined by said customer in place of a default rule for addressing said absolute capacity limitation; The non-transitory computer-readable medium.

16. 16. The non-transitory computer-readable medium of claim 15, wherein the default rules include at least one of a round-robin algorithm or a first-in-first-out algorithm.

17. 16. The non-transitory computer-readable medium of claim 15, wherein a first priority level is associated with the first client device and a second priority level is associated with the second client device, the first priority level being higher than the second priority level.

18. 20. The non-transitory computer-readable medium of claim 17, wherein the first priority level applies to a first class of client devices and the second priority level applies to a second class of client devices.

19. 16. The non-transitory computer-readable medium of claim 15, wherein configuring the network capability set to implement the at least one rule in place of the default rule for addressing a capacity limitation of the client device causes the network capability set to provide the network service to the first client device and discontinue the network service to the second client device under the at least one rule rather than denying network service to the first client device under the default rule.

20. 16. The non-transitory computer-readable medium of claim 15, wherein configuring the network capability set to implement the at least one rule in place of the default rule for addressing the absolute capacity limitation causes the network capability set to dynamically allocate a network slice having quality of service requirements to the first client device.

Images