Provisioning locality rules for radio-based networks

Abstract

Various embodiments are disclosed for provisioning locality rules for a radio-based network. In one embodiment, at least one locality rule associated with an organization is accessed. The locality rule requires at least a subset of network traffic for a radio-based network to remain within a particular geographic region. The radio-based network includes a radio access network and an associated core network. A topology for the radio-based network is determined based at least in part on the locality rule. The radio-based network is provisioned or reconfigured for the organization to cause the topology to comply with the at least one locality rule.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to U.S. Patent Application 17 / 544,817, filed December 7, 2021, entitled “Provisioning Radio-Based Networks with Locality Rules”. This application also claims priority and benefit to U.S. Patent Application 17 / 544,825, filed December 7, 2021, entitled “Locality-Based Network Slicing in Radio-Based Networks”. Both applications are incorporated herein by reference in their entirety. Background Technology

[0003] 5G is the fifth-generation technology standard for broadband cellular networks, and is planned to eventually replace the fourth-generation (4G) standard of Long Term Evolution (LTE). 5G technology will significantly increase bandwidth, thus expanding the cellular market beyond smartphones to provide last-mile connectivity for desktops, set-top boxes, laptops, Internet of Things (IoT) devices, and more. Some 5G cells may use spectrum similar to 4G, while others may use millimeter-wave spectrum. Millimeter-wave cells have relatively smaller coverage areas but significantly higher throughput than 4G. Attached Figure Description

[0004] Many aspects of this disclosure can be better understood by referring to the following figures and description. The components in the figures are not necessarily drawn to scale, but rather the emphasis is on clearly illustrating the principles of this disclosure. Furthermore, the same reference numerals in the figures designate corresponding portions throughout multiple views.

[0005] FIG. 1A This is a diagram illustrating examples of communication networks deployed and managed according to various implementation schemes of this disclosure.

[0006] FIG. 1B Examples of radio-based networks used on an organizational campus with multiple buildings and deployed according to various embodiments of this disclosure.

[0007] FIG. 2A Examples of networking environments according to some embodiments of the present disclosure, including a cloud provider network and various provider underlying extensions of the cloud provider network, are shown, said networking environments being able to FIG. 1A It is used in various locations within the communication network.

[0008] FIG. 2B Some embodiments according to this disclosure are described. FIG. 1A Examples of the cellularization and geographical distribution of communication networks.

[0009] FIG. 3 Some embodiments according to this disclosure are shown. FIG. 2A Examples of networked environments, which include geographically dispersed provider underlying extensions.

[0010] FIG. 4 It is based on the various implementation schemes of this disclosure. FIG. 2A A schematic block diagram of a networked environment.

[0011] FIG. 5 This illustrates various embodiments of the present disclosure. FIG. 4 A flowchart illustrating an example of a function implemented as part of a radio-based network management service in a networked computing environment.

[0012] FIG. 6A , FIG. 6B and FIG. 7 This illustrates the implementation of various embodiments according to this disclosure as follows: FIG. 4 A flowchart illustrating examples of the functionality of various parts of a local orchestration service executed in a computing environment within a networked environment.

[0013] FIG. 8 Based on the various embodiments provided in this disclosure FIG. 4 A schematic block diagram illustrating an exemplary computing environment used in a networked environment. Detailed Implementation

[0014] This disclosure relates to implementing localization rules for network topologies in radio-based networks, such as 4G, 5G, and sixth-generation (6G) radio-based networks that are at least partially implemented using cloud provider network infrastructure. The radio-based network may include a radio access network (RAN) and an associated core network. The radio-based network can be deployed on behalf of customers using cloud provider network infrastructure in an at least partially automated manner. A customer may correspond to an organization or enterprise with a large number of users and user devices receiving services from the radio-based network. In some scenarios, the customer's radio-based network may be a private network limited to use only by the customer's devices. In other scenarios, the customer's radio-based network may be open to external users.

[0015] A customer located in a certain geographic region can have a specific data locality requirement to limit data access or data transfer to only that geographic region. In some implementations, the geographic region can be a single jurisdiction (e.g., one country or state) or a group of jurisdictions (e.g., the European Union). Such data locality requirements can be driven by data sovereignty, export controls, privacy regulations (e.g., the General Data Protection Regulation (GDPR) of the European Union), security considerations, or other reasons. For example, an operator of a radio-based network within a particular country can offer a restricted service that allows users to call each other over a secure line that guarantees that network traffic never leaves that particular country. To provide such a service, the entire network topology used to make the call (including communication links, data storage services, and network functions) would need to be located in the particular country.

[0016] Various embodiments of the present disclosure introduce the automatic creation of a radio-based network or a radio-based network slice that satisfies network topology locality rules. A network topology locality rule is a rule that specifies a restriction on the geographic region in which the physical network infrastructure used to create the radio-based network or network slice can be located. The present disclosure enables a customer to specify network topology locality rules for a radio-based network, network slice, or application that uses a radio-based network, e.g., via one or more APIs. The customer can indicate that all network traffic or certain types of network traffic (e.g., voice calls, text messages, and / or data) need to remain within a specified geographic region. The service can automatically restrict the deployment of network functions that operate on such network traffic to the specified geographic region. In some cases, the specified geographic region can correspond to a geographic region with predefined boundaries (e.g., a country, a state, a ZIP code), while in other cases, the customer can define a geographic region with arbitrary boundaries (e.g., a corporate campus). Thus, the components of the radio-based network are automatically deployed and provisioned such that the communication links, data storage, and network functions are located within the specified geographic region to support the customer’s locality requirements. In some scenarios, portions of the core network are deployed on a cloud provider network that has resources both inside and outside of the specified geographic region. To satisfy the customer’s locality requirements, the resources within the specified geographic region in the cloud provider network are selected for the customer’s radio-based network or network slice.

[0017] Prior radio-based network deployments relied on manual deployment and configuration at each step of the process. This proved to be extremely time consuming and expensive. Furthermore, in previous generations, the software was inherently tied to vendor-specific hardware, preventing customers from deploying alternative software. In contrast, for 5G, the hardware is decoupled from the software stack, allowing for greater flexibility and allowing components of the radio-based network to execute on cloud provider infrastructure. Using a cloud distribution model for radio-based networks, such as 5G networks, can facilitate handling network traffic from hundreds to billions of connected devices and compute-intensive applications while delivering faster speeds, lower latency, and greater capacity compared to other types of networks.

[0018] Historically, businesses had to choose between performance and price when evaluating their enterprise connectivity solutions. Cellular networks can provide high performance, good indoor and outdoor coverage, and advanced Quality of Service (QoS) connectivity features, but dedicated cellular networks can be expensive and complex to manage. While Ethernet and Wi-Fi require less upfront investment and are easier to manage, businesses often find them less reliable, require significant effort to get optimal coverage, and do not provide QoS features such as guaranteed bit rates, latency, and reliability.

[0019] The disclosed radio-based network service can provide businesses with the best of both worlds - the performance, coverage, and QoS of a telecom-grade cellular network, and the ease of deployment and operation and cost associated with Wi-Fi. The disclosed service can provide suitable hardware in different form factors that businesses can deploy to their sites and integrate with software that runs the entire network, from small cell sites to Internet distribution points. Businesses can freely deploy a variety of 5G devices and sensors across their enterprise (factory floors, warehouses, halls, and communication centers) and manage these devices, register users, and allocate QoS through a management console. With the disclosed technology, customers can allocate constant bit rate throughput for all their devices (e.g., cameras, sensors, or IoT devices), provide reliable low-latency connectivity for devices running on factory floors, and allocate broadband connectivity for all handheld devices. The disclosed service can manage all the software needed to provide connectivity that meets specified constraints and requirements. This enables a whole new class of applications with strict QoS or high IoT device density requirements that could not traditionally run on Wi-Fi networks.

[0020] The disclosed service supports multiple deployment scenarios. In a cloud-only deployment, the service can provide small radio cells that an enterprise customer can place on-premise, while network functions and other network software run in the nearest cloud provider availability zone or edge location (or one of several nearest cloud provider availability zones or edge locations). These cloud provider locations can be selected based at least in part on the network topology locality rules described herein. For enterprises that want to deploy on-premise, the disclosed service provides cloud provider hardware, such as the underlying extensions described herein. In this mode, the network and applications remain on-premise, enabling the enterprise to securely store and process data that needs to remain local (e.g., for regulatory compliance, security considerations, etc.). Further, the disclosed service can allow any compute and storage that is not used to run the radio-based network to be used to run any local workloads via the same APIs that customers can use to run workloads in traditional cloud provider zones. Advantageously, as a result, the enterprise does not have to worry about over-provisioning and wasting capacity, as the service will enable any excess capacity to be used for local processing and will provision new hardware and software as network demand changes. Further, the disclosed service can provide application development APIs that expose and manage 5G capabilities such as QoS, enabling customers to build applications that can take full advantage of the latency and bandwidth capabilities of their network without needing to understand the details of the network.

[0021] Additionally, the disclosed service can provide a dedicated zone to run local applications within the cloud provider network. This dedicated zone can be connected to and made an effective part of the broader regional zones and allow customers to manage the dedicated zone using the same APIs and tools used in the cloud provider network. Like availability zones, a dedicated zone can be assigned a virtual private network subnet. APIs can be used to create subnets and assign them to all zones that a customer wants to use, including the dedicated zone and other existing zones. A management console can provide a simplified process to create a dedicated zone. Virtual machine instances and containers can be launched in the dedicated zone just like in the regional zones. Customers can configure network gateways to define routes, assign IP addresses, set up network address translation (NAT), etc. Auto-scaling can be used to scale the capacity of virtual machine instances or containers in the dedicated zone as needed. The same management and authentication APIs of the cloud provider network can be used within the dedicated zone. In some cases, cloud services available in the regional zones can be accessed without upgrading or modifying the local deployment since they can be accessed remotely from the dedicated zone through a secure connection.

[0022] Various embodiments of the present disclosure provide methods that allow customers to order and deploy radio-based networks and associated core networks in an automated manner. Customers can include enterprises and organizations that wish to establish a radio-based network for internal use (e.g., a private 5G network). Through various user interfaces, customers can specify their network plans or requirements (e.g., physical site layout and device / application types and quantities and network topology locality rules), and various components needed to implement a radio-based network for the customer can be automatically determined and provisioned. Hardware such as antennas, radios, and computer servers can be pre-configured for the customer’s radio-based network and shipped to the customer. The process of installing the pre-configured hardware is largely plug-and-play, and the radio-based network can be activated through a user interface or API. In addition to deploying a radio-based network (such as all or a portion of a new radio access network), various embodiments of the present disclosure can facilitate modification and management of a radio-based network, including deployment of pre-configured equipment for additional cells and allocation of QoS constraints for particular devices or applications on their radio-based network.

[0023] Various embodiments of the present disclosure can also introduce the concepts of elasticity and utility computing from cloud computing models to radio-based networks and associated core networks. For example, the disclosed technology can run core and radio access network functions and associated control plane management functions on a cloud provider infrastructure, creating a cloud-native core network and / or a cloud-native radio access network (RAN). In some implementations, such core and RAN network functions can be based on Third Generation Partnership Project (3GPP) specifications. By providing a cloud-native radio-based network, customers can dynamically scale their radio-based network based on utilization, latency requirements, and / or other factors. In some cases, the hardware shipped to the customer includes sufficient capacity to run programs for operating and managing the radio-based network as well as the customer’s other workloads (e.g., their applications), such that any capacity not used for the radio-based network is accessible to run workloads under a utility computing model. Advantageously, the customer’s radio-based network can scale to such excess capacity as needed, for example, allowing for increased hardware usage requirements for the radio-based network even before new physical hardware is provisioned for the customer. The customer can also configure thresholds to receive alerts related to radio-based network usage and excess capacity usage of the infrastructure provisioned for them, in order to more effectively manage provisioning of new infrastructure or de-provisioning of existing infrastructure based on their dynamic network and workload requirements.

[0024] A network slice is a capability that enables the deployment and operation of multiple logical networks on a common physical network infrastructure in a way that each logical network (i.e., network slice) can be customized and sized to best meet a specific set of requirements. Typically, network slices are created manually and provisioned to a specific organization or business entity. In accordance with the present disclosure, a software application running on a radio-based network can make an API request to a service for a network slice that meets a set of QoS constraints and / or network topology locality rules provided for the application. In response, the service can automatically provision such a network slice for use by network traffic associated with the application. A network slice can reserve a certain amount of different hardware resources (e.g., radio resources, RAN and core processing resources) throughout the network for use by traffic associated with a particular application to achieve a required QoS. The service can also manage such network slices at scale across a large number of different software applications, for example, by provisioning “complementary” slices (which have complementary requirements across a set of different hardware components) on the same underlying hardware to achieve more efficient resource utilization, or by over-provisioning slices based on predicted utilization that indicates that all applicable QoS constraints can still be met.

[0025] As will be appreciated by those skilled in the art in light of the disclosure, certain embodiments can be capable of achieving certain advantages, including some or all of the following: (1) improving user experience by allowing an organization to deploy its own radio-based network in a highly automated, plug-and-play fashion; (2) increasing flexibility of computer systems by allowing computing hardware previously dedicated to network functions in a radio-based network and associated core network to be repurposed for other applications; (3) increasing flexibility of computer systems by allowing computing hardware previously dedicated to a first radio-based network to be automatically repurposed for a second radio-based network (or repurposed between RAN and core network functions of the same radio-based network as needed); (4) improving user experience in deploying a radio-based network by preconfiguring antennas, radios, and other hardware, thereby providing a plug-and-play installation experience; (5) improving performance of a radio-based network by optimizing cell deployment and spectrum utilization; (6) improving performance and management of a radio-based network by monitoring performance metrics and adding, removing, and reconfiguring cells as needed to maintain acceptable performance; (7) improving scalability and overall performance of a radio-based network by offloading network functions previously provided by proprietary hardware to virtual machine instances flexibly operated by a cloud computing provider under a utility computing model; (8) reducing latency in a radio-based network by offloading network functions to virtual machine instances executing on computing devices of a cloud service provider at a cell site; (9) increasing flexibility of a communication network by allowing network topology for a radio-based network to be automatically determined based at least in part on data locality requirements; (10) increasing flexibility of a communication network by allowing network topology for a network slice of a radio-based network to be automatically determined based at least in part on data locality requirements; and so on.

[0026] One of the benefits of the present disclosure is the ability to deploy and link network functions together to deliver end-to-end services that meet specified constraints and requirements. According to the present disclosure, network functions organized into microservices work together to provide end-to-end connectivity. One set of network functions is part of the radio network, running in cell towers and performing the conversion of wireless signals to IP. Other network functions run in large data centers, performing subscriber-related business logic and routing IP traffic to and from the Internet. For applications using 5G new capabilities such as low latency communication and reserved bandwidth, these two types of network functions need to work in concert to properly schedule and reserve wireless spectrum, and perform real-time computation and data processing. The presently disclosed technology provides edge-located hardware (as described further below) that integrates with network functions running across the entire network from cell sites to Internet breakout points, and orchestrates the network functions to meet the required quality of service (QoS) constraints. This enables a whole new set of applications with strict QoS requirements, from factory-based Internet of Things (IoT) to augmented reality (AR), virtual reality (VR), game streaming, autonomous navigation support for connected vehicles, which were previously not possible to run on mobile networks.

[0027] The described "elastic 5G" service provides and manages all the hardware, software, and network functions needed to build a network. In some embodiments, network functions can be developed and managed by cloud service providers; however, the described control plane can manage a range of providers' network functions so that customers can use a single set of APIs to invoke and manage their choices of network functions on cloud infrastructure. The elastic 5G service advantageously automatically creates an end-to-end 5G network from hardware to network functions, reducing the time to deploy the network and the operational cost to operate the network. By providing APIs that expose network capabilities, the disclosed elastic 5G service enables applications to simply specify the required QoS as constraints, and then deploy and link network functions together to deliver end-to-end services that meet the specified requirements, making it possible to easily build new applications.

[0028] The present disclosure describes embodiments related to the creation and management of cloud-native 5G core and / or cloud-native 5G RAN, and associated control plane components. Cloud-native refers to an approach to building and running applications that takes advantage of the benefits of cloud computing distribution models, such as dynamic scalability, distributed computing, and high availability (including geographic distribution, redundancy, and failover). Cloud-native refers to how these applications are created and deployed to fit in a public cloud. While cloud-native applications can (and often do) run in a public cloud, they can also run in on-premises data centers. Some cloud-native applications can be containerized, e.g., packaging different parts, functions, or sub-units of an application in their own containers that can be dynamically orchestrated for active scheduling and management of each part to optimize resource utilization. These containerized applications can be built using a microservices architecture to improve the overall agility and maintainability of the application.

[0029] In a microservices architecture, an application is arranged as a series of smaller sub-units (“microservices”) that can be deployed and scaled independently of one another and that can communicate with one another over a network. These microservices are typically fine-grained in that they have a specific technical and functional granularity and typically implement lightweight communication protocols. The microservices of an application can perform different functions from one another, can be deployed independently, and can use different programming languages, databases, and hardware / software environments from one another. Breaking an application into smaller services facilitates improved modularity of the application, enables on-demand replacement of individual microservices, and parallelizes development by enabling teams to develop, deploy, and maintain their microservices independently of one another. In certain examples, microservices can be deployed using virtual machines, containers, or serverless functions. The disclosed core and RAN software can follow a microservices architecture such that the described radio-based network is composed of independent sub-units that can be deployed and scaled on demand.

[0030] Turning now to FIG. 1A illustrates an example of a communication network 100 deployed and managed in accordance with various embodiments of the present disclosure. The communication network 100 includes a radio-based network 103, which can correspond to a cellular network such as a fourth generation (4G) Long-Term Evolution (LTE) network, a fifth generation (5G) network, a 4G-5G hybrid core with 4G and 5G RANs, or another network that provides wireless network access. The radio-based network 103 can be operated by a cloud services provider for an enterprise, a non-profit organization, a school system, a government entity, or other organization. Although referred to as a private network, the radio-based network 103 can use private network addresses or public network addresses in various embodiments.

[0031] Various deployments of the radio-based network 103 can include one or more of a core network and a RAN network, as well as control plane for running the core and / or RAN networks on cloud provider infrastructure. As noted above, these components can be developed in a cloud-native manner, e.g., using a microservices architecture, such that centralized control and distributed processing are used to efficiently scale traffic and transactions. These components can be based on 3GPP specifications, following a Control Plane and User Plane Separation Application Architecture (CUPS Architecture).

[0032] The radio-based network 103 provides wireless network access to a plurality of wireless devices 106, which can be mobile devices or fixed location devices. In various examples, the wireless devices 106 can include smartphones, connected vehicles, IoT devices, sensors, machines such as in a manufacturing facility, hotspots, and other devices. Such wireless devices 106 are sometimes referred to as user equipment (UE) or customer premises equipment (CPE).

[0033] The radio-based network 103 can include a radio access network (RAN) that provides wireless network access to a plurality of wireless devices 106 through a plurality of cells 109. Each cell 109 can be equipped with one or more antennas and one or more radio units that transmit wireless data signals to and receive wireless data signals from the wireless devices 106. The antennas can be configured for one or more frequency bands, and the radio units can also be frequency agile or frequency tunable. The antennas can be associated with a particular gain or beamwidth in order to focus signals in a particular direction or azimuth range, potentially allowing reuse of frequencies in different directions. Further, the antennas can be horizontally polarized, vertically polarized, or circularly polarized. In some examples, the radio units can utilize multiple-input multiple-output (MIMO) technology to transmit and receive signals. In this way, the RAN implements a radio access technology to enable radio connections with the wireless devices 106 and provide connectivity to the core network of the radio-based network. Components of the RAN include base stations and antennas that cover a given physical area, as well as core network items needed to manage connections with the RAN.

[0034] Data traffic is typically routed through a fiber transport network consisting of multiple hops by Layer 3 routers, for example, at aggregation sites, to a core network. The core network is typically housed in one or more data centers. The core network typically aggregates data traffic from end devices, authenticates subscribers and devices, applies individualized policies, and manages mobility of devices before routing the traffic to carrier services or the Internet. For example, a 5G core can be decomposed into multiple microservice elements with control plane and user plane separation. The 5G core can include virtualized, software-based network functions, for example, deployed as microservices, rather than physical network elements, and thus can be instantiated in a Multi-Access Edge Computing (MEC) cloud infrastructure. Network functions of the core network can include a User Plane Function (UPF), an Access and Mobility Management Function (AMF), and a Session Management Function (SMF), which will be described in more detail below. For data traffic destined to locations outside of the communication network 100, the network functions typically include a firewall through which traffic can pass to or from the communication network 100 to external networks, such as the Internet or a cloud provider network. Note that in some embodiments, the communication network 100 can include facilities that allow traffic to enter or exit from sites further downstream from the core network, for example, at aggregation sites or the radio-based network 103.

[0035] The UPF provides a point of interconnection between the mobile infrastructure and a data network (DN), i.e., encapsulation and decapsulation of the General Packet Radio Service (GPRS) Tunneling Protocol for User Plane (GTP-U). The UPF can also provide a session anchor for mobility within the RAN, including sending one or more end marker packets to a RAN base station. The UPF can also handle packet routing and forwarding, including directing traffic to specific data networks based on traffic matching filters. Another feature of the UPF includes per-flow or per-application QoS handling, including transport level packet marking for uplink (UL) and downlink (DL), and rate limitation. The UPF can be implemented as a cloud-native network function using modern microservice methods, for example, deployable within a serverless framework (abstracting the underlying infrastructure on which the code runs via a managed service).

[0036] The AMF can receive connection and session information from the wireless device 106 or the RAN and can handle connection and mobility management tasks. For example, the AMF can manage handovers between base stations in the RAN. In some examples, by terminating certain RAN control plane and wireless device 106 traffic, the AMF can be considered an access point of the 5G core. The AMF can also implement encryption and integrity protection algorithms.

[0037] The SMF can handle session establishment or modification, for example, by creating, updating, and deleting protocol data unit (PDU) sessions and managing session contexts within UPFs. The SMF can also implement dynamic host configuration protocol (DHCP) and IP address management (IPAM). The SMF can be implemented using modern microservices approaches as a cloud-native network function.

[0038] Various network functions implementing the radio-based network 103 can be deployed in distributed computing devices 112, which can correspond to general-purpose computing devices configured to perform network functions. For example, the distributed computing devices 112 can execute one or more virtual machine instances, which in turn are configured to execute one or more services performing network functions. In one embodiment, the distributed computing devices 112 are ruggedized machines deployed at each cell site.

[0039] In contrast, one or more centralized computing devices 115 can perform various network functions at a central site operated by the customer. For example, the centralized computing devices 115 can be located at a central for the customer’s premises in a conditioned server room. The centralized computing devices 115 can execute one or more virtual machine instances, which in turn are configured to execute one or more services performing network functions.

[0040] In one or more embodiments, network traffic from the radio-based network 103 is backhauled to one or more core computing devices 118, which can be located at one or more data centers remote from the customer’s sites. The core computing devices 118 can also perform various network functions, including routing network traffic to and from a network 121, which can correspond to the Internet and / or other external public or private networks. The core computing devices 118 can perform functionality related to management of the communication network 100 (e.g., billing, mobility management, etc.) as well as transport functionality to relay traffic between the communication network 100 and other networks.

[0041] Moving to FIG. 1B FIG. 1 shows an example of a radio-based network 150 deployed on an organization campus having multiple buildings 153, such as a premises of an enterprise, such as a company, a school, or other organization, and using and in accordance with various embodiments of the present disclosure. Although FIG. 1B While an example with multiple buildings is depicted, it should be understood that the disclosed technology can be similarly applied to any layout of a site, which can include one or more buildings and / or one or more outdoor spaces, such as a stadium or other outdoor venue.

[0042] The radio-based network 150 in this non-limiting example includes four cells 156a, 156b, 156c, and 156d to fully cover the organization campus. The cells 156 can slightly overlap in order to provide exhaustive coverage within each building 153. Adjacent or overlapping cells 156 are configured to operate at non-interfering frequencies. For example, cell 156a can use frequency A, cell 156b can use frequency B, and cell 156c can use frequency C, which are all different frequencies when the coverage of the respective cells 156a, 156b, and 156c overlap. However, cell 156d can use, for example, frequency A or B because the coverage of cell 156d does not overlap with cell 156a or 156b.

[0043] Note that cells 156 can be added to or removed from the radio-based network 150 depending on usage or other network metrics. In some cases, the signal strength to a cell 156 can be increased to reduce the number of cells 156, or decreased to increase the number of cells 156, while allowing for spectrum reuse between cells 156. Additionally, computing capacity can be added within the geographic area of the organization campus or within the cloud provider network in order to reduce latency of the radio-based network 150, maintain security, and increase reliability as needed. In some cases, the computing capacity to implement the software of the radio-based network 150 can be mostly or entirely provisioned in the cloud provider network rather than at the customer’s premises, such as in a regional data center of a cloud service provider. This software can implement various network functions, such as UPFs, AMFs, SMFs, etc., which can correspond to core network functions, central unit network functions, and distributed unit network functions. Some network functions, such as distributed unit network functions, can remain at cell sites.

[0044] FIG. 2A An example of a networking environment 200 that includes a cloud provider network 203 and also includes various provider substrate extensions of the cloud provider network is shown in accordance with some embodiments, which can be used in conjunction with a local customer deployment within a communication network 100 of FIG. 1A The cloud provider network 203 (sometimes simply “the cloud”) refers to a pool of network-accessible computing resources (such as computing, storage, and networking resources, applications, and services), which can be virtualized or bare-metal. The cloud can provide convenient, on-demand network access to a shared pool of configurable computing resources that can be programmatically provisioned and released in response to customer commands. These resources can be dynamically provisioned and reconfigured to adjust to variable load. Thus, cloud computing can be viewed as the delivery of applications as services over a publicly accessible network (e.g., the Internet, a cellular communication network) and the hardware and software in the cloud provider data centers that provide those services.

[0045] The cloud provider network 203 can provide users, through a network, an on-demand scalable computing platform, e.g., allowing users to have scalable “virtual computing devices” at their disposal via which they use computing servers (providing compute instances using one or both of central processing units (CPUs) and graphics processing units (GPUs) optionally with local storage) and block storage servers (providing virtualized persistent block storage for specified compute instances) for their use. These virtual computing devices have the attributes of personal computing devices, including hardware (various types of processors, local memory, random access memory (RAM), hard disk, and / or solid state drive (SSD) storage), operating system options, networking capabilities, and preloaded application software. Each virtual computing device can also virtualize its console input and output (e.g., keyboard, display, and mouse). This virtualization allows users to connect to their virtual computing devices using computer applications such as browsers, APIs, software development kits (SDKs), etc., in order to configure and use their virtual computing devices like personal computing devices. Unlike personal computing devices, which have a fixed number of hardware resources available to users, the hardware associated with virtual computing devices can scale up or down depending on the resources needed by the user.

[0046] As indicated above, users can connect to virtualized computing devices and other cloud provider network 203 resources and services via the intermediary network 212 using various interfaces 206 (e.g., APIs) and configure and manage telecommunication networks such as 5G networks. An API refers to an interface and / or communication protocol between a client device 215 and a server, such that if the client makes a request in a predefined format, the client should receive a response in a specific format or cause a defined action to be initiated. In the context of a cloud provider network, APIs provide a gateway for customers to access cloud infrastructure by allowing customers to obtain data from or cause actions within the cloud provider network, thereby enabling the development of applications that interact with resources and services hosted in the cloud provider network. APIs can also enable different services of the cloud provider network to exchange data with each other. Users can choose to deploy their virtual computing systems to provide network-based services for their own use and / or for use by their customers or clients.

[0047] The cloud provider network 203 can include a physical network, referred to as the underlay (e.g., metal plates, cables, rack hardware). The underlay can be considered a network structure that contains the physical hardware that runs the provider network’s services. The underlay can be isolated from the rest of the cloud provider network 203, e.g., it can not be possible to route from underlay network addresses to addresses in the production network that runs the cloud provider’s services, or to customer networks that host customer resources.

[0048] The cloud provider network 203 can also include an overlay network of virtualized computing resources running on an underlay. In at least some embodiments, hypervisors or other devices or processes on the network underlay can use encapsulation protocol technology to encapsulate and route network packets (e.g., client IP packets) between client resource instances on different hosts within the provider network over the network underlay. Encapsulation protocol technology can be used on the network underlay to route encapsulated packets (also referred to as network underlay packets) between endpoints on the network underlay via overlay network paths or routes. The encapsulation protocol technology can be viewed as providing a virtual network topology that overlays the network underlay. As such, network packets can be routed along the underlay network according to constructs in the overlay network (e.g., virtual networks that can be referred to as virtual private clouds (VPCs), port / protocol firewall configurations that can be referred to as security groups). A mapping service (not shown) can coordinate the routing of these network packets. The mapping service can be a distributed lookup service that maps combinations of overlay internet protocol (IP) and network identifiers to underlay IP such that distributed underlay computing devices can look up where to send packets.

[0049] For purposes of illustration, each physical host device (e.g., compute server, block storage server, object storage server, control server) can have an IP address in the underlay network. Hardware virtualization technology can enable multiple operating systems to run concurrently on a host computer, e.g., as virtual machines (VMs) on a compute server. A hypervisor or virtual machine monitor (VMM) on the host assigns the host’s hardware resources among the various VMs on the host and monitors the execution of the VMs. Each VM can be provisioned with one or more IP addresses in the overlay network, and the VMM on the host can be aware of the IP addresses of the VMs on the host. The VMM (and / or other devices or processes on the network underlay) can use encapsulation protocol technology to encapsulate network packets (e.g., client IP packets) and route the network packets between virtualized resources on different hosts within the cloud provider network 203 over the network underlay. Encapsulation protocol technology can be used on the network underlay to route encapsulated packets between endpoints on the network underlay via overlay network paths or routes. The encapsulation protocol technology can be viewed as providing a virtual network topology that overlays the network underlay. The encapsulation protocol technology can include a mapping service that maintains a mapping directory that maps IP overlay addresses (e.g., client- visible IP addresses) to underlay IP addresses (client-invisible IP addresses) that can be accessed by various processes on the cloud provider network 203 for routing packets between endpoints.

[0050] As shown, in various embodiments, the traffic and operations of the cloud provider network infrastructure can be broadly subdivided into two categories: control plane traffic carried over a logical control plane 218 and data plane operations carried over a logical data plane 221. While the data plane 221 represents the movement of user data through the distributed computing system, the control plane 218 represents the movement of control signals through the distributed computing system. The control plane 218 generally includes one or more control plane components or services distributed across and implemented by one or more control servers. Control plane traffic generally includes management operations such as establishing isolated virtual networks for various customers, monitoring resource usage and health, identifying particular hosts or servers to launch a requested compute instance, provisioning additional hardware as needed, and so forth. The data plane 221 includes customer resources (e.g., compute instances, containers, block storage volumes, databases, file storage) implemented on the cloud provider network. Data plane traffic generally includes non-management operations such as transferring data to and from customer resources.

[0051] The control plane components are generally implemented on a separate set of servers from the data plane servers, and control plane traffic and data plane traffic can be sent over separate / different networks. In some embodiments, control plane traffic and data plane traffic can be supported by different protocols. In some embodiments, messages (e.g., packets) sent over the cloud provider network 203 include a flag to indicate whether the traffic is control plane traffic or data plane traffic. In some embodiments, the payload of the traffic can be inspected to determine its type (e.g., control plane or data plane). Other techniques for distinguishing traffic types are possible.

[0052] As shown, the data plane 221 can include one or more compute servers, which can be bare metal (e.g., single-tenant) or can be virtualized by a hypervisor to run multiple VMs (sometimes referred to as “instances”) or microVMs for one or more customers. These compute servers can support virtualized compute services (or “hardware virtualization services”) of the cloud provider network. The virtualized compute services can be part of the control plane 218, allowing customers to issue commands via the interface 206 (e.g., an API) to launch and manage compute instances (e.g., VMs, containers) of their applications. The virtualized compute services can provide virtual compute instances with different compute and / or memory resources. In one embodiment, each of the virtual compute instances can correspond to one of a number of instance types. An instance type can be characterized by its hardware type, compute resources (e.g., number, type, and configuration of CPUs or CPU cores), memory resources (e.g., capacity, type, and configuration of local memory), storage resources (e.g., capacity, type, and configuration of locally accessible storage), network resources (e.g., characteristics of its network interfaces and / or network capabilities), and / or other suitable descriptive characteristics. Using instance type selection functionality, an instance type can be selected for a customer, e.g., based at least in part on input from the customer. For example, a customer can select an instance type from a set of predefined instance types. As another example, a customer can specify desired resources of an instance type and / or requirements of a workload that the instance is to run, and the instance type selection functionality can select an instance type based on such specifications.

[0053] Data plane 221 may also include one or more block storage servers, which may include persistent storage devices for storing customer data volumes and software for managing these volumes. These block storage servers may support managed block storage services on a cloud provider network. The managed block storage service may be part of control plane 218, allowing customers to issue commands via interface 206 (e.g., API) to create and manage volumes of applications running on their compute instances. Block storage servers include one or more servers on which data is stored as blocks. A block is a sequence of bytes or bits, typically containing a certain integer number of records, with a maximum length of block size. Blocked data is typically stored in a data buffer and read or written in whole blocks at a time. Generally, a volume may correspond to a logical collection of data, such as a set of data maintained on behalf of a user. A user volume consists of one or more blocks stored on a block storage server, and the user volume may be considered as a single hard drive ranging in size from, for example, 1 GB to 1 terabyte (TB) or larger. While considered as a single hard drive, it should be understood that a volume may be stored as one or more virtualized devices implemented on one or more underlying physical host devices. A volume can be partitioned several times (e.g., up to 16 times), with each partition hosted by a different host. Volume data can be replicated across multiple devices within a cloud provider's network to provide multiple copies of the volume (where such copies collectively represent the volume on the computing system). Volume replicas in a distributed computing system can beneficially provide automatic failover and recovery, for example, by allowing users access to a primary copy of the volume or a secondary copy of the volume synchronized with the primary copy at the block level, so that failure of the primary or secondary copy does not prevent access to volume information. The primary copy's role can be to facilitate reads and writes on the volume (sometimes referred to as "input / output operations" or simply "I / O operations") and propagate any writes to the secondary copy (preferably synchronously along the I / O path, but asynchronous replication can also be used). The secondary copy can be updated synchronously with the primary copy and provide a seamless transition during failover operations, whereby the secondary copy assumes the role of the primary copy, and the former primary copy is designated as the secondary copy or a new replacement secondary copy is provisioned. While some examples in this document discuss primary and secondary copies, it should be understood that a logical volume can include multiple secondary copies. Compute instances can virtualize their I / O to the volume via clients. The client represents instructions that enable compute instances to connect to remote data volumes (e.g., data volumes stored on physically separate compute devices accessible over a network) and perform I / O operations at the remote data volume. The client can be implemented on an off-board card of a server that includes the processing units (e.g., CPUs or GPUs) of the compute instances.

[0054] The data plane 221 can also include one or more object storage servers, which represent another type of storage device within the cloud provider network. The object storage servers include one or more servers on which data is stored as objects within resources called buckets and can be used to support a managed object storage service of the cloud provider network. Each object typically includes the stored data, variable amounts of metadata that enable various capabilities of the object storage servers with respect to analyzing the stored objects, and a globally unique identifier or key that can be used to retrieve the object. Each bucket is associated with a given user account. Customers can store many objects in their buckets as desired, can write, read, and delete objects in their buckets, and can control access to their buckets and the objects contained therein. Moreover, in implementations with multiple different object storage servers distributed on different regions of the above-described regions, users can select the region (or regions) in which to store a bucket, e.g., to optimize latency. Customers can use buckets to store various types of objects, including machine images that can be used to launch VMs, and snapshots that represent point-in-time views of volume data.

[0055] The provider substrate extensions 224 (“PSEs”) provide resources and services of the cloud provider network 203 within separate networks, such as telecommunications networks, thereby extending the functionality of the cloud provider network 203 to new locations (e.g., for reasons related to latency in communicating with customer devices, legal compliance, security, etc.). In some implementations, the PSEs 224 can be configured to provide capacity for cloud-based workloads to run within the telecommunications networks. In some implementations, the PSEs 224 can be configured to provide core and / or RAN functionality of the telecommunications networks, and can be configured with additional hardware (e.g., radio access hardware). Some implementations can be configured to allow both, e.g., by allowing capacity that is not used for core and / or RAN functionality to be used for running cloud-based workloads.

[0056] As indicated, such provider substrate extensions 224 can include cloud provider network-managed provider substrate extensions 227 (e.g., formed by servers located in facilities managed by the cloud provider separate from those associated with the cloud provider network 203), communications service provider substrate extensions 230 (e.g., formed by servers associated with communications service provider facilities), customer-managed provider substrate extensions 233 (e.g., formed by servers located on-premises at customer or partner facilities), and other possible types of substrate extensions.

[0057] As shown in the example provider substrate extension 224, the provider substrate extension 224 can similarly include a logical separation between a control plane 236 and a data plane 239 that extend the control plane 218 and data plane 221, respectively, of the cloud provider network 203. The provider substrate extension 224 can be pre-configured with an appropriate combination of hardware and software and / or firmware elements, for example, by a cloud provider network operator, to support various types of compute-related resources, and to do so in a manner that reflects the experience of using the cloud provider network. For example, one or more provider substrate extension location servers can be provisioned by the cloud provider for deployment within the provider substrate extension 224. As described above, the cloud provider network 203 can offer a set of predefined instance types, each with different types and amounts of underlying hardware resources. Each instance type can also be offered in various sizes. To enable customers to continue using the same instance types and sizes in the provider substrate extension 224 as they do in the region, the servers can be heterogeneous servers. The heterogeneous servers can support multiple instance sizes of the same type concurrently, and can also be reconfigured to host any instance type whose underlying hardware resources they support. The reconfiguration of the heterogeneous servers can occur on the fly using the available capacity of the servers, i.e., while other VMs are still running and consuming other capacity of the provider substrate extension location servers. This can improve the utilization of compute resources within the edge location by allowing better packing of running instances on the servers, and also provide a seamless experience with respect to instance usage on the cloud provider network 203 and the provider substrate extension 227 managed by the cloud provider.

[0058] The provider substrate extension servers can host one or more compute instances. The compute instances can be VMs, or containers that package code and all of its dependencies, so that applications can run quickly and reliably across compute environments (e.g., including VMs and microVMs). Additionally, the servers can host one or more data volumes, if needed by the customers. In a region of the cloud provider network 203, such volumes can be hosted on dedicated block storage servers. However, due to the possibility of having significantly less capacity at the provider substrate extension 224 than in the region, if the provider substrate extension 224 includes such dedicated block storage servers, it can not be possible to provide the best utilization experience. Therefore, the block storage service can be virtualized in the provider substrate extension 224, such that one of the VMs runs the block storage software and stores the data of the volumes. Similar to the operation of the block storage service in a region of the cloud provider network 203, the volumes within the provider substrate extension 224 can be replicated for durability and availability. The volumes can be provisioned in their own isolated virtual network within the provider substrate extension 224. The compute instances and any volumes together constitute an extension of the data plane 239 of the data plane 221 of the provider network within the provider substrate extension 224.

[0059] In some implementations, servers within the provider substrate extension 224 can host certain local control plane components, e.g., components that enable the provider substrate extension 224 to continue operating in the event of an outage in connectivity back to the cloud provider network 203. Examples of these components include a migration manager that can move compute instances between provider substrate extension servers if needed to maintain availability, and a key-value data store that indicates where volume replicas are located. However, control plane 236 functionality for the provider substrate extension will generally remain in the cloud provider network 203 in order to allow customers to use as much of the resource capacity of the provider substrate extension as possible.

[0060] The migration manager can have a centralized coordination component running in the region, and local controllers running on PSE servers (as well as servers in the cloud provider data center). When a migration is triggered, the centralized coordination component can identify a target edge location and / or target host, while the local controller can coordinate the transfer of data between the source host and the target host. The described movement of resources between hosts in different locations can take one of several forms of migration. Migration refers to moving a virtual machine instance (and / or other resources) between hosts in a cloud computing network or between a host outside the cloud computing network and a host within the cloud. There are different types of migration, including live migration and reboot migration. During reboot migration, a customer experiences an interruption and effective power cycle of their virtual machine instance. For example, a control plane service can coordinate a reboot migration workflow that involves tearing down the current domain on the original host, followed by creating a new domain for the virtual machine instance on the new host. The instance is rebooted by being shut down on the original host and started again on the new host.

[0061] Live migration refers to a process of moving a running virtual machine or application between different physical machines without significantly interrupting the availability of the virtual machine (e.g., end users do not notice downtime of the virtual machine). When a control plane performs a live migration workflow, it can create a new “inactive” domain associated with the instance, while the original domain of the instance continues to run as an “active” domain. The memory of the virtual machine (including any in-memory state of running applications), storage, and network connectivity are transferred from the original host with the active domain to the destination host with the inactive domain. The virtual machine can be briefly paused while the memory contents are transferred to the destination host to prevent state changes. The control plane can transition the inactive domain to become the active domain and demote the original active domain to become the inactive domain (sometimes referred to as “flipping”), after which the inactive domain can be discarded.

[0062] Techniques for various types of migration involve managing a critical phase: the time that the virtual machine instance is unavailable to the customer, which should be kept as short as possible. This can be particularly challenging in the currently disclosed migration techniques, as the resources are being moved between hosts in geographically separated locations that are connectable through one or more intermediate networks. For live migration, the disclosed techniques can dynamically determine the amount of memory state data to pre-copy (e.g., while the instance is still running on the source host) and to post-copy (e.g., after the instance starts running on the destination host), for example, based on latency between locations, network bandwidth / usage patterns, and / or based on which memory pages the instance most frequently uses. In addition, the specific times to transfer memory state data can be dynamically determined based on network conditions between locations. This analysis can be performed by the migration management component in the region, or by the migration management component running locally in the source edge location. If the instance has access to virtualized storage, both the source and target domains can be attached to the storage simultaneously to enable uninterrupted access to its data during migration and in case of a rollback to the source domain.

[0063] The server software running on the provider substrate extension 224 can be designed by the cloud provider to run on the cloud provider substrate network, and the software can be able to run unmodified in the provider substrate extension 224 to create a private copy of the substrate network within the edge location (“shadow substrate”) by using a local network manager 242. The local network manager 242 can run on the provider substrate extension 224 server and bridge the shadow substrate with the provider substrate extension 224 network, for example, by acting as a virtual private network (VPN) endpoint or an endpoint between the provider substrate extension 224 and agents 245, 248 in the cloud provider network 203 and by implementing a mapping service (for traffic encapsulation and decapsulation) to correlate data plane traffic (from data plane agent 248) and control plane traffic (from control plane agent 245) with the appropriate servers. By implementing a local version of the substrate overlay mapping service of the provider network, the local network manager 242 allows resources in the provider substrate extension 224 to communicate seamlessly with resources in the cloud provider network 203. In some implementations, a single local network manager 242 can perform these actions for all servers hosting compute instances in the provider substrate extension 224. In other implementations, each of the servers hosting compute instances can have a dedicated local network manager 242. In a multi-rack edge location, inter-rack communication can be through the local network managers 242, with the local network managers maintaining open tunnels between each other.

[0064] Provider underlay extension locations can utilize secure networking tunnels through the provider underlay extension 224 network to the cloud provider network 203, for example, to maintain the security of customer data as it traverses the provider underlay extension 224 network and any other intermediary networks (which can include the public Internet). Within the cloud provider network 203, these tunnels are composed of virtual infrastructure components including isolated virtual networks (e.g., in an overlay network), control plane agents 245, data plane agents 248, and underlay network interfaces. Such agents 245, 248 can be implemented as containers running on compute instances. In some embodiments, each server in a provider underlay extension 224 location hosting compute instances can utilize at least two tunnels: one tunnel for control plane traffic (e.g., Constrained Application Protocol (CoAP) traffic), and one tunnel for encapsulated data plane traffic. Connectivity managers (not shown) within the cloud provider network 203 manage the cloud provider network side lifecycle of these tunnels, for example, by provisioning them and maintaining them in a healthy operational state automatically when needed. In some embodiments, direct connections between the provider underlay extension 224 locations and the cloud provider network 203 can be used for control and data plane communications. Direct connections can provide constant bandwidth and more consistent network performance compared to VPNs over other networks, as their network paths are relatively fixed and stable.

[0065] A control plane (CP) agent 245 can be provisioned in the cloud provider network 203 to represent a particular host in an edge location. The CP agent 245 is an intermediary between the control plane 218 in the cloud provider network 203 and the control plane target in the control plane 236 of the provider underlay extension 224. That is, the CP agent 245 provides the infrastructure for tunneling management API traffic destined for the provider underlay extension servers from the regional underlay to the provider underlay extension 224. For example, a virtualized compute service of the cloud provider network 203 can issue a command to the VMM of a server of the provider underlay extension 224 to launch a compute instance. The CP agent 245 maintains a tunnel (e.g., VPN) to the local network manager 242 of the provider underlay extension. Software implemented in the CP agent 245 ensures that only well-formed API traffic leaves and returns to the underlay. The CP agent 245 provides a mechanism to expose remote servers on the cloud provider underlay while still protecting underlay security material (e.g., encryption keys, security tokens) from leaving the cloud provider network 203. The one-way control plane traffic tunnel imposed by the CP agent 245 also prevents any (potentially compromised) devices from calling back to the underlay. The CP agent 245 can be instantiated one-to-one with a server at the provider underlay extension 224, or can be able to manage control plane traffic for multiple servers in the same provider underlay extension.

[0066] Data plane (DP) agents 248 can also be provisioned in the cloud provider network 203 to represent particular servers in the provider underlay expansion 224. The DP agents 248 act as shadows or anchors for the servers and can be used by services within the cloud provider network 203 to monitor the health of the hosts (including their availability, used / free compute and capacity, used / free storage and capacity, and network bandwidth usage / availability). The DP agents 248 also allow isolated virtual networks to span the provider underlay expansion 224 and the cloud provider network 203 by acting as proxies for the servers in the cloud provider network 203. Each DP agent 248 can be implemented as a packet forwarding compute instance or container. As shown, each DP agent 248 can maintain a VPN tunnel with a local network manager 242 that manages traffic to the server that the DP agent 248 represents. The tunnel can be used to send data plane traffic between the provider underlay expansion servers and the cloud provider network 203. Data plane traffic flowing between the provider underlay expansion 224 and the cloud provider network 203 can pass through the DP agent 248 associated with the provider underlay expansion 224. For data plane traffic flowing from the provider underlay expansion 224 to the cloud provider network 203, the DP agent 248 can receive the encapsulated data plane traffic, verify its correctness, and allow it to enter the cloud provider network 203. The DP agent 248 can forward the encapsulated traffic from the cloud provider network 203 directly to the provider underlay expansion 224.

[0067] The local network manager 242 can provide secure network connectivity with the proxies 245, 248 established in the cloud provider network 203. After a connection has been established between the local network manager 242 and the proxies 245, 248, a customer can issue commands via the interface 206 to use the provider underlay expansion resources to instantiate (and / or perform other operations using) compute instances in a similar manner to how such commands would be issued for compute instances hosted within the cloud provider network 203. From the customer’s perspective, the customer can now seamlessly use local resources within the provider underlay expansion (as well as resources located in the cloud provider network 203, if needed). Compute instances set up on servers at the provider underlay expansion 224 can communicate with electronic devices located in the same network, as well as other resources set up in the cloud provider network 203 as needed. A local gateway 251 can be implemented to provide network connectivity between the provider underlay expansion 224 and the network associated with the expansion (e.g., the communications service provider network in the example of the communications service provider underlay expansion 230).

[0068] There can be cases where data needs to be transferred between the object storage service and the provider substrate extension (PSE) 224. For example, the object storage service can store machine images used to launch VMs, as well as snapshots representing point-in-time backups of volumes. The object gateway can be provided on a PSE server or a dedicated storage device, and provide customers with configurable per-bucket caching of object storage bucket contents in their PSE 224 to minimize the impact of PSE regional latency on customer workloads. The object gateway can also temporarily store snapshot data from snapshots of volumes in the PSE 224, and then synchronize with the object servers in the region as possible. The object gateway can also store machine images that customers specify for use within the PSE 224 or on customer premises. In some implementations, data within the PSE 224 can be encrypted with unique keys, and the cloud provider can limit sharing of keys from the region to the PSE 224 for security reasons. Accordingly, data exchanged between the object storage servers and the object gateway can utilize encryption, decryption, and / or re-encryption in order to preserve security boundaries with respect to encryption keys or other sensitive data. The transformation intermediary can perform these operations, and can create PSE buckets (on the object storage servers) using PSE encryption keys to store snapshot data and machine image data.

[0069] In the manner described above, the PSE 224 forms an edge location, as it provides resources and services of the cloud provider network 203 outside of a traditional cloud provider data center and closer to customer premises. An edge location as referred to herein can be structured in a variety of ways. In some implementations, an edge location can be an extension of the cloud provider network substrate, including a limited amount of capacity provided outside of an availability zone (e.g., in a small data center of the cloud provider or other facility located close to customer workloads and possibly far from any availability zone). Such an edge location can be referred to as a “far zone” (due to being far from other availability zones) or a “near zone” (due to being close to customer workloads). A near zone can be connected to a publicly accessible network such as the Internet in a variety of ways (e.g., directly, via another network, or via a dedicated connection to a zone). While a near zone typically has more limited capacity as compared to a zone, in some cases a near zone can have considerable capacity, such as thousands or more racks.

[0070] In some implementations, an edge location can be an extension of the cloud provider network underlay by one or more servers located on-premises at a customer or partner facility, where such servers communicate with a nearby availability zone or region of the cloud provider network over a network (e.g., a publicly accessible network such as the Internet). This type of underlay extension located outside of a cloud provider network data center can be referred to as a “outpost” of the cloud provider network. Some outposts can be integrated into a communication network, for example, as a multi-access edge computing (MEC) site whose physical infrastructure is distributed across telecommunication data centers, telecommunication aggregation sites, and / or telecommunication base stations within a telecommunication network. In the local example, the limited capacity of the outpost can be available only to the customer that owns the premises (and any other accounts permitted by the customer). In the telecommunication example, the limited capacity of the outpost can be shared among multiple applications (e.g., games, virtual reality applications, healthcare applications) that send data to users of the telecommunication network.

[0071] An edge location can include data plane capacity that is at least partially controlled by the control plane of a nearby availability zone of the provider network. As such, a group of availability zones can include a “parent” availability zone and any “child” edge locations that are attributed to (e.g., at least partially controlled by the control plane of) the parent availability zone. Certain limited control plane functionality (e.g., features that require low-latency communication with customer resources, and / or features that enable an edge location to continue operating when disconnected from a parent availability zone) can also be present in some edge locations. Thus, in the above example, an edge location refers to an extension of at least data plane capacity located at the edge of the cloud provider network, proximate to customer devices and / or workloads.

[0072] In FIG. 1A the example, the distributed computing devices 112 FIG. 1A , the centralized computing devices 115 FIG. 1A , and the core computing devices 118 FIG. 1A may be implemented as provider underlay extensions 224 of the cloud provider network 203. The installation or positioning of the provider underlay extensions 224 within the communication network 100 can vary depending on the particular network topology or architecture of the communication network 100. The provider underlay extensions 224 can generally be connected to any location within the communication network 100 where packet-based traffic (e.g., IP-based traffic) can be interrupted. Additionally, the communication between a given provider underlay extension 224 and the cloud provider network 203 is generally secured from at least a portion of the communication network 100 (e.g., via a secure tunnel, virtual private network, direct connection, etc.).

[0073] In 5G wireless network development efforts, edge locations can be considered as a possible implementation of Multi-Access Edge Computing (MEC). Such edge locations can be connected to various points in the 5G network that provide breaks for data traffic as part of a User Plane Function (UPF). Older wireless networks can also contain edge locations. For example, in a 3G wireless network, an edge location can be connected to a packet-switched network portion of the communication network 100, such as to a Serving General Packet Radio Service Support Node (SGSN) or to a Gateway General Packet Radio Service Support Node (GGSN). In a 4G wireless network, an edge location can be connected to a Serving Gateway (SGW) or Packet Data Network Gateway (PGW) as part of a core network or Evolved Packet Core (EPC). In some embodiments, traffic between the provider substrate extension 224 and the cloud provider network 203 can be taken out of the communication network 100 without being routed through the core network.

[0074] In some embodiments, the provider substrate extension 224 can be connected to more than one communication network 100 associated with a respective customer. For example, the provider substrate extension 224 can be connected to two communication networks 100 of a respective customer when the networks share or route traffic through a common point. For example, each customer can allocate a certain portion of its network address space to the provider substrate extension, and the provider substrate extension can include a router or gateway that can distinguish between traffic exchanged with each of the communication networks 100. For example, traffic from one network to the provider substrate extension 224 can have a different destination IP address, source IP address, and / or virtual local area network (VLAN) tag than traffic received from the other network. Traffic originating from a destination on one of the networks that is destined for the provider substrate extension can be similarly encapsulated to have the appropriate VLAN tag, source IP address (e.g., from a pool assigned to the provider substrate extension from the destination network address space), and destination IP address.

[0075] FIG. 2B An example 253 of cellularization and geographic distribution of a communication network 100 FIG. 1A ) for providing highly available user plane functions (UPFs) is depicted. In FIG. 2BIn particular embodiments, user devices 254 communicate with request routers 255 to route requests to one of a plurality of control plane cells 257a and 257b. Each control plane cell 257 can include a network service API gateway 260, network slice configuration 262, network service monitoring function 264, site planning data 266 (including layouts describing customer site requirements, device types, device quantities, etc.), network service / function catalog 268, orchestration network function 270, and / or other components. Larger control planes can be divided into multiple cells to reduce the likelihood of large-scale errors affecting a wide range of customers, for example by having one or more cells per customer, per network, or per independently operated region.

[0076] Network service / function catalog 268 is also known as the NF Repository Function (NRF). In a service-based architecture (SBA) 5G network, control plane functions and a common data repository can be delivered through a set of interconnected network functions built using a microservices architecture. The NRF can maintain a record of available NF instances and their supported services, allowing other NF instances to subscribe and be notified of registrations from NF instances of a given type. Thus, the NRF can support service discovery by receiving discovery requests from NF instances and detailed information of which NF instances support particular services. Orchestration network function 270 can perform NF lifecycle management, including instantiation, scale-out / scale-in, performance measurement, event correlation, and termination. Network function orchestrator 270 can also onboard new NFs, manage migration to new or updated versions of existing NFs, identify sets of NFs suitable for a particular network slice or larger network, and orchestrate NFs among different computing devices and sites that make up radio-based network 103.

[0077] Control plane cells 257 can communicate with one or more cell sites 272, one or more customer local data centers 274, one or more local regions 276, and one or more regional regions 278. Cell sites 272 include computing hardware 280 that executes one or more Distributed Unit (DU) network functions 282. Customer local data centers 274 include computing hardware 283 that executes one or more DU or Central Unit (CU) network functions 284, network controllers, UPFs 286, one or more edge applications 287 corresponding to customer workloads, and / or other components.

[0078] The regional zone 276 (which can be located in a data center operated by a cloud service provider) can execute one or more core network functions 288, such as an AMF, an SMF, a network exposure function (NEF) that securely exposes services and capabilities of other network functions, a unified data management (UDM) function that manages subscriber data for authorization, registration, and mobility management. The regional zone 276 can also execute a UPF 286, a metrics processing service 289, and one or more edge applications 287.

[0079] The regional zone 278 (which can be located in a data center operated by a cloud service provider) can execute one or more core network functions 288; a UPF 286; an operations support system (OSS) 290 that supports network management systems, service delivery, service implementation, service assurance, and customer service; an Internet Protocol Multimedia Subsystem (IMS) 291; a business support system (BSS) 292 that supports product management, customer management, revenue management, and / or order management; one or more portal applications 293, and / or other components.

[0080] In the example, the communication network 100 employs a cellular architecture to reduce the blast radius of individual components. At the top level, the control plane is located in multiple control plane cells 257 to prevent individual control plane failures from impacting all deployments.

[0081] Within each control plane cell 257, multiple redundant stacks can be provided, with the control plane diverting traffic to secondary stacks as needed. For example, a cell site 272 can be configured to utilize a nearby regional zone 276 as its default core network. In the event that the regional zone 276 experiences an outage, the control plane can redirect the cell site 272 to use a backup stack in a regional zone 278. Traffic that is typically routed from the Internet to the regional zone 276 can be diverted to an endpoint in the regional zone 278. Each control plane cell 257 can implement a “stateless” architecture that shares a common session database across multiple sites, such as across availability zones or edge sites.

[0082] FIG. 3 A cloud provider infrastructure 220 is shown that includes geographically dispersed provider underlay extensions 224 (e.g., regional zones 276 and regional zones 278) that are connected to a core network 280, according to some embodiments. FIG. 2A)(or“edge locations 303”). As shown, the cloud provider network 203 can be formed into a plurality of regions 306, where a region is a separate geographic area in which the cloud provider has one or more data centers 309. Each region 306 can include two or more availability zones (AZs) that are connected to each other via a dedicated high-speed network such as, for example, fiber-optic communication connections. An availability zone refers to an isolated failure domain that includes one or more data center facilities that have separate power, separate networking, and separate cooling from other availability zones. The cloud provider can work to locate availability zones within a region far enough apart from each other such that a natural disaster, widespread power outage, or other unexpected event does not take more than one availability zone offline at the same time. Customers can connect to resources within an availability zone of the cloud provider network via a publicly accessible network (e.g., the Internet, a cellular communication network, a communication service provider network). Transit centers (TCs) are the primary backbone locations that link customers to the cloud provider network and can be co-located in other network provider facilities (e.g., Internet service providers, telecommunication providers). Each region can operate two or more TCs for redundancy. The regions 306 are connected to a global network that includes private networking infrastructure (e.g., fiber-optic connections controlled by the cloud service provider) that connects each region 306 to at least one other region. The cloud provider network 203 can deliver content from these points of presence (PoPs) that are outside of but networked to the regions 306 through edge locations 303 and regional edge cache servers. This partitioning and geographic distribution of computing hardware enables the cloud provider network 203 to provide low-latency access to resources for customers across the globe with a high degree of fault tolerance and stability.

[0083] The number of edge locations 303 can be much higher compared to the number of regional data centers or availability zones. This widespread deployment of edge locations 303 can provide low-latency connectivity to the cloud for much larger groups of end-user devices than those that happen to be very close to a regional data center. In some embodiments, each edge location 303 can peer to a certain portion of the cloud provider network 203 (e.g., a parent availability zone or regional data center). This peering allows various components operating in the cloud provider network 203 to manage the computing resources of the edge location 303. In some cases, multiple edge locations 303 can be located or installed in the same facility (e.g., separate racks of computer systems) and managed by different zones or data centers to provide additional redundancy. It should be noted that while edge locations 303 are generally depicted herein as being located in communication service provider networks or radio-based networks 103 FIG. 1AIn some cases, such as when the cloud provider network facilities are relatively close to the communication service provider facilities, the edge location 303 can remain within the physical premises of the cloud provider network 203 while connecting to the communication service provider network via fiber or other network links.

[0084] The edge locations 303 can be structured in a variety of ways. In some implementations, the edge locations 303 can be extensions of the cloud provider network infrastructure, including a limited amount of capacity provided outside of availability zones (e.g., in small data centers of the cloud provider or other facilities located close to customer workloads and possibly far from any availability zones). Such edge locations 303 can be referred to as localities (due to being more local or closer to a set of users than traditional availability zones). The localities can be connected to a publicly accessible network such as the Internet in a variety of ways, such as directly, via another network, or via a dedicated connection to the regions 306. While generally the localities have more limited capacity than the regions 306, in some cases the localities can have considerable capacity, e.g., thousands or more racks. Some localities can use similar infrastructure to a typical cloud provider data center, rather than the edge location 303 infrastructure described herein.

[0085] As indicated herein, the cloud provider network 203 can be formed into a plurality of regions 306, where each region 306 represents a geographic area in which the cloud provider clusters data centers. Each region 306 can also include a plurality (e.g., two or more) availability zones (AZs) that are connected to each other via a dedicated high-speed network, e.g., a fiber communication connection. The AZs can provide isolated failure domains including one or more data center facilities with separate power, separate networking, and separate cooling from those in another AZ. Preferably, the AZs within a region 306 are far enough apart from each other such that the same natural disaster (or other failure-inducing event) does not affect more than one AZ at a time or take more than one AZ offline. A customer can connect to the AZs of the cloud provider network via a publicly accessible network, e.g., the Internet, a cellular communication network.

[0086] Given edge locations 303 as parents of AZs or regions 306 of cloud provider network 203 can be based on many factors. One such parent factor is data sovereignty. For example, to keep data originating from a certain country in that country, edge locations 303 deployed in that communication network 100 can be parents of AZs or regions 306 of that country. Another factor is availability of services. For example, some edge locations 303 can have different hardware configurations, such as whether components such as local non-volatile storage for customer data (e.g., solid state drives), graphics accelerators, etc. are present. Some AZs or regions 306 can lack services that take advantage of these additional resources, so edge locations can be parents of AZs or regions 306 that support services that use these resources. Another factor is latency between AZs or regions 306 and edge locations 303. While there are latency benefits to deploying edge locations 303 in a communication network 100, those benefits can be offset by making edge locations 303 parents of distant AZs or regions 306, which introduces significant latency to edge location 303 to region traffic. Thus, edge locations 303 are often parents of nearby (in terms of network latency) AZs or regions 306.

[0087] In some embodiments, a customer can configure one or more geographic regions 312 in one or more locality rules such that data in the customer’s radio-based network 103( FIG. 1A ) needs to be kept within the defined geographic region 312. As shown in FIG. 3 , an example geographic region 312 encompasses a portion of two regions 306, including two data centers 309 and six edge locations 303, but not including two other edge locations 303 in the regions 306. In other words, regions 306, edge locations 303, and communication links outside of the geographic region 312 cannot be used to process, store, or route network traffic that needs to remain within the geographic region 312, so that the portion of the topology corresponding to the portion of the radio-based network 103 that processes network traffic of one or more identified categories can be composed of physical hardware located within the geographic region 312. In other examples, a geographic region 312 can encompass a single region 306 or a portion of a single region 306 or more than two regions 306.

[0088] Referring to FIG. 4The diagram illustrates a networking environment 400 according to various implementation schemes. The networking environment 400 includes a computing environment 403, one or more client devices 406, one or more pre-deployed devices 409, a spectrum reservation service 410, and one or more radio-based networks 103 that communicate with each other via a network 412. The network 412 includes, for example, the Internet, an intranet, an extranet, a wide area network (WAN), a local area network (LAN), a wired network, a wireless network, a cable television network, a satellite network, or other suitable networks, or any combination of two or more such networks.

[0089] Computing environment 403 may include, for example, server computers or any other system providing computing capacity. Optionally, computing environment 403 may employ multiple computing devices, which may be arranged, for example, in one or more server groups or computer groups or other arrangements. Such computing devices may be located in a single device or may be distributed across many different geographical locations. For example, computing environment 403 may include multiple computing devices, which together may include hosted computing resources, grid computing resources, and / or any other distributed computing arrangements. In some cases, computing environment 403 may correspond to elastic computing resources, where the allocated capacity of processing, networking, storage, or other computing-related resources may vary over time. For example, computing environment 403 may correspond to cloud provider network 203 ( FIG. 2A The utility-based computing model charges customers based on their computing resource usage.

[0090] In some implementations, computing environment 403 may correspond to a virtualized private network within a physical network, including, for example, virtual machine instances executed on physical computing hardware via a hypervisor. The virtual machine instances and any containers running on these instances can obtain network connectivity through virtualized network components enabled by physical network components such as routers and switches.

[0091] According to various implementation schemes, various applications and / or other functions can be executed in the computing environment 403. Furthermore, various data are stored in data storage areas 415 accessible to the computing environment 403. It is understood that data storage area 415 may represent multiple data storage areas 415. The data stored in the data storage areas 415 is associated, for example, with the operation of the various applications and / or functional entities described below.

[0092] The computing environment 403, as part of a cloud provider network that provides utility computing services, includes computing devices 418 and other types of computing devices. The computing devices 418 can correspond to different types of computing devices 418 and can have different computing architectures. The computing architectures can differ by using processors with different architectures, such as x86, x86_64, ARM, Scalable Processor Architecture (SPARC), PowerPC, and the like. For example, some computing devices 418 can have x86 processors, while other computing devices 418 can have ARM processors. The computing devices 418 can also differ in terms of available hardware resources, such as local storage, graphics processing units (GPUs), machine learning extensions, and other features.

[0093] The computing devices 418 can have various forms of allocated computing capacity 421, which can include virtual machine (VM) instances, containers, serverless functions, and the like. VM instances can be instantiated from VM images. To this end, a customer can specify that a virtual machine instance should be launched in a particular type of computing device 418, as opposed to other types of computing devices 418. In various examples, one VM instance can be executed alone on a particular computing device 418, or multiple VM instances can be executed on a particular computing device 418. Moreover, a particular computing device 418 can execute different types of VM instances, which can provide different amounts of available resources via the computing device 418. For example, certain types of VM instances can provide more memory and processing power than other types of VM instances.

[0094] For example, components executing on the computing environment 403 include a radio-based network (RBN) management service 424, a network slice allocation service 425, an RBN hardware configuration service 427, an RBN hardware deployment service 430, a locality orchestration service 433, and other applications, services, processes, systems, engines, or functions not discussed in detail herein.

[0095] The RBN management service 424 is executed to manage, configure, and monitor radio-based networks 103 operated on behalf of customers by the cloud service provider. To this end, the RBN management service 424 can generate a plurality of user interfaces that allow a customer to order a new radio-based network 103, scale up or scale down an existing radio-based network 103, modify the operation of an existing radio-based network 103, configure wireless devices 106 that are permitted to use a radio-based network 103 FIG. 1A ), provide statistics and metrics regarding the operation of a radio-based network 103, reserve spectrum for a customer’s private network via the spectrum reservation service 410, allow a customer to define one or more locality rules 434 and one or more geographic regions 312 to which the customer applies FIG. 3), etc. For example, the RBN management service 424 can generate one or more web pages, such as web pages, that include user interfaces. Also, the RBN management service 424 can support this functionality through APIs that can be called by the client application 436. In addition to facilitating interaction with users, the RBN management service 424 also implements orchestration of deployment and configuration changes to the radio-based network 103 and ongoing monitoring of performance parameters. In some cases, the RBN management service 424 can generate network plans 439 for customers based at least in part on specifications for customer locations, automated site surveys for unmanned aerial vehicles, and / or other input parameters.

[0096] The network slice allocation service 425 is executed to allocate network slices to applications and / or client devices 406 that connect to the RAN of the radio-based network 103 with an associated core network. As used herein, the term “network slice” refers to a specific network traffic that is assigned one or more specific locality rules 434, a priority according to one or more quality of service requirements, and / or is provided with a hardware capacity reservation in order to receive, transmit, or manage network traffic. The network traffic of a network slice can be identified at one or more network layers, such as an application layer (e.g., through deep packet inspection), a session layer, a transport layer, a network layer, or a data link layer. A network slice can be ephemeral, or have a specific duration in time or amount of data, or can exist until released or cancelled. The network slice allocation service 425 can support application programming interfaces (APIs) that can be called by applications on the client devices 406 and / or backend services that interact with these applications in order to request that a network slice be allocated, modified, or released. Although the network slice allocation service 425 allocates network slices on the radio-based network 103, there can be one or more devices that are coupled to the radio-based network 103 through one or more fixed or wired links, and network slices determined by the network slice allocation service 425 can also apply to such devices.

[0097] To allocate network slices, the network slice allocation service 425 can dynamically configure one or more network functions in the radio-based network 103 to implement the locality rules 434 and / or quality of service requirements for network traffic that meets the network slice definition. Note that a network slice can have a higher or lower priority than normal traffic, which can have a higher or lower corresponding cost than normal usage cost. In some scenarios, the network slice allocation service 425 can increase or decrease the compute capacity 421 allocated for network function workloads to meet the specified quality of service requirements. For example, allocating more compute capacity 421 for network functions that implement a network slice can provide shorter latency. In some embodiments, the network slice allocation service 425 can also rearrange network function workloads at different points in the radio-based network 103 to meet the quality of service requirements.

[0098] In some embodiments, an application developer or application owner can specify the required network slice configuration in an application template for deploying a particular application in the cloud provider network 203, so that the application can provide this information to the network slice allocation service 425 when making an API-based request for a network slice. In some embodiments, the network slice allocation service 425 automatically determines one or more optimal network slices for a customer’s application or their entire radio-based network 103. To do so, the network slice allocation service 425 can train one or more machine learning models to identify network slice configurations from conditions for a customer or across multiple customers. The machine learning models can then be used to identify the optimal network slice to allocate to a given device type or application that is determined to exist in the radio-based network 103. For example, the network slice allocation service 425 can receive detailed network information or automatically probe the radio-based network 103 to learn about devices, applications, latency, usage patterns, etc. The network slice allocation service 425 can then provide this information to a machine learning model to automatically determine one or more network slices to optimize latency, bandwidth, reliability, or other metrics. These automatically determined network slices can then be automatically allocated by the network slice allocation service 425.

[0099] The RBN hardware configuration service 427 is executed to implement configuration changes to hardware that implements the radio-based network 103. The hardware can include radio units, antennas, VM instances or containers that execute network functions, routers, switches, fiber termination equipment, and so on. For example, an antenna can be configured for operation at a particular frequency. A radio unit can be programmed to operate at a particular frequency, join a particular radio-based network 103, and backhaul traffic to a particular VM instance or container.

[0100] In some scenarios, the RBN hardware configuration service 427 is executed to reconfigure hardware already present in the radio-based network 103. In other scenarios, the RBN hardware configuration service 427 is executed to pre-configure groups of hardware to be deployed to an existing or new radio-based network 103. To this end, the RBN hardware configuration service 427 can implement configurations on one or more pre-deployment devices 409 temporarily connected to the network 412 to facilitate pre-configuration prior to shipping the pre-deployment devices 409 to a customer for deployment in a radio-based network 103.

[0101] The RBN hardware deployment service 430 is executed to automate and arrange the deployment of hardware to implement the radio-based network 103. Based on a network plan 439 submitted by a customer or generated for a customer, the RBN hardware deployment service 430 can arrange for the procurement of hardware components needed to implement the radio-based network 103 according to the network plan 439. This can involve automatically placing orders with vendors to purchase new equipment, reserving equipment already in a provider’s inventory, or reallocating equipment already present at a customer site or other customer sites where the equipment is no longer being used. In this regard, the RBN hardware deployment service 430 can send instructions to a customer to return equipment that is no longer being used, where the equipment can be sent directly to another customer for use in another deployment. In another scenario, the RBN hardware deployment service 430 can send instructions to a customer to move a piece of equipment from one site where it is no longer being used to another site where it will be used. The RBN hardware deployment service 430 can manage the connection of equipment to the network 412 for pre-configuration as pre-deployment devices 409. The RBN hardware deployment service 430 can also arrange for the shipping of equipment to customer locations, including multiple customer locations that can correspond to individual cell sites.

[0102] The local orchestration service 433 is executed to enforce one or more local rules 434 configured for the customer regarding radio-based network 103 or network slices on radio-based network 103. In some implementations, the local orchestration service 433 may be executed when the RBN management service 424 is planning, creating, or provisioning radio-based network 103, such that all elements of radio-based network 103 (including network functions, data storage, cell equipment, and communication links) are within one or more geographic areas 312 as defined in the local rules 434. In other implementations, the local orchestration service 433 is executed when the network slice allocation service 425 allocates network slices in radio-based network 103 such that all elements implementing the network slices (including network functions, data storage, cell equipment, and communication links) are within one or more geographic areas 312 as defined in the local rules 434. When resources allocated to radio-based network 103 or network slices conforming to locality rule 434 are scaled, locality orchestration service 433 may also be performed to ensure that additional resources added to radio-based network 103 or network slices conform to locality rule 434. In some implementations, locality orchestration service 433 may create one or more gateways to connect restricted radio-based network 103 or restricted network slices to other segments of radio-based network 103 or the Internet, and then apply policy rules to limit and restrict network traffic, allowing clients to know and confirm where violations of locality rule 434 exist.

[0103] The data stored in data storage area 415 includes, for example, one or more network plans 439, one or more cellular topologies 442, one or more spectrum assignments 445, device data 448, one or more RBN metrics 451, customer billing data 454, radio unit configuration data 457, antenna configuration data 460, network function configuration data 463, one or more network function workloads 466, one or more customer workloads 469, data about one or more network slices 470 (including one or more QoS requirements 471 and one or more locality rules 434), one or more locality rules 434 and / or potentially other data.

[0104] Network planning 439 is a specification for the radio-based network 103 to be deployed for customers. For example, network planning 439 may include the locations, geographical areas 312 to be covered, the number of cells 109, etc. FIG. 1A), device identification information and permissions, a desired maximum network latency, a desired bandwidth or network throughput for one or more classes of devices, one or more quality of service parameters for an application or service, and / or other parameters that can be used to create the radio-based network 103. The customer can manually specify one or more of these parameters via a user interface. One or more parameters can be pre-populated as default parameters. In some cases, a network plan 439 can be generated for the customer based at least in part on an automated site survey using unmanned aerial vehicles. The parameter values that define the network plan 439 can be used as a basis for the cloud service provider to bill the customer under a utility computing model. For example, in a service level agreement (SLA), the customer can be charged a higher fee for lower latency targets and / or higher bandwidth targets, and can be charged to the customer on a device basis, on a cell basis, based on a service geographic area 312, based on spectrum availability, etc. In some cases, the network plan 439 can contain threshold and reference parameters determined at least in part from an automated probe of the customer's existing private network.

[0105] The cellular topology 442 includes an arrangement of a plurality of cells 109 for the customer that takes into account re-use of spectrum given the locations of the cells 109. The cellular topology 442 can be automatically generated given a site survey. In some cases, the number of cells 109 in the cellular topology 442 can be automatically determined based on a desired geographic area 312 to be covered, availability of backhaul connections at various sites, signal propagation, available spectrum, and / or other parameters. For the radio-based network 103, the cellular topology 442 can be developed to cover one or more buildings in an organization campus, one or more schools in a school district, one or more buildings in a university or university system, and other areas. The cellular topology 442 can be determined such that the cells 109 remain within one or more geographic areas 312 specified by the locality rules 434.

[0106] The spectrum allocation 445 includes spectrum that is available for allocation to the radio-based network 103 and spectrum that is currently allocated to the radio-based network 103. The spectrum can include spectrum that is publicly accessible without restriction, spectrum that is owned or leased by the customer, spectrum that is owned or leased by the provider, spectrum that is available for free use but requires reservation, etc.

[0107] Device data 448 corresponds to data describing wireless devices 106 that are permitted to connect to radio-based network 103. This device data 448 includes respective users, account information, billing information, data plans, permitted applications or uses, an indication of whether the wireless device 106 is mobile or fixed, location, current cell, network address, device identifier (e.g., International Mobile Equipment Identity (IMEI) number, Electronic Serial Number (ESN), Media Access Control (MAC) address, Subscriber Identity Module (SIM) number, etc.), and the like. Device data 448 can be subject to data storage requirements in locality rules 434, such that device data 448 is required to be stored within one or more particular geographic regions 312.

[0108] RBN metrics 451 include various metrics or statistics indicative of the performance or health of radio-based network 103. Such RBN metrics 451 can include bandwidth metrics, packet drop metrics, signal strength metrics, latency metrics, and the like. RBN metrics 451 can be aggregated on a per-device basis, on a per-cell basis, on a per-customer basis, and the like.

[0109] Customer billing data 454 specifies the fees incurred by a customer for the provider to operate radio-based network 103 for the customer. Fees can include fixed costs based on equipment deployed to the customer and / or usage costs based on utilization determined by tracked usage metrics. In some cases, a customer can pre-purchase equipment and can only be charged for bandwidth or backend network costs. In other cases, a customer can not incur any upfront costs and can only be charged on a usage basis. By providing equipment to customers based on utility computation models, a cloud service provider can choose optimal configurations of equipment to meet customer target performance metrics while avoiding over-provisioning unnecessary hardware. Customer billing data 454 can be subject to data storage requirements in locality rules 434, such that customer billing data 454 is required to be stored within one or more particular geographic regions 312.

[0110] Radio unit configuration data 457 can correspond to configuration settings for radio units deployed in radio-based network 103. Such settings can include frequencies to use, protocols to use, modulation parameters, bandwidth, network routing and / or backhaul configuration, and the like.

[0111] Antenna configuration data 460 can correspond to configuration settings for antennas, including frequencies to use, azimuth, vertical or horizontal orientation, beam tilt, and / or other parameters that can be automatically controlled or manually controlled by instructing a user to install the antenna or make physical changes to the antenna in a certain way (e.g., through a motor connected to the network and a control on the antenna).

[0112] Network function configuration data 463 corresponds to configuration settings for configuring the operation of various network functions for radio-based network 103. In various embodiments, network functions can be deployed in VM instances or containers located in computing devices 418 located at cell sites, customer aggregation sites, or data centers remote from customers. The location of network functions can be controlled at least in part by locality rules 434. Non-limiting examples of network functions can include access and mobility management functions, session management functions, user plane functions, policy control functions, authentication server functions, unified data management functions, application functions, network exposure functions, network function repositories, network slice selection functions, and / or other functions. Network function workloads 466 correspond to machine images, containers, or functions to be launched in allocated computing capacity 421 to perform one or more network functions.

[0113] Customer workloads 469 correspond to machine images, containers, or functions of customers that can be executed with or instead of network function workloads 466 in allocated computing capacity 421. For example, customer workloads 469 can provide or support customer applications or services. In various examples, customer workloads 469 involve factory automation, autonomous robots, augmented reality, virtual reality, design, monitoring, and so on.

[0114] Network slices 470 correspond to network traffic flows that have been designated for one or more locality rules 434 and / or one or more specific quality of service requirements 471. These flows can correspond to flows associated with a particular application executing on a particular client device 406, all network traffic from a particular client device 406, flows from all client devices 406 to a particular destination, flows from a particular client device 406 to a particular destination, flows of a particular type of traffic (e.g., voice calls, video, text messages, general data, etc.), and so on. In one example, network slices 470 are identified by source port, source network address, destination port, destination network address, and / or other information. Network slices 470 can be valid for a particular time period or a particular amount of data, or network slices 470 can be valid until canceled or released. In one example, network slices 470 are allocated on-demand for a particular application executing on a client device 406. In some scenarios, network slices 470 have a particular recurring validity time period (e.g., every weekday evening from midnight to 5 a.m.), or the quality of service requirements 471 for network slices 470 can change based on a recurring time period, current cost levels, and / or other factors or events.

[0115] The quality of service requirements 471 can correspond to minimum or maximum bandwidth, minimum or maximum latency, minimum or maximum reliability measure, minimum or maximum signal strength, etc. The quality of service requirements 471 can be associated with a corresponding cost level, which can include a fixed component, a usage-based component, and / or a congestion-based component. For example, the quality of service requirements 471 can be associated with a recurring monthly fixed cost, a per-session or per-megabyte cost, and / or a dynamic cost based on congestion at a cell site or particular network link. In some cases, a customer can select quality of service requirements 471 that provide a high level of service. In other cases, however, a customer can select quality of service requirements 471 that provide a low level of cost but reduce the quality of service at certain times or in certain respects. For example, a customer can select quality of service requirements 471 that allow for high-throughput overnight and other lower-priority throughput in order to send backup data through the network at a low cost.

[0116] The locality rule 434 is a customer-specified rule that causes at least a subset of network traffic on the radio-based network 103 operated by the customer or network traffic on a network slice 470 on the radio-based network 103 to remain within one or more particular geographic areas 312. The locality rule 434 can include an identification of one or more geographic areas 312 with predefined boundaries, such as a country, state, city, zip code, trade area, etc. In some cases, the locality rule 434 can include an arbitrarily defined geographic area 312 that can correspond to an organization campus or other area, such as a distance from a location, bounds and boundaries, cells within a grid, etc. In addition to defining a geographic area 312, the locality rule 434 can specify the time, event, or season for which the locality rule 434 applies; procedures for requesting and approving an exemption; the type or category of network traffic to which the locality rule 434 applies; the user, device, or application to which the locality rule 434 applies; whether use of network traffic or metadata outside the geographic area 312 is permitted (e.g., storing billing records outside the geographic area 312 can be deemed acceptable or unacceptable); and other parameters.

[0117] Client devices 406 represent a number of client devices that can be coupled to network 412. A client device 406 can comprise, for example, a processor-based system such as a computer system. Such a computer system can be embodied in the form of a desktop computer, a laptop computer, a personal digital assistant, a cellular telephone, a smartphone, a set-top box, a music player, a web appliance, a tablet computer system, a game console, an e-book reader, a smartwatch, a head-mounted display, a voice interface device, or other device. A client device 406 can include a display, which can comprise, for example, one or more devices such as a liquid crystal display (LCD), a gas plasma-based flat-panel display, an organic light-emitting diode (OLED) display, an electrophoretic ink (E ink) display, a LCD projector, or other display device, etc.

[0118] Client devices 406 can be configured to execute a variety of applications, such as client application 436 and / or other applications. Client application 436 can be executed in client device 406, for example, to access network content provided by computing environment 403 and / or other servers, to present a user interface on a display. To this end, client application 436 can comprise, for example, a browser, a special-purpose application, etc., and the user interface can comprise web pages, application screens, etc. Client device 406 can be configured to execute applications other than client application 436, such as, for example, email applications, social networking applications, word processors, spreadsheets, and / or other applications.

[0119] In some embodiments, spectrum reservation service 410 provides spectrum reservations for customers' private networks. In one scenario, spectrum reservation service 410 is operated by an entity such as a third party to manage reservations and coexistence in publicly accessible spectrum. One example of such spectrum is the Citizens Broadband Radio Service (CBRS). In another scenario, spectrum reservation service 410 is operated by a telecommunications service provider in order to sell or sub-license portions of spectrum owned or licensed by the provider.

[0120] Referring next to FIG. 5 , a flow diagram showing one example of the operation of a portion of RBN management service 424, in accordance with various embodiments, is shown. It can be appreciated that FIG. 5 the flow diagram of FIG. 5 may be viewed as depicting an example of elements of a method implemented in computing environment 403 FIG. 4 ) in accordance with one or more embodiments.

[0121] From block 503, the RBN management service 424 generates a user interface for ordering or provisioning an RBN 103( FIG. 1A ). For example, the user interface can include components for specifying a network plan 439( FIG. 4 ) or parameters of a network plan 439. Such parameters can include, for example, a number of cells, a map or site plan of a customer premises or geographic area to be covered, a target bandwidth, information about wireless devices 106( FIG. 1A ) or users, a target minimum latency, an expected cost, and / or other parameters. The user interface can include components for uploading one or more data files including this information. The user interface can be sent as a web page or other web data over the network 412( FIG. 4 ) for presentation by a client application 436( FIG. 4 ) executing in a client device 406( FIG. 4 ). Alternatively, the client application 436 can make one or more API calls to place an order or provision an RBN with a provider.

[0122] In block 506, the RBN management service 424 receives a request from an organization to provision an RBN. For example, a user can submit a form or otherwise interact with the user interface to cause the request to be submitted. Alternatively, the client application 436 can make one or more API calls to request that an RBN be provisioned.

[0123] In block 507, the RBN management service 424 can automatically initiate a probe of an existing private network of the organization. For example, the organization can have an existing network, such as a wired, Ethernet, Wi-Fi network, or other type of network. When appropriate security credentials and access to endpoints on the network are received, the RBN management service 424 can automatically probe the network to determine the needs of the customer for a new RBN 103 and associated core network. For example, the RBN management service 424 can automatically determine a number of user devices on the existing network, network bandwidth and request volume associated with various applications or services, existing latency observed for accessing various applications or services, reliability of the existing system, and so forth. These observations can be used to set initial thresholds for latency, bandwidth, and so forth in the RBN to be deployed.

[0124] In block 509, the RBN management service 424 determines a network topology for the RBN 103. For example, the network topology can be based at least in part on one or more locality rules 434( FIG. 4 ). The network topology can include cells 109( FIG. 1A) of the one or more buildings. Both interior and exterior areas of the buildings can be covered as needed. This determination can include receiving information about the coverage area, including indoor and outdoor coverage areas, the number of devices to be connected, and the layout of physical network connections and power. In this regard, the RBN management service 424 can determine the number of cells 109 according to the network plan 439. Optionally, the RBN management service 424 can automatically determine the optimal number of cells 109 based at least in part on parameters such as target latency, bandwidth, signal strength, and reliability, given the customer area and / or premises to be covered. In one embodiment, an unmanned aerial vehicle can be used to conduct a site survey of the area to be covered, potentially recording signal strength to observe the state of the spectrum to determine available frequencies. The RBN management service 424 can record the placement of the cells 109 in the cellular topology 442( FIG. 4 ) as the site survey is conducted. In some scenarios, the RBN management service 424 can use machine learning to determine the optimal placement of the cells 109 by deploying various placements of the RBN 103 and evaluating performance. Over time, by observing these deployments, the RBN management service 424 can learn which placements perform better or worse than others, and use these results to train a machine learning model.

[0125] In block 512, the RBN management service 424 automatically reserves spectrum allocations for the RBN 103. To do so, the RBN management service 424 can automatically determine available frequencies for the cellular topology 442 from publicly available frequencies, customer-owned frequencies, and / or provider-owned frequencies. The frequency determination can take into account polarization, directionality, beam tilt, and / or other factors that can allow or interfere with frequency reuse. The RBN management service 424 can record the reservations in the spectrum allocations 445( FIG. 4 ). Additionally, the RBN management service 424 can communicate with external services that implement spectrum reservation systems, such as the spectrum reservation service 410( FIG. 4 ), via the network 412 to make the reservations and / or determine available frequencies.

[0126] In block 515, the RBN management service 424 identifies the equipment needed to implement the RBN 103 according to the network topology that complies with the locality rules 434. This can include antennas, radio units, equipment for implementing the provider substrate extension 224( FIG. 2Acomputing devices, cables, switches, routers, fiber termination devices, and so on. In some cases, the computing devices can be included in an external unit to be installed outside the customer premises. In some examples, such a unit can be self-contained, in addition to power and network connections. In some scenarios, the RBN management service 424 can use machine learning to determine the optimal placement of devices by deploying various placements of the RBN 103 and evaluating performance. Over time, by observing these deployments, the RBN management service 424 can learn which placements perform better or worse than others, and use these results to train a machine learning model.

[0127] The RBN management service 424 can also determine, through machine learning, the optimal distribution of devices for individual customers. For example, it can make sense to run network function workloads 466 FIG. 1A for a particular type of customer in the data center core computing devices 118 FIG. 4 , while for another type of customer, it can be optimal to run network function workloads 466 at the distributed computing devices 112 FIG. 1A or the centralized computing devices 115 FIG. 1A . Thus, the RBN management service 424 can determine not to deploy computing devices 112 or 115, and instead favor a cloud-only deployment of core computing devices 118.

[0128] In block 518, the RBN management service 424 initiates procurement of devices for the RBN 103 via the RBN hardware deployment service 430 FIG. 4 . This can include reserving devices from an existing inventory of the provider, and / or placing one or more orders for devices with one or more suppliers.

[0129] In block 521, the RBN management service 424 causes the RBN hardware configuration service 427 FIG. 4 to pre-configure one or more devices, such as computing devices for implementing network functions, radio units, antennas, routers, and so on. Such devices can be connected to the network 412 as pre-deployed devices 409 FIG. 4 . The RBN hardware configuration service 427 uses radio unit configuration data 457 FIG. 4 , antenna configuration data 460 FIG. 4 , and network function configuration data 463 FIG. 4 to implement the pre-configuration.

[0130] In box 522, RBN management service 424 can configure one or more network slices for radio-based network 103, which can provide differentiated quality of service (QoS) levels for different user devices, applications, or services. QoS levels can provide different latency, bandwidth / throughput, signal strength, reliability, and / or other service factors. For example, a customer may have a group of devices requiring very low latency, so RBN management service 424 can configure network slices to provide below-threshold latency for these devices. In another example, a first QoS level can be provided for a first application, and a second QoS level can be provided for a second application.

[0131] In box 524, RBN management service 424 enables RBN hardware deployment service 430 to deliver a device, including a pre-configured pre-deployment device 409, to the customer. Various instructions for installing the device can be transmitted to the customer via client application 436. After this, part of the operation of RBN management service 424 ends.

[0132] Go to FIG. 6A The diagram illustrates a flowchart of an example of the operation of providing a portion of the localized orchestration service 433 according to various implementation schemes. It should be understood that... FIG. 6A The flowchart only provides examples of many different types of functional layouts that can be used to implement parts of the local orchestration service 433 as described in this document. Alternatively, FIG. 6A The flowchart can be viewed as depicting a computing environment 403 according to one or more implementation schemes. FIG. 4 Examples of elements of the methods implemented in ).

[0133] Starting with box 603, local orchestration service 433 is located on client device 406 ( FIG. 4 The locality orchestration service 433 receives one or more locality rules 434 (for managing users of the certified organization). In box 606, the locality orchestration service 433 receives one or more locality rules 434. FIG. 4 The specification of ) . For example, administrative users can specify one or more geographic regions 312 by selecting or identifying one or more predefined geographic regions 312 (e.g., country, state, city, postal code, etc.) or by arbitrarily defining geographic regions 312 by means of distance from a location, boundaries and limits, cells on a grid, etc. FIG. 3 Users can also specify various parameters for the types of network traffic to which locality rule 434 applies, when and under what conditions locality rule 434 applies, exemption mechanisms, what user records (such as subscriber data, keys, text messages, voicemails, billing records, etc.) will be retained in geographic region 312, and so on.

[0134] In box 609, the locality orchestration service 433 determines the locality for radio-based network 103 based at least in part on locality rules 434. FIG. 1A The network topology ensures that the elements of the radio-based network 103 are entirely within the geographic region 312. In some cases, the locality orchestration service 433 can access previously defined locality rules 434 or locality rules 434 pre-configured by the cloud provider rather than the organization.

[0135] This topology may include elements of a radio-based network 103, such as communication links (e.g., links between the RAN and the associated core network), provider underlying extensions 224, etc. FIG. 2A ) or cloud provider network 203 ( FIG. 2A Other capacity on which network functions are performed or data or metadata is stored, cell 109 ( FIG. 1A Location and other elements. In some cases, users can provide topology specifications, such as Topology Orchestration Specification for Cloud Applications (TOSCA) or other formats, such as JavaScript Object Notation (JSON). In some implementations, the locality orchestration service 433 can also automatically generate topology specifications in this format. For example, locality rule 434 can require one or more identified categories of network traffic (e.g., voice, video, text messages, data) to remain within geographic region 312.

[0136] In various implementations, the locality orchestration service 433 may select the location of cell 109 at least in part based on the location of cell 109 for deploying RBN 103 being within geographic region 312; select the location at least in part based on the location of provider underlying extension 224 for deploying cloud provider network 203 being within geographic region 312; and select the first region 306 from multiple regions 306 (or local region) of cloud provider network 203 at least in part based on the first region 306 for deploying at least a portion of the associated core network (compared to a second region 306 outside geographic region 312) being within geographic region 312. FIG. 3 (or local area); the specific communication link is selected at least in part based on the fact that the specific communication link used to connect the RAN and the associated core network 312 is entirely within the geographical area; the specific data storage service is selected at least in part based on the fact that the specific data storage service used to store data records associated with RBN 103 is within the specific geographical area 312; and so on.

[0137] In block 612, the locality orchestration service 433 initially provisions a radio-based network 103 with a topology that complies with the locality rules 434, or reconfigures an existing radio-based network 103 to have a compliant topology. The radio-based network 103 can be configured to enforce the locality rules 434 by means of firewall rules, network access control lists, security groups, and / or other mechanisms to ensure that data does not leave the secure virtual private cloud network within the cloud provider network 203. The locality orchestration service 433 can also determine or verify that the generated topology complies with the locality rules 434. At least a portion of the radio-based network 103 is provisioned in the cloud provider network 203. Thereafter, the operation of the locality orchestration service 433 ends.

[0138] Turning to FIG. 6B , a flow diagram illustrating one example of the operation of another portion of the locality orchestration service 433 in accordance with various embodiments is shown. It should be understood that FIG. 6B the flow diagram merely provides an example of the many different types of functional arrangements that can be used to implement the operation of portions of the locality orchestration service 433 as described herein. As an alternative FIG. 6B the flow diagram of FIG. 6 can be viewed as depicting an example of elements of a method implemented in the computing environment 403 FIG. 4 ) in accordance with one or more embodiments.

[0139] Beginning with block 615, the locality orchestration service 433 or another component of the RBN 103 FIG. 1A receives data sent by the client device 406 FIG. 4 ) via the RBN 103. Alternatively, the data can be sent to the client device 406 by another computing device, internal or external to the RBN 103. The data can correspond to control plane data (e.g., device registration data, authentication data, etc.), user plane data (e.g., voice call data, text message data, etc.), or a combination of both. In block 618, the locality orchestration service 433 determines that the transfer of the data to the destination would violate one or more locality rules 434 FIG. 4 ) configured for the RBN 103. For example, the destination can be outside of a particular geographic region 312 FIG. 3 ) specified in the locality rules 434 and the locality rules 434 would otherwise apply to the type of data being sent. In another example, the locality orchestration service 433 can determine that at least a portion of the route of the data to the destination is outside of the particular geographic region 312 or that the data will be processed in some manner outside of the particular geographic region 312. If the data is incoming data, the source can be outside of the particular geographic region 312 and the destination can be inside of the particular geographic region 312, thereby violating the locality rules 434 from an incoming direction.

[0140] In block 621, the locality orchestration service 433 implements one or more actions in response to determining that transmitting the data to the destination would violate one or more locality rules 434. For example, the locality orchestration service 433 can prevent the data from being transmitted to the destination until an exemption to the locality rule 434 is approved. In other examples, violating the locality rule 434 can trigger a lawful intercept of the data, cause throttling or slowing of the communication of the data, cause disabling of throttling of the data, generate a notification or log entry, and / or cause other actions to be performed. In some cases, one type of processing (e.g., logging, transcoding, intercepting, etc.) is generally performed on network traffic that complies with the locality rules 434, but the processing is not performed on network traffic that does not comply with the locality rules 434.

[0141] In block 622, the locality orchestration service 433 determines whether to request an exemption to the locality rule 434. In some examples, in some or all cases, an exemption can be automatically approved in response to the detected resource constraint, otherwise the detected resource constraint would not allow the communication to pass. In other cases, one or more notifications for an exemption request are selectively sent according to a configured policy. If the locality orchestration service 433 determines to request an exemption, in block 624, the locality orchestration service 433 sends a notification of the exemption request for an exemption to the locality rule 434. In some cases, the locality orchestration service 433 can send the notification to an administrative user of the organization for which the RBN 103 is provisioned. In other cases, a user at the client device 406 can be sent the notification and be able to approve the exemption request. In certain cases, approval of the exemption can be required from users on both ends of a data connection (e.g., a voice call, a text message, or other exchange of data).

[0142] In block 627, the locality orchestration service 433 determines whether the required approval is received. If the locality orchestration service 433 receives approval of the exemption from the client device 406 or an administrative user's client device 406 via a user interface or an application programming interface (API), the locality orchestration service 433 continues from block 627 to block 630. In block 630, the locality orchestration service 433 continues to transmit the data to the destination. Thereafter, the operation of the portion of the locality orchestration service 433 ends. If no approval is received in block 627 or if no exemption is requested in block 622, the operation of the portion of the locality orchestration service 433 also ends.

[0143] Referring next to FIG. 7 , a flow diagram is shown that provides one example of the operation of another portion of the locality orchestration service 433 according to various embodiments. It should be understood that FIG. 7The flow diagrams depicted herein merely provide an example of the many different types of functional arrangements that can be utilized by an apparatus to implement the operations described herein. As one will appreciate, the functionality of the locality orchestration service 433, as described herein, can be spread across various components in addition to or other than the architecture illustrated in FIG. 4. Also, the FIG. 7 The flow diagrams depicted herein are to be considered in context with respect to the described embodiments. Accordingly, the flow diagrams can be viewed as depicting an example of a method implemented by a computing environment 403 FIG. 4 ) in accordance with one or more embodiments.

[0144] At block 703, the locality orchestration service 433 receives a request to provision one or more locality rules 434 for a network slice 470. The request to provision the network slice 470 can include a definition of a particular geographic region 312 for which the locality rules 434 apply. In block 706, the locality orchestration service 433 provisions the network slice 470 using the network slice allocation service 425 such that the network slice 470 can be implemented with a topology that complies with the locality rules 434. For example, the locality orchestration service 433 can deploy one or more provider underlay extensions 224 of the cloud provider network 203 within the particular geographic region 312. The locality orchestration service 433 can configure the RBN 103 to store data records related to network traffic for the network slice 470 in a data storage service within the geographic region 312. The RBN 103 can have one or more other network slices 470 that do not have the locality rules 434 or have different locality rules 434. FIG. 4 FIG. 4 FIG. 3 FIG. 4 FIG. 2A FIG. 2A FIG. 1A

[0145] At block 709, the locality orchestration service 433 receives network traffic sent via the network slice 470. The locality orchestration service 433 can determine that the network traffic is sent via the network slice 470 based on at least one of: a source application of the network traffic, a source client device of the network traffic, or a tag in the network traffic. In some cases, the network traffic is to be sent to a destination outside of the geographic region 312 specified in the locality rules 434, in which case the network traffic can be blocked until an appropriate exemption is approved.

[0146] At block 712, the locality orchestration service 433 determines whether the RBN 103 has resources that will comply with the locality rules 434 in processing the network traffic. In block 715, the locality orchestration service 433 evaluates whether the resources are adequate (e.g., meet a threshold for a quality of service requirement 471). FIG. 4

[0147] ​​​​​​​​If the resources are determined to be insufficient, the locality orchestration service 433 proceeds from block 715 to block 718 and increases resources in the geographic region 312 specified in the locality rule 434. That is, the locality orchestration service 433 can scale the computing resources allocated to the network slice 470 within the geographic region 312. For example, the locality orchestration service 433 can allocate one or more additional instances of a network function in the particular geographic region 312, increase the capacity of a communication link in the particular geographic region 312, and so on. The locality orchestration service 433 then proceeds to block 721. If the resources in the geographic region 312 are determined to be sufficient, the locality orchestration service 433 proceeds from block 715 to block 721.

[0148] In some cases, the locality orchestration service 433 can not be able to scale resources in the particular geographic region 312. For example, resources can not be available. In such scenarios, the locality orchestration service 433 can drop the traffic, request an exemption to the locality rule 434, or apply the exemption to the locality rule 434 automatically.

[0149] In block 721, if the resources are sufficient or scaled to be sufficient, the locality orchestration service 433 takes action on the network traffic to handle the network traffic in the geographic region 312 that complies with the locality rule 434. For example, the locality orchestration service 433 can select a path for the network traffic in the RBN 103 to comply with the locality rule 434. The locality orchestration service 433 can select a particular instance of a network function (e.g., AMF, SMF, UPF, etc.) to handle the network traffic, where the particular instance is selected from a plurality of instances of the network function in the RBN 103 based at least in part on the particular instance being within the particular geographic region 312. The locality orchestration service 433 can also select a particular base station or group of base stations in the RAN such that the base station is within the particular geographic region 312 and used for the network traffic. The locality orchestration service 433 can also select a particular communication link between the RAN and the associated core network for the path selection, where the particular communication link as a whole is within the particular geographic region 312. Thereafter, the operation of the portion of the locality orchestration service 433 ends.

[0150] Reference FIG. 8 A schematic block diagram of a computing environment 403 in accordance with an embodiment of the present disclosure is shown in FIG. 8. The computing environment 403 includes one or more computing devices 800. Each computing device 800 includes at least one processor circuit, e.g., having a processor 803 and a memory 806, both coupled to a local interface 809. For

[0151] Stored in memory 806 are data and several components that are executable by processor 803. In particular, the following can be stored in memory 806 and executable by processor 803: RBN management service 424, network slice allocation service 425, RBN hardware configuration service 427, RBN hardware deployment service 430, locality orchestration service 433, and potentially other applications. Also stored in memory 806 can be data store 415 and other data. In addition, an operating system can be stored in memory 806 and executable by processor 803.

[0152] It should be appreciated that there can be other applications stored in memory 806 and executable by processor 803, as can be appreciated. Where any component discussed herein is implemented in the form of software, any one of a number of programming languages can be employed such as, for example, C, C++, C#, Perl, PHP, Visual Ruby, or other programming languages.

[0153] Many software components are stored in memory 806 and executable by processor 803. In this regard, the term "executable" means a program file in its final form that can be run on processor 803. Examples of executable programs can be, for example, compiled programs that can be translated into machine code that can be loaded into a random access portion of memory 806 and run by processor 803, source code that can be expressed in an appropriate format such as object code that can be loaded into a random access portion of memory 806 and executed by processor 803, or source code that can be interpreted by another executable program to generate instructions in a random access portion of memory 806 to be executed by processor 803, and so forth. Executable programs can be stored in any portion of memory 806 including, for example, random access memory (RAM), read only memory (ROM), hard disk drive, solid-state drive, USB flash drive, memory card, optical disk such as a compact disk (CD) or digital versatile disk (DVD), floppy disk, magnetic tape, or other storage component.

[0154] Memory 806 is defined herein as including both volatile and nonvolatile memory and data storage components. Volatile components are those that do not retain data values upon loss of power. Nonvolatile components are those that retain data values upon a loss of power. Thus, memory 806 can comprise, for example, random access memory (RAM), read-only memory (ROM), hard disk drives, solid-state drives, USB flash drives, memory cards accessed via a memory card reader, floppy disks accessed via an associated floppy disk drive, optical discs accessed via an optical disc drive, magnetic tapes accessed via an appropriate tape drive, and / or other memory components, or a combination of any two or more of these memory components. Moreover, RAM can comprise, for example, static random access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM) and other such devices. ROM can comprise, for example, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other like memory.

[0155] Also, processor 803 can represent a plurality of processors 803 and / or a plurality of processor cores, and memory 806 can represent a plurality of memories 806 operating in parallel with the plurality of processors 803, respectively. In this case, local interface 809 can be an appropriate network that facilitates communication between any two of the plurality of processors 803, between any processor 803 and any of the memory 806, or between any two of the memories 806, etc. Local interface 809 can include appropriate circuitry for coordinating communication between the processors 803 and the memories 806, including, for example, over multiple buses of the local interface 809, according to techniques known in the art. The processors 803 can be electrical or some other available technology.

[0156] Although the RBN management service 424, the network slice allocation service 425, the RBN hardware configuration service 427, the RBN hardware deployment service 430, the locality orchestration service 433, and other various systems described herein can be implemented in software or code executed by general purpose hardware, as described above, they can also be implemented in dedicated hardware or a combination of software and dedicated hardware. If implemented in software, each of the service 424, 425, 427, 430, 433, and other systems described herein can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media include both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. In this manner, a computer-readable medium can take many forms, including but not limited to, a tangible floppy disk, a tangible compact disk, tangible tape, tangible magnetic

[0157] FIGS. 5-7The flow diagrams illustrate the functionality and operations of an implementation of portions of the RBN management service 424 and the locality orchestration service 433. If embodied in software, each block can represent a module, segment, or portion of code that includes transitory or non-transitory program instructions to implement the specified logical function(s). The program instructions can be embodied in the form of source code that includes human- readable statements written in a programming language that can be used or compiled into an object code using an object code translation system, machine code that includes number instructions recognizable by a suitable execution system such as the processor 803 in a computer system or other system. The machine code can be translated from source code, etc. If embodied in hardware, each block can represent a circuit or a number of interconnected circuits to implement the specified logical function(s).

[0158] Although FIGS. 5-7 The flow diagrams illustrate a specific order of execution, but it is to be understood that the order of execution can differ from that which is depicted. For example, two or more blocks can be executed in parallel or in a different order than that which is depicted. Also, FIGS. 5-7 Two or more blocks shown in succession can be executed concurrently or with partial concurrence. Also, specific operations shown in the diagrams can be performed in one piece or in two or more separate pieces of operation. In addition, the separation of various system components in the embodiments described herein is for illustrative purposes. In some embodiments, the boundaries between the system components can be different than as outlined herein. For example, one or more functions can be performed by a single system component or by one or more system components in combination. FIGS. 5-7 One or more of the blocks shown in the diagrams can be skipped or omitted. In addition, any number of counters, state variables, warning semaphores, or messages can be added to the logic described herein for purposes of enhanced utility, accounting, performance measurement, or providing fault diagnostic assistance, among other reasons. It is understood that all such variations are within the scope of the present disclosure.

[0159] In addition, any of the logic or application described herein that comprises software or code (including the RBN management service 424, the network slice allocation service 425, the RBN hardware configuration service 427, the RBN hardware deployment service 430, and the locality orchestration service 433) can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as the processor 803 in a computer system or other system. In this sense, the logic can comprise, for example, statements including instructions and declarations that can be fetched from such computer-readable medium and executed by the instruction execution system. In the context of the present disclosure, a "computer-readable medium" can be any medium that can contain, store, or maintain the logic or application described herein for use by or in connection with the instruction execution system.

[0160] A computer-readable medium can include any one of many physical media as examples of storage media such as, for instance, magnetic, optical, or semiconductor media. More specific examples of suitable computer-readable media would include, but are not limited to, magnetic tapes, magnetic floppy diskettes, magnetic hard drives, memory cards, solid-state drives, USB flash drives, or optical discs. Also, computer- readable media can be random access memory (RAM) including, for example, static random access memory (SRAM) and dynamic random access memory (DRAM), or magnetic random access memory (MRAM). In addition, computer-readable media can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other type of memory device. A computer-readable medium can also include a propagated signal or computer- readable media transmitted from one place to another. For instance, a propagated signal can be a signal configured to encode a piece of information for transmission. Such a signal can be transmitted, for example, through a wired medium or through a wireless medium. The wired medium can include, for example, wires, wires wrapped in insulation, electric cables, optical cables, fiber optic cables, dielectric cables, or other wired media. The wireless medium can include, for example, acoustic waves, radio waves, infrared light, visible light, or other wireless media. Combinations of the above should also be included within the scope of computer-readable media.

[0161] Further, any of the logic or applications described herein, including the RBN management service 424, the network slice allocation service 425, the RBN hardware configuration service 427, the RBN hardware deployment service 430, and the locality orchestration service 433, can be implemented in numerous ways, and structured in a variety of ways. For example, the logic or applications described as a single application can be implemented as multiple applications or components. Further, multiple applications described herein can be implemented as a single application or a combination of applications, on a shared or separate computing device, or some combination thereof. For example, multiple applications described herein can be implemented on the same computing device 800, or multiple computing devices 800 in the same computing environment 403.

[0162] Unless specifically stated otherwise, as apparent from the preceding discussions, it is appreciated that, throughout the specification, discussions utilizing terms such as “at least one of,” “including,” “comprising,” “consisting of,” “by” or the like, are intended to convey to the reader on a disjunctive sense, that, for example, the terms “at least one of,” “including,” “comprising,” “consisting of,” “by” or the like, are used inclusively, such that any one of the referenced items are individually permissible from among the overall group of items, or the features of a group of items. For example, if a method is described to include A, B, or C, it is intended that the method can include A alone or B alone or C alone or any combination thereof. In addition, certain embodiments can be described as comprising, consisting of or consisting essentially of certain components. It is understood that, in this description, non-limiting embodiments consisting of, consisting essentially of or consisting of the recited components can include other components as well, unless the context clearly dictates otherwise.

[0163] Embodiments of the disclosure can be described in terms of the following clauses:

[0164] Clause 1. A system comprising: at least one computing device in a cloud provider network, the at least one computing device configured to at least: receive a request to provision a radio-based network for an organization, the radio-based network comprising a radio access network and an associated core network, at least a portion of the associated core network to be provisioned in the cloud provider network; determine a topology for the radio-based network that complies with at least one locality rule, the at least one locality rule requiring one or more identified categories of network traffic for the radio-based network to remain within a particular geographic region, such that a portion of the topology corresponding to a portion of the radio-based network that handles the one or more identified categories of network traffic is composed of physical hardware located within the particular geographic region; and provision the radio-based network with the topology that complies with the at least one locality rule for the organization.

[0165] Clause 2. The system of Clause 1, wherein the at least one computing device is further configured to at least: receive a specification of the at least one locality rule from a client device associated with the organization; and receive an arbitrary definition of the particular geographic region from the client device.

[0166] Clause 3. The system of Clauses 1-2, wherein the topology defines at least one communication link that is entirely within the particular geographic region between the radio access network and the associated core network.

[0167] Clause 4. A computer-implemented method comprising: accessing at least one locality rule associated with an organization, the at least one locality rule requiring at least one subset of network traffic for a radio-based network to remain within a particular geographic region, the radio-based network comprising a radio access network and an associated core network; determining a topology for the radio-based network based at least in part on the at least one locality rule; and provisioning or reconfiguring the radio-based network for the organization to cause the topology to comply with the at least one locality rule.

[0168] Clause 5. The computer-implemented method of Clause 4, further comprising receiving the at least one locality rule from a client device associated with the organization.

[0169] Clause 6. The computer-implemented method of Clauses 4-5, wherein determining the topology for the radio-based network based at least in part on the at least one locality rule further comprises: selecting a location for deployment of a cell of the radio access network based at least in part on the cell being within the particular geographic region.

[0170] Clause 7. The computer-implemented method of clauses 4-6, wherein determining the topology for the radio-based network based at least in part on the at least one locality rule further comprises selecting a location for a provider substrate extension of a cloud provider network to deploy based at least in part on the location being within the particular geographic region.

[0171] Clause 8. The computer-implemented method of clauses 4-7, wherein determining the topology for the radio-based network based at least in part on the at least one locality rule further comprises selecting a first region of a plurality of regions of a cloud provider network to deploy at least a portion of the associated core network based at least in part on the first region being within the particular geographic region, wherein at least a second region of the plurality of regions is outside the particular geographic region.

[0172] Clause 9. The computer-implemented method of clauses 4-8, wherein determining the topology for the radio-based network based at least in part on the at least one locality rule further comprises selecting a particular communication link to connect the radio access network and the associated core network based at least in part on the particular communication link being entirely within the particular geographic region.

[0173] Clause 10. The computer-implemented method of clauses 4-9, wherein determining the topology for the radio-based network based at least in part on the at least one locality rule further comprises selecting a particular data storage service to store data records associated with the radio-based network based at least in part on the particular data storage service being within the particular geographic region.

[0174] Clause 11. The computer-implemented method of clauses 4-10, wherein the at least one locality rule requires one or more user records associated with the radio-based network to remain within the particular geographic region.

[0175] Clause 12. The computer-implemented method of clauses 4-11, wherein the at least one locality rule requires one or more identified categories of network traffic to remain within the particular geographic region, the one or more identified categories comprising at least one of: voice calls, text messages, or data.

[0176] Clause 13. The computer-implemented method of clauses 4-12, further comprising receiving an arbitrary definition of the particular geographic region from a client device associated with the organization.

[0177] Clause 14. The computer-implemented method of clauses 4-13, wherein the particular geographic region comprises a plurality of designated geographic regions.

[0178] Clause 15. The computer-implemented method of clauses 4-14, wherein the particular geographic region corresponds to at least one of: a country, a state, a city, or a zip code.

[0179] Clause 16. A computer-implemented method comprising: receiving data transmitted via a radio-based network operated for an organization, the radio-based network comprising a radio access network and an associated core network, the radio-based network enforcing at least one locality rule requiring at least one subset of network traffic for the radio-based network to remain within a particular geographic region; determining that transferring the data to a destination would violate the at least one locality rule; and implementing one or more actions in response to determining that transferring the data to the destination would violate the at least one locality rule.

[0180] Clause 17. The computer-implemented method of clause 16, wherein the one or more actions comprise at least one of: preventing the data from being transferred to the destination, causing the data to be lawfully intercepted, or causing the data to be throttled.

[0181] Clause 18. The computer-implemented method of clauses 16-17, further comprising: sending a notification to a user; receiving approval from the user for an exemption of the at least one locality rule; and transferring the data to the destination in response to the approval of the exemption of the at least one locality rule.

[0182] Clause 19. The computer-implemented method of clauses 16-18, wherein determining that transferring the data to the destination would violate the at least one locality rule further comprises determining that the destination is outside the particular geographic region.

[0183] Clause 20. The computer-implemented method of clauses 16-19, wherein determining that transferring the data to the destination would violate the at least one locality rule further comprises determining that at least a portion of a path of the data to the destination is outside the particular geographic region.

[0184] Clause 21. A system comprising: a radio-based network comprising a radio access network and an associated core network, at least a portion of the associated core network provisioned in a cloud provider network; and at least one computing device in the cloud provider network, the at least one computing device configured to at least: receive a request to provision a network slice in the radio-based network, the network slice associated with at least one locality rule that requires data transmitted via the network slice to remain within a particular geographic area; provision the network slice in the radio-based network that complies with the at least one locality rule by using physical hardware located within the particular geographic area; receive network traffic transmitted via the network slice; and process the network traffic in the radio-based network within the particular geographic area so as to comply with the at least one locality rule.

[0185] Clause 22. The system of clause 21, wherein the at least one computing device is further configured to at least scale, based at least in part on utilization of the network slice, at least computing resources allocated to the network slice within the particular geographic area.

[0186] Clause 23. The system of clauses 21-22, wherein provisioning the network slice in the radio-based network further comprises deploying at least one provider substrate extension within the particular geographic area in response to the request.

[0187] Clause 24. The system of clauses 21-23, wherein the request to provision the network slice comprises a definition of the particular geographic area.

[0188] Clause 25. The system of clauses 21-24, wherein the at least one computing device is further configured to at least store data records related to the network traffic in a data storage service within the particular geographic area.

[0189] Clause 26. The system of clauses 21-25, wherein at least one other network slice in the radio-based network is not associated with the at least one locality rule.

[0190] Clause 27. The system of clauses 21-26, wherein the at least one computing device is further configured to at least: receive other network traffic transmitted via the network slice to a destination outside of the particular geographic area; and block the other network traffic from being communicated to the destination until an exemption to the at least one locality rule is approved.

[0191] Clause 28. A computer-implemented method comprising: receiving network traffic via a network slice in a radio-based network, the radio-based network comprising a radio access network and an associated core network, the network slice being associated with at least one locality rule requiring the network traffic to remain within a particular geographic area; determining that the radio-based network has resources that comply with the at least one locality rule; and taking action on the network traffic in the radio-based network in order to comply with the at least one locality rule.

[0192] Clause 29. The computer-implemented method of clause 28, further comprising storing a data record associated with the network traffic within the particular geographic area in order to comply with the at least one locality rule.

[0193] Clause 30. The computer-implemented method of clauses 28-29, further comprising: receiving other network traffic via the network slice in the radio-based network; determining that the radio-based network does not have the resources that comply with the at least one locality rule in routing the other network traffic; and sending a notification to a user device indicating that an exemption to the at least one locality rule is needed to route the other network traffic.

[0194] Clause 31. The computer-implemented method of clauses 28-30, further comprising: receiving other network traffic via a different network slice in the radio-based network, the different network slice being not associated with the at least one locality rule; and selecting a different path for the other network traffic in the radio-based network, wherein the other network traffic exits the particular geographic area.

[0195] Clause 32. The computer-implemented method of clauses 28-31, further comprising determining that the network traffic is sent via the network slice based at least in part on at least one of: a source application of the network traffic, a source client device of the network traffic, or a tag in the network traffic.

[0196] Clause 33. The computer-implemented method of clauses 28-32, wherein determining that the radio-based network has the resources that comply with the at least one locality rule further comprises allocating one or more additional resources to the radio-based network in the particular geographic area in order to comply with the at least one locality rule.

[0197] Clause 34. The computer-implemented method of clauses 28-33, wherein taking action on the network traffic in the radio-based network to comply with the at least one locality rule further comprises selecting a particular communication link between the radio access network and the associated core network for path selection, wherein the particular communication link as a whole is within the particular geographic region.

[0198] Clause 35. The computer-implemented method of clauses 28-34, wherein taking action on the network traffic in the radio-based network to comply with the at least one locality rule further comprises selecting a particular instance of a network function to process the network traffic, the particular instance selected from a plurality of instances of the network function in the radio-based network based at least in part on the particular instance being within the particular geographic region.

[0199] Clause 36. The computer-implemented method of clause 35, wherein at least one other instance of the plurality of instances of the network function in the radio-based network is outside the particular geographic region.

[0200] Clause 37. A computer-implemented method comprising: receiving network traffic via a network slice in a radio-based network, the radio-based network comprising a radio access network and an associated core network, the network slice being associated with at least one locality rule requiring the network traffic to remain within a particular geographic region; determining that a capacity of a network function in the particular geographic region is insufficient to handle the network traffic; allocating one or more instances of the network function in the particular geographic region to increase the capacity; and handling the network traffic by the one or more instances of the network function in the particular geographic region to comply with the at least one locality rule.

[0201] Clause 38. The computer-implemented method of clause 37, wherein determining that the capacity of the network function in the particular geographic region is insufficient to handle the network traffic further comprises: determining, based at least in part on the at least one locality rule, that existing capacity of the network function in the radio-based network but outside the particular geographic region cannot be used to handle the network traffic.

[0202] Clause 39. The computer-implemented method of clauses 37-38, further comprising identifying the network slice based at least in part on at least one of: a source client device of the network traffic, a source application of the network traffic, or a label in the network traffic.

[0203] Clause 40. The computer-implemented method of clauses 37-39, further comprising: determining that a capacity of a communication link in the particular geographic region is insufficient to communicate the network traffic; increasing the capacity of the communication link; and communicating the network traffic via the communication link in the particular geographic region so as to comply with the at least one locality rule.

[0204] It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations, set forth for a clear understanding of the principles of the present disclosure. Many variations and modifications can be made to the above-described embodiments without substantially departing from the principles of the present disclosure. All such modifications and variations are intended to be included herein within the scope of the present disclosure and the present protection.

Claims

1. A system comprising: at least one computing device in a cloud provider network, the at least one computing device configured to at least: receive a request to provision a radio-based network for an organization, the radio-based network comprising a radio access network and an associated core network, at least a portion of the associated core network to be provisioned in the cloud provider network; determine a topology for the radio-based network that complies with at least one locality rule that requires one or more identified categories of network traffic for the radio-based network to remain within a particular geographic region, such that a portion of the topology corresponding to a portion of the radio-based network that handles the one or more identified categories of network traffic consists of physical hardware located within the particular geographic region; and provision the radio-based network for the organization with the topology that complies with the at least one locality rule.

2. The system of claim 1, wherein the at least one computing device is further configured to at least: receive a specification of the at least one locality rule from a client device associated with the organization; and receive an arbitrary definition of the particular geographic region from the client device.

3. The system of claim 1, wherein the topology defines at least one communication link that is entirely within the particular geographic region between the radio access network and the associated core network.

4. A computer-implemented method comprising: accessing at least one locality rule associated with an organization, the at least one locality rule requiring at least a subset of network traffic for a radio-based network to remain within a particular geographic region, the radio-based network comprising a radio access network and an associated core network; determining a topology for the radio-based network based at least in part on the at least one locality rule, wherein determining the topology further comprises at least one of: selecting a location for deployment of a provider substrate of a cloud provider network based at least in part on the location being within the particular geographic region; selecting a first region of a plurality of regions of the cloud provider network for deployment of at least a portion of the associated core network based at least in part on the first region being within the particular geographic region, wherein at least a second region of the plurality of regions is outside the particular geographic region; selecting a particular communication link to connect the radio access network and the associated core network based at least in part on the particular communication link being entirely within the particular geographic region; or selecting a particular data storage service to store data records associated with the radio-based network based at least in part on the particular data storage service being within the particular geographic region; and provisioning or reconfiguring the radio-based network for the organization to have the topology comply with the at least one locality rule.

5. The computer-implemented method of claim 4, further comprising receiving the at least one locality rule from a client device associated with the organization.

6. The computer-implemented method of claim 4, wherein determining a topology for the radio-based network based at least in part on the at least one locality rule further comprises: selecting a location for deploying a cell of the radio access network based at least in part on the cell being within the particular geographic area.

7. The computer-implemented method of claim 4, wherein determining a topology for the radio-based network based at least in part on the at least one locality rule further comprises: selecting a location for deploying a provider substrate extension of the cloud provider network based at least in part on the location being within the particular geographic area.

8. The computer-implemented method of claim 4, wherein determining a topology for the radio-based network based at least in part on the at least one locality rule further comprises: selecting the first region of a plurality of regions of the cloud provider network for deploying at least a portion of the associated core network based at least in part on the first region being within the particular geographic area.

9. The computer-implemented method of claim 4, wherein determining a topology for the radio-based network based at least in part on the at least one locality rule further comprises: selecting the particular communication link to connect the radio access network and the associated core network based at least in part on the particular communication link being entirely within the particular geographic area.

10. The computer-implemented method of claim 4, wherein determining a topology for the radio-based network based at least in part on the at least one locality rule further comprises: selecting the particular data storage service to store data records associated with the radio-based network based at least in part on the particular data storage service being within the particular geographic area.

11. The computer-implemented method of claim 4, wherein the at least one locality rule requires one or more user records associated with the radio-based network to remain within the particular geographic area.

12. The computer-implemented method of claim 4, wherein the at least one locality rule requires one or more identified categories of network traffic to remain within the particular geographic area, the one or more identified categories comprising at least one of: voice calls, text messages, or data.

13. The computer-implemented method of claim 4, further comprising receiving an arbitrary definition of the particular geographic area from a client device associated with the organization.

14. The computer-implemented method of claim 4, wherein the particular geographic area comprises a plurality of specified geographic areas.

15. The computer-implemented method of claim 4, wherein the particular geographic area corresponds to at least one of: a country, a state, a city, or a zip code.

16. A computer-implemented method comprising: receiving data transmitted via a radio-based network operated for an organization, the radio-based network including a radio access network and an associated core network, the radio-based network enforcing at least one locality rule requiring at least a subset of network traffic for the radio-based network to remain within a particular geographic area; determining that transmitting the data to a destination would violate the at least one locality rule; and implementing one or more actions in response to determining that transmitting the data to the destination would violate the at least one locality rule, wherein determining that transmitting the data to the destination would violate the at least one locality rule further comprises determining that at least a portion of a path of the data to the destination is outside the particular geographic area.

17. The computer-implemented method of claim 16, wherein the one or more actions comprise at least one of preventing the data from being transmitted to the destination, causing the data to be lawfully intercepted, or causing the data to be throttled.

18. The computer-implemented method of claim 16, further comprising: sending a notification to a user; receiving approval from the user for an exemption to the at least one locality rule; and transmitting the data to the destination in response to the approval of the exemption to the at least one locality rule.

19. The computer-implemented method of claim 16, wherein determining that transferring the data to the destination would violate the at least one locality rule further comprises: determining that the destination is outside the particular geographic area.

Images