Organizer and interconnection fabric mapper for virtual wireless base stations - Patents.com

Abstract

A virtual radio base station is disclosed that can dynamically scale its capacity to meet changing connectivity demands. The virtual radio base station includes a plurality of virtual baseband modules, a plurality of interface / router modules, an organizer module, and a fabric mapper module. Each of the plurality of virtual baseband modules is coupled to the interface / router module by a low-latency switch fabric. The organizer determines current and near-future connectivity demands within the virtual radio base station and connects new virtual baseband processors to meet increased demand, or shuts down underutilized virtual baseband processors if demand is insufficient.

Description

[Technical field]

[0001] The present invention relates to wireless communications, and more particularly to systems and components that enable virtual wireless base stations. [Background technology]

[0002] A wireless communication network, such as an LTE or 5G network, is generally made up of a radio access network, a core network, and an interface to the Internet. It is through the radio access network (RAN) that handsets and other user devices (commonly referred to as User Equipment (UEs)) exchange messages and data packets that make up a user's phone calls, emails, texts, and web browsing. The radio access network consists of multiple base stations, each coupled to one or more antennas via a radio remote unit. Each base station generates and receives radio signals at a prescribed frequency and coding scheme that allows UEs within range of the radio signals to connect to and communicate with the Internet. These base stations provide complex signaling between each UE and the core network, which maintains proper connectivity with each UE. In doing so, transmission occurs using dynamically determined frequencies, modulation schemes, and multiple input multiple output (MIMO) layers that optimally allocate data communication resources to each UE. Summary of the Invention [Problem to be solved by the invention]

[0003] Conventional wireless access networks, and their component base stations, suffer from the following drawbacks: Conventional base stations are individually built to accommodate the expected peak concurrency of wireless communication devices and their corresponding connections, and thus are over-designed to meet fixed peak capacity levels. Peak concurrency often occurs only for a short period of time on any given day or week, and usage patterns often vary widely from one service area to another. For example, peak concurrency in office buildings in commercial business districts may occur at 2:00 PM, peak concurrency in apartment buildings in residential areas may occur at 8:00 PM, and peak concurrency in stadiums may occur only once every few weeks for a few hours. In the case of a stadium, the contrast between normal low demand and peak demand can vary between 500 and 100,000 actively connected users. This results from a network of base stations that are individually over-designed at fixed peak capacity levels, which results in significantly higher costs for each individual base station, as well as the overall wireless communication system collectively.

[0004] What is needed, therefore, is a virtualized base station that can scale its capacity to meet the current demands for connectivity within a given coverage area, and to do so to meet the strict latency requirements of telecommunications standards such as LTE and 5G. [Means for solving the problem]

[0005] An aspect of the present invention relates to a persistent machine-readable memory encoded with instructions that, when executed by one or more processors, cause the one or more processors to perform a process comprising the steps of: determining a connectivity demand with a wireless communication access network; determining a combination of component modules based on the connectivity demand, the component modules including at least one baseband module; determining a number of interconnection channels between the at least one baseband module and other component modules within the combination of component modules; instantiating each component module within the combination of component modules; connecting the at least one baseband module to at least one external network; and connecting the at least one baseband module to at least one user equipment (UE).

[0006] Another aspect of the present invention relates to a non-transitory machine-readable memory encoded with instructions which, when executed by one or more processors, cause the one or more processors to perform a method comprising the steps of: determining a connectivity demand with a wireless communication access network, the wireless communication access network having at least one baseband module; determining whether a capacity corresponding to the at least one baseband module is sufficient to meet the demand; instantiating at least one additional baseband module; connecting the at least one additional baseband module to at least one external network component; and instructing at least one of the plurality of baseband modules to hand over one or more UEs to the at least one additional baseband module.

[0007] Another aspect of the invention relates to a non-transitory machine-readable memory encoded with instructions that, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising the steps of: determining a connectivity demand in a wireless communication access network, the wireless communication access network having a plurality of baseband modules; designating one of the plurality of baseband modules as an underutilized virtual baseband module based on the connectivity demand; instructing the underutilized baseband module to hand over a plurality of connected UEs to an accepting baseband module in the plurality of baseband modules, the accepting baseband module corresponding to a neighboring cell group; and deactivating the underutilized baseband module. Equipped with.

[0008] Another aspect of the present invention relates to a virtual radio base station comprising a plurality of routing means, a plurality of baseband processing means, an orchestrating means, a switching means for low latency switching coupled between said plurality of routing means and said plurality of baseband processing means, and a mapping means for fabric mapping coupled to said orchestrating means and to said switching means for low latency switching.

[0009] Another persistent machine-readable memory is a persistent machine-readable memory encoded with instructions that, when executed by one or more processors, cause the one or more processors to implement a process for mitigating inter-cell interference between two cell groups, each of the two cell groups having a corresponding baseband module, each baseband module coupled to a low-latency switch fabric, the process comprising the steps of: identifying two baseband modules experiencing inter-cell interference with at least one UE within a common coverage area of ​​the two cell groups; identifying a frequency band corresponding to the inter-cell interference; instantiating a coordinator module coupled to the low-latency switch fabric; coupling the coordinator module to the baseband modules via the low-latency switch fabric; and performing intra-frame reconciliation between the two baseband modules.

[0010] Another aspect of the invention relates to a radio base station comprising: a plurality of cell groups, each having a corresponding baseband module, each of the baseband modules coupled to a low-latency switch fabric; and a hardware computing environment having non-transitory machine-readable medium instructions coded to execute a process comprising the steps of: identifying two baseband modules experiencing inter-cell interference with at least one UE within a common coverage area of ​​the two cell groups, identifying a frequency band corresponding to the inter-cell interference, instantiating a coordinator module, coupling the coordinator module to the baseband modules via the low-latency switch fabric, and performing intra-frame reconciliation between the two baseband modules. [Brief description of the drawings]

[0011] [Figure 1] 1 illustrates an exemplary virtual base station system according to the present disclosure. [Diagram 2] 1 illustrates an exemplary baseband processing module according to the present disclosure. [Diagram 3] 1 illustrates an exemplary process for launching and initializing a new virtual base station according to the present disclosure. [Figure 4] 1 illustrates an exemplary process for allocating component modules to meet current and near-future demands for connectivity, according to the present disclosure. [Diagram 5] 1 illustrates an exemplary method of operation for a virtualized base station in accordance with the present disclosure. [Figure 6] 1 illustrates an example method for implementing inter-cell interference mitigation in accordance with the present disclosure. [Figure 7] 13 illustrates an exemplary configuration during a low connection demand scenario. [Figure 8] 1 illustrates an exemplary configuration during a high connection demand scenario. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] A virtual base station hosting environment and architecture is disclosed that allows for partitioning of one or more radio base stations (e.g., one or more eNodeBs) into virtualized components that can be individually and dynamically created, realized, and terminated.

[0013] Virtual execution allows base station components to be dynamically created, realized, and terminated in response to demand fluctuations. Moreover, substantial cost savings can be achieved by leveraging off-the-shelf, readily available server hardware. However, deploying virtual base stations presents technical challenges. For example, given the strict latency requirements compliant with telecommunications standards such as LTE and 5G, inter-task communication between virtualized components places tremendous demands on traditional computer hardware, especially when these virtualized components are dynamically created, realized, and terminated in response to demand fluctuations.

[0014] For LTE, according to the present disclosure, virtual base stations or eNodeBs can be partitioned such that their S1, X2, GTP, and M2M interface functions are encapsulated in separate software objects (hereinafter referred to as "interface / router" components), which can be shared by several virtual baseband processors. In doing so, for example, a standard software-based S1 interface can act as a router between multiple software-based baseband processors and one or more MMEs. Furthermore, the virtual baseband processor can be partitioned between its protocol stack layers, such that bottleneck protocol stack functions are parallelized into multiple components or process threads for greater speed. This can create one or more baseband processors with many cells that can be easily balanced. This can simplify communication with MMEs over the S1 interface, i.e., there are fewer baseband processors with higher capacity. If there are two interfering cells under different virtual baseband processors, these two interfering cells can be migrated to a standalone baseband processor so that all component UEs are scheduled by a single scheduler, or a new baseband processor can be created to host the two interfering cells.

[0015] Enabling these features requires a server hardware computing environment with multiple multi-processor circuit boards interconnected by a high-speed switching fabric (fabric). An example is multiple server circuit boards, each equipped with a PCIe adapter, which are connected between servers by an InfiniBand® switch (e.g., 36-port 100 Gbps) operating at 100 Gbps or 200 Gbps. The InfiniBand components have ultra-low latency (<600 ns on the PCIe card, <90 ns between ports on the switch) with sufficient bandwidth to enable high performance clustering. Different virtual baseband processors and interface / router components can communicate either directly over a shared memory within a single circuit board, or over interconnect channels between circuit boards such as InfiniBand, by RDMA (Remote Direct Memory Access). Furthermore, the use of low-latency switch fabrics (fabrics) can be enhanced by incorporating data plane improvement technologies such as DPDK (Data Plane Development Kit) that enable packet processing workload acceleration. It will be understood that variations to this hardware computing environment are possible and are within the scope of the present disclosure.

[0016] Proper use of this hardware platform and host multiple partitioned baseband processor along with the centralized S1 / X2 / GTP / M2M interface module requires two software entities: an orchestrator and an inter-module channel mapper (hereinafter referred to as the "Fabric Mapper"), described below.

[0017] FIG. 1 illustrates an exemplary virtual base station system 100 according to the present disclosure. The system 100 includes a number of component modules 105. Each component module 105 couples to an Ethernet switch 110 via an Ethernet connection 177 and to a high-bandwidth, low-latency switch fabric 120. An example of a suitable high-bandwidth, low-latency switch fabric 120 is an InfiniBand switch as described above, although other switching technologies such as Omni-Path® or Ethernet can be used if the latency requirements can be met. In the illustrated exemplary system 100, the component modules 105 include three baseband processor unit (BBU) modules 155 and four interface / router component modules, namely, M2Mux 135, S1Mux 140, GTPMux 145, and X2Mux 150. Each BBU module 155 can be coupled to each of the four interface / router component modules via a dedicated interconnect channel established in the low-latency switch fabric 120.

[0018] Coupled to the Ethernet switch 110 are standard external network components 115. In an exemplary LTE-based embodiment, the external components 115 are a Mobility Management Entity (MME) 180, a Multicell / Multicast Coordination Entity (MCE) 182, a Serving Gateway (SGW) 185, a Multimedia Broadcast Multicast Service Gateway (MBMS GW), and one or more external eNodeBs 190. Each of these external network components is coupled to a corresponding interface / router component module via the Ethernet switch 110 according to a corresponding interface protocol (e.g., SCTP between the MME 180 and the S1Mux 140, GTP between the SGW 185 and the GTPMux 145 and between the MBMS GW 187 and the GTPMux 145, M2 between the MCE 182 and the M2Mux 135, and X2 between the one or more external eNodeBs 190 and the X2Mux 150).

[0019] Each of the three illustrated BBUs 155 may be coupled to a corresponding POI / DAS (Point of Interface / Distributed Antenna System) component module 160 via the low latency switch fabric 120. Additionally, each POI / DAS component module 160 may be coupled to a wireless remote unit 170 via a CPRI (Common Public Radio Interface) connection 162, which in turn may be coupled to one or more antennas (not shown) via an RF distribution connection. Each CPRI connection 162 may be implemented with a dedicated PCIe CPRI card that may be installed on one or more server circuit boards hosting system 100. Although system 100 is illustrated with individual dedicated CPRI connections 162, it will be appreciated that these connections may be implemented with a CPRI switch (not shown) capable of routing CPRI traffic between POI / DAS component modules 160 and remote units 170. In another variation, the CPRI connections 162 can be implemented using a low-latency switch fabric 120, such as an InfiniBand switch, such that the remote units 170 can be directly coupled to the low-latency switch fabric 120. In yet another variation, one or more BBU modules 155 can be directly coupled to one or more remote units 170 via the CPRI connections 162, such that no POI / DAS module 160 is involved. Furthermore, given latency requirements, one or more of the CPRI connections 162 can instead be implemented with an Ethernet connection. It will be understood that such variations are possible and are within the scope of the present disclosure.

[0020] In another variation, one or more BBUs 155 may instead of implementing a standard LTE protocol stack may be a special purpose protocol stack, for example to provide special services to bandwidth limited NB-IoT UEs. In some cases, this special purpose BBU module 155 may be assigned its own spectrum and operate independently of other BBU modules 155. In this case, the special purpose BBU module 155 may be coupled to a dedicated remote unit 170. In a variation, the NB-IoT BBU module 155 may operate within a guard band in the frequency band of a separate standard LTE BBU module 155. In this case, the special purpose BBU module 155 may be coupled to a remote unit 170 shared with the standard LTE BBU module 155, where their corresponding RF signals merge within the remote unit 170. In other cases, harmonization within a given band is required, in which case the special purpose BBU module 155 and the standard LTE BBU module 155 can share a set of resource blocks within a given LTE frame, and harmonization is handled by the harmonizer (coordinator) module 157, as will be further described below.

[0021] Each wireless remote unit 170 and its corresponding antenna (not shown) can cover a given cell group 175. A cell group 175 as used herein can be considered a baseline unit of granularity within the scaling system 100. A cell group 175 can be defined, at a minimum, as a single antenna range covering all bands served by a given antenna / remote combination including all MIMO layers present at a given time. In the case of multiple bands, any UE in that cell group can transmit and receive on any combination of available bands, and the corresponding BBU module 155 independently schedules all UEs in a given cell group 175 using carrier aggregation techniques. At a maximum, a cell group 175 can have multiple antenna gain patterns, multiple bands within each gain pattern, and all MIMO layers available at a given time. In this case, the corresponding BBU module 155 can communicate with multiple POI / DAS modules 160 and multiple capable remote units 170 and independently schedule all UEs within range. In each of these ranges, a single BBU module 155 handles a single cell group 175, and the size and complexity of the cell group can vary.

[0022] Modifications to the remote unit 170 are possible and are within the scope of this disclosure. For example, the remote unit 170 can be a conventional remote radio head for DAS, an active antenna system, or an advanced remote unit such as a Cell Hub unit (provided by JMA Wireless). Although the exemplary system 100 is illustrated as having three POI / DAS (Point of Interface / Distributed Antenna System) 160, it will be readily understood that modifications are possible. For example, any of them can be macro cells, small cells, etc., and any combination thereof, are possible and are within the scope of this disclosure. Furthermore, while three BBUs 155 and POI / DAS 165 are illustrated, more or less are possible and are within the scope of this disclosure.

[0023] The system 100 further includes an orchestrator module 130, a fabric mapper module 125, and one or more coordinator modules 157, each of which is described below.

[0024] Each of the four interface / router modules includes functionality that is typically implemented by a separate eNodeB. In accordance with this disclosure, these interface functions are considered separate from the BBU functionality and may be encapsulated in separate components that can service more than one BBU 155 (three in the exemplary embodiment of system 100, although more or fewer BBUs 155 are possible and within the scope of this disclosure). Thus, system 100 may be considered to have three eNodeBs, whereby an eNodeB has a BBU 155 and the corresponding interface functionality is implemented by the interface / router component modules.

[0025] All component modules 105 may have machine-readable instructions encoded in one or more memory devices and executed on one or more processors coupled to the Ethernet connection 110 and the low-latency switch fabric 120. As used herein, the term "non-transitory memory" may refer to any tangible storage medium (as opposed to an electromagnetic or optical signal) and may refer to the medium itself and not be limited to data storage (e.g., RAM vs. ROM). For example, a non-transitory medium may refer to an embedded memory encoded with instructions such that the memory must be reloaded with the appropriate machine-readable instructions after a power cycle. Each component module 105 may be hosted on one or more multi-processor server circuit boards according to the hardware computing environment described above. Variations in how the component modules 105 are deployed on the hardware computing environment are possible. For example, in one variation, component modules 105 on the same server circuit board can communicate via shared memory, and component modules on different server circuit boards can communicate over interconnection channels via the low-latency switch fabric 120. In another variation, an entire set of base station component modules 105 can be deployed in virtual machines or containers on a physical machine, using RDMA between base station component modules that share a server circuit board and using RDMA over interconnection channels in the low-latency switch fabric 120 between base station component modules on different server circuit boards. In another variation, component modules 105 can be deployed in virtual machines or containers on a physical machine, and all component modules 105 communicate over the low-latency switch fabric by virtual addressing, regardless of whether the lower layer physical instantiations of any pair of component modules 105 share a physical node (server circuit board) or not.Exemplary software containers may be Docker containers or rkt pods, or any similar software packages that allow containerization of applications. Additionally, a hybrid of containers, VMs, and processors running on bare metal is also possible. However, the advantage of containerization is that it can be spun up very quickly and is very lightweight in terms of size due to the fact that it encapsulates minimal OS components. However, the advantage of one or more VMs is that it allows for persistent memory usage and state-driven operation. Exemplary variations of these are possible and are within the scope of this disclosure.

[0026] The Ethernet connection 110 is coupled to an orchestrator 130. The orchestrator 130 may be a software module having machine-readable instructions encoded in a memory, such as a persistent memory, which when executed by one or more processors performs the following functions, in no particular order: First, the orchestrator 130 determines the current and near-future traffic demand for the system 100 during the startup and operation of the system 100. Second, the orchestrator 130 monitors the performance of the system 100 by measuring the performance of each BBU module 155 and / or receiving measurement reports from each BBU module 155, and adjusts the capacity of the system 100 in response to the monitored performance and demand changes. The orchestrator 130 can adjust the capacity of the system 100 by performing certain functions including creating new BBUs 155, shutting down unnecessary BBUs 155, and merging or splitting cell groups, thereby adjusting the capacity in a certain BBU module 155. Third, the orchestrator 130 can improve the performance of the system 100 by coordinating the allocation of cells 175 to corresponding BBUs 155. To minimize the complexity when there are multiple UEs in two cell group ranges and have two BBU modules 155 that must coordinate the scheduling of multiple UEs in two overlapping antenna ranges, the orchestrator 130 can merge the two existing cell groups 175 into a single cell group 175, so that a single scheduler in the single BBU module 130 can handle the scheduling in a larger service area. Alternatively, the orchestrator 130 can instantiate one or more coordinator modules 157 that can help coordinate the scheduling between two interfering BBUs 155 on a serving frequency (described further below).Alternatively, the orchestrator 130 can execute commands to create a new BBU 155 and transition both simultaneous interfering cells 175 to it. The orchestrator 130 can be coupled to each component via an Ethernet connection 110 to issue commands, receive status information, and so on.

[0027] Another function of the orchestrator 130 is to issue commands to the fabric mappers 125 to set up, change, and take down interconnect channels within the low latency switch fabric 120, thereby allowing the component modules 105 to communicate with each other with minimal latency and sufficient bandwidth.

[0028] The orchestrator 130 may be implemented using a container orchestration engine (COE), such as Kubernetes or other software suite that provides additional functionality to the hypervisor / container runtime. The orchestrator 130 may delegate some functionality (e.g., component module 105 health checks, on-demand deployment, etc.) to the COE.

[0029] As mentioned above, each of the four interface / router components, namely, M2Mux 135, S1Mux 140, GTPMux 145, and X2Mux 150, implements standard communications between each BBU module 155 and its corresponding external network component 115. S1Mux 140 serves as an interface and router between each of the BBUs 155 and the MME 180. S1Mux 140 may have two ports, one that connects to the Ethernet switch 110 using SCTP and one that connects to the low latency switch fabric 120. The connection from S1Mux 140 to the low latency switch fabric 120 may be in the form of S1-AP messages packaged in the form of InfiniBand packets. For DL ​​(Downlink) communication from MME 180, S1Mux 140 executes instructions to intercept SCTP formatted messages from MME 180, retrieve eNodeB identifiers from the messages, strip SCTP related information from the messages, convert them to S1-AP messages, and route them to interconnection channels in InfiniBand switch 110 corresponding to the eNodeB identifier of the intended BBU 155. In doing so, S1Mux 140 can execute instructions to perform, for example, RDMA memory writes to memory allocated to the target component module via dedicated interconnection channels in low latency switch fabric 120. For UL (Uplink) communication from a BBU 155 to MME 180, S1Mux 140 receives messages from BBU 155 via the BBU's dedicated interconnection channel in low latency switch fabric 120, converts the S1-AP messages to SCTP format, and transmits the messages to MME 180 via Ethernet connection 110. The structure and functionality of other interface / router components are largely similar, with the primary difference being in the specific interface protocol translation, which translates messages between Ethernet-based protocols and formats for relaying through the low-latency switch fabric 120.

[0030] The GTPMux 145 primarily serves as a router between the SGW 185 and each BBU module 155. The GTPMux 145 may have the following ports: an Ethernet port that communicates with the SGW 185 via the Ethernet switch 110, and one or more second ports for communicating with each BBU module 155 over the low latency switch fabric 120. As mentioned above, the ports of the GTPMux 145 are bidirectional. Similar to the S1Mux 140, the GTPMux 145 may be configured such that the SGW 185 is unaware that it is communicating directly with an intervening interface / router component module and not directly with each BBU module 155 as it would with a normal eNodeB. The M2Mux 135 serves as a router in a similar manner to the GTPMux 145, but handles Multimedia Broadcast Multicast Services (MBMS) traffic from the MBMS GW 184 to each BBU module 155.

[0031] FIG. 2 illustrates an exemplary embodiment of a BBU module 155. The term baseband processor or BBU as used herein may refer to a software-implemented vertical LTE or 5G protocol stack whose interface functionality is partitioned and handled by one of the interface / router modules. Each BBU module 155 may be a software module having machine-readable instructions encoded in one or more persistent memory components in the hardware computing environment of the system 100, such that when executed by one or more processors in the hardware computing environment of the system 100, each BBU 155 implements a protocol stack and scheduling function. As shown in FIG. 2, the exemplary BBU 155 may include a Packet Data Convergence Protocol (PDCP) component 210, a Radio Link Control (RLC) component 220, a Media Access Control (MAC) component 230, and a Physical layer (PHY) component 250. Each BBU module 155 communicates with the external network component 115 via a corresponding interface / router component as described above. Each BBU module 155 is further coupled to a CPRI connection 160 for bidirectional transmission of I / Q (In-phase / Quadrature) signal data between the BBU module 155 and its corresponding POI / DAS 165. This is an alternative where each BBU module 115 can be directly coupled to a POI / DAS module 160 via a CPRI connection.

[0032] 2, for each BBU module 155, its PDCP module 210 can be coupled to the low latency switch fabric 120 for relaying uplink / downlink data traffic through the GTPMux 145. The PDCP module 210 and the RLC module 220 can be linked by an uplink / downlink connection 215, each of which can be implemented by a shared memory or by an interconnection channel through the low latency switch fabric 120. A similar implementation can be used for the uplink / downlink connection 225 between the RLC layer 220 and the MAC layer 230. The uplink / downlink connections 215 and 225 can be bearer-based, meaning that data is transmitted on these connections after the UE is connected to the BBU module 155, and the extent of traffic on the uplink / downlink connections 215 and 225 is a function of the number of active bearers.

[0033] The schedule module 230 and the MAC module 240 (which may be subcomponents of the scheduler 230) may perform most of the scheduler functions and also control the plane signaling of the BBU module 155. The MAC module 240 may incorporate additional functionality to coordinate scheduling with other BBU modules 155 via one or more coordinators 157, receive configuration information from the orchestrator 130 and the fabric mapper 125, and provide measurement reports and status information to the orchestrator 130. To accommodate these functions, the MAC module 240 may have an Ethernet connection 232 to the Ethernet switch 110, thereby allowing it to communicate with these other component modules. The MAC module 240 may also be coupled to the low latency switch fabric 120 for communicating S1-AP signaling messages to and from the S1Mux 140.

[0034] PHY module 250 implements the PHY layer functionality expected in LTE or 5G implementations. PHY module 250 can operate as separate components or threads, one thread per carrier, one thread for each MIMO layer for each carrier, and one thread for the uplink and one thread for the downlink. Each PHY module 250 can communicate with MAC module 240 via PHY link 235. PHY link 235 can have a dedicated communication link for each PHY module thread 250. These links can be over shared memory or over dedicated interconnect channels on low latency switch fabric 120. Each PHY module thread 250 can also have a dedicated interconnect channel on low latency switch fabric 120 to a designated POI / DAS module 160. Alternatively, each PHY module thread 250 can be coupled over a high speed link (e.g., PCIe) to a CPRI card, which is coupled to remote unit 170 over CPRI connection 162.

[0035] Returning to FIG. 1, the fabric mapper module 125 may be a software module having machine-readable instructions encoded in a memory, such as a persistent memory, which, when executed by one or more processors, performs the following functions: It establishes interconnect channels in the low-latency switch fabric 120 (as an example) for each interconnect channel between the base station component modules 105. For example, the fabric mapper 125 establishes multiple interconnect channels (e.g., one for each LTE bearer) between each BBU 155 and each interface / router component, such that each interconnect channel can provide RDMA access between the two components, so that data can be exchanged with sufficient bandwidth and low enough latency for each BBU to perform the base station functionality within the given telecommunications standard (e.g., LTE) requirements. Based on the nature of the interconnect (what is connected to what), latency and bandwidth requirements may vary. To accommodate this, the fabric mapper 125 has configuration data tables corresponding to the latency and bandwidth requirements of each interconnect channel combination (e.g., BBU / S1Mux, BBU / GTPMux, BBU1 / BBU2 via X2Mux, etc.) and can allocate the interconnect channels appropriately (e.g., within the memory allocated for RDMA, etc.). If each BBU is further partitioned into protocol subcomponents, the fabric mapper 125 can set up interconnect channels between them (e.g., between the MAC module 240 of a BBU 155 and its corresponding PHY module 250).

[0036] Based on how orchestrator 130 has configured resources for each base station component module, fabric mapper 125 determines paths through low latency switch fabric 120 to create interconnect channels between the component modules. After fabric mapper 125 determines the interconnect channels, it communicates the associated interconnect channel information (e.g., memory addresses, port gas numbers, etc.) to each of the other module components.

[0037] Based on the expected bandwidth requirements between two component modules 105, the fabric mapper 125 can establish memory arrays to be shared between the two component modules over dedicated interconnect channels within the low latency switch fabric 120.

[0038] The Fabric Mapper 125 interfaces with the Orchestrator 130 such that if a BBU module 155 goes down, the Orchestrator 130 can execute instructions to instantiate a replacement BBU module 155 and provide associated address information to the Fabric Mapper 125, which in turn can execute instructions to map a new set of interconnect channels within the low latency switch fabric 120 and provide information regarding the new set of interconnect channels to the new replacement BBU 155. In doing so, the Fabric Mapper 125 can act as (or in conjunction with) an InfiniBand Subnet Manager as part of its functionality.

[0039] The fabric mapper 125 may also provide redundancy to increase the robustness of the system 100. For example, the orchestrator 130 may determine that a significant increase in traffic is imminent (e.g., halftime at a football stadium for a set of cells 175 covering the stadium's concession area). Given this situation, the orchestrator 130 may execute instructions to inform the fabric mapper 125 of the need for additional resources. In response, the fabric mapper 125 may execute instructions to pre-map interconnect channels to additional BBU modules 155 and, if necessary, to keep them in a pool of interconnect channels for rapid deployment.

[0040] Low latency switch fabric 120 may have one or more hardware components with interconnect processors within a single server circuit board as well as interconnect processors spread across multiple server circuit boards. In embodiments where low latency switch fabric 120 is an InfiniBand switch, according to the InfiniBand specification, the actual hardware connections may include optical fibers, circuit board wiring, twisted pair wired connections, etc. Associated software to perform specific InfiniBand functions may be stored and run on the hardware components of low latency switch fabric 120.

[0041] Although this disclosure describes an interconnection channel and hardware computing environment using InfiniBand components, it will be understood that other interconnection fabric technologies are possible and within the scope of this disclosure, so long as they allow sufficiently high bandwidth and sufficiently low latency for inter-task communication between the component modules of the base station to comply with LTE and / or 5G specifications.

[0042] Startup (setup) and configuration

[0043] FIG. 3 illustrates an exemplary process for activating and configuring a virtual base station 100 according to the present disclosure.

[0044] In step 305, orchestrator 130 executes instructions to determine the current demand, near future demand, and demand volatility for the wireless network in which virtual base station 100 is deployed. In doing so, orchestrator 130 may retrieve information from a configuration file or other information source. In an exemplary embodiment, orchestrator 130 executes instructions from the IEEE 802.11b, which is incorporated herein by reference in its entirety. The current demand, near future demand, and demand volatility can be obtained via an Adaptive Connection Control System (ACCS) such as that described in U.S. Patent Application Serial No. 15 / 918,799, entitled "SYSTEM AND METHOD FOR ADAPTIVELY TRACKING AND ALLOCATING CAPACITY IN A BROADLY-DISPERSED WIRELESS NETWORK." However, the information obtained may include the following: That is, (a) an estimate of the current demand for connectivity, which may include the number of actively connected (e.g., in LTE RRC connected) and idle (e.g., RRC idle) devices in the vicinity where the system 100 is deployed, the device category of these devices (e.g., Cat 0-8, Cat M, etc.), and expected Quality of Service Class Indicators (QCIs) of bearers for connected UEs; (b) trend information involving estimated near-term demand for connectivity, which may include extrapolation of recent trend data or estimates based on historical data; and (c) volatility of the demand for connectivity, which may include estimated deviation of demand over a finite course of time. An example of high volatility is an urban area near a subway station, where pedestrians periodically surge in and out of the coverage area of ​​a cell group 175 during rush hour as each subway train discharges passengers. Another example of volatility is a venue such as a stadium, where a large number of spectators may suddenly move from one area to another when an event being held has a break. In each of these examples, there may be any steady demand for connectivity, as well as periods of spikes and drops in demand due to devices of different device categories. The orchestrator 130 can obtain and use this data to appropriately configure the system 100 being deployed.

[0045] In determining the current and near future demand, orchestrator 130 may execute instructions to obtain or retrieve (either on its own or via the ACCS described above) historical information regarding the demand for connectivity at the locations corresponding to system 100. This may be the time of day, the day of the week, the month, the occurrence of certain events (e.g., sporting events, holidays, concerts, inclement weather forecasts, etc.). Orchestrator 130 may take this information into account to perform a look-ahead algorithm to determine the current and near future demand. It will be understood that modifications to step 305 are possible and are within the scope of this disclosure.

[0046] In step 310 , the orchestrator 130 executes instructions to determine an appropriate allocation of component modules 105 to adequately meet the demand and volatility determined in step 305 .

[0047] FIG. 4 illustrates an example process by which orchestrator 130 may perform step 310.

[0048] In step 405, the orchestrator 130 executes instructions to decompose the demand determined in step 305 into an expected number of UEs by device category and expected capacity.

[0049] In step 410, orchestrator 130 executes instructions to determine demand concentrations by physical location and area within the coverage area of ​​system 100. In doing so, orchestrator 130 can retrieve cell identifiers corresponding to UEs in both connected and idle states from the demand data (obtained in step 305) and associate physical location information corresponding to the cell identifiers. The result is a map-like representation of the area covered by system 100 and the expected locations and concentrations of UEs corresponding to expected demand, and potentially the expected times corresponding to these concentrations and distributions.

[0050] In step 415, the orchestrator 130 executes instructions to allocate cell groups 175 to accommodate expected demand concentrations by physical location and area. This may depend on the particular configuration of the remote units 170 and their corresponding antennas. For example, some remote units 170 and antennas may support different bands that allow more or less carrier aggregation opportunities. Additionally, the orchestrator 130 may have access to historical data corresponding to the system 100 and the physical RF environment of each cell group. For example, one cell group 175 may be in a physical location that has been proven to be more receptive to more MIMO opportunities than the locations of other cell groups 175. For example, the antenna pattern of the remote unit 170 may be in a location that is between buildings with many reflective surfaces, resulting in many MIMO opportunities. These MIMO opportunities may be a function of the band. For example, an RF environment for the antenna pattern of a remote unit 170 may have many reflective surfaces, but may also have a lot of trees and other foliage, resulting in a more reflective environment (and therefore more MIMO opportunities) in the lower frequency bands, but more attenuation in the millimeter wave bands.

[0051] In this case, based on expected demand and known RF environment, orchestrator 130 may select one such remote unit 170 over other neighboring or overlapping remote units 170 if this remote unit supports more bands or has a more favorable antenna gain pattern. The result of step 415 is a set of selected remote units 170 to be activated and expected computational resources, each corresponding to a POI / DAS module 160 and a BBU module 155. For example, for a remote unit 170 that is expected to support many bands (more carrier aggregation opportunities) and also many MIMO layers (based on the known RF environment), orchestrator 130 may predetermine individual computational resource requirements for each BBU module 155: number of PHY module 250 threads, scheduler 230 complexity, etc.

[0052] Returning to FIG. 3, in step 315, the orchestrator 130 executes an instruction to allocate a shared memory for each BBU module 155 to store UE context data 191, a common data set covering all necessary configurations and device specific information for a UE. The array of UE context data 191 created by the orchestrator 130 should reside in the shared memory with convenient access to the corresponding BBU module 155. The template of UE context data 191 can be common to all UEs, which can include UE information such as maximum allowed bit rate and bandwidth capacity. Thus, considering that the expected demand for connectivity is determined by the orchestrator 130 in step 305, the size of the UE context data 191 can be an integer multiple of the size of a single UE context data template. Each UE context data template is added to the UEs connected to the BBU module 155, and each added UE context data template needs to be accessed by the subcomponents of the BBU module, namely the Packet Data Convergence Protocol (PDCP) component 210, the Radio Link Control (RLC) component 220, the Media Access Control (MAC) component 230, and the Physical layer (PHY) component 250. Each of these subcomponents may only need to access certain information in the added UE context data template. Thus, the orchestrator 130 may instantiate and configure each BBU module 155, and its constituent subcomponents are configured to access only those memory areas in the UE context data template that are needed. In addition to this, the orchestrator 130 may provide access to certain BBU module UE context data 191 to other BBU modules 155 and / or one or more optional coordinator modules 157.Access to other BBU modules enables coordination such as Coordinated Multi-Point (COMP) and Intercell Interference Coordination (ICIC).

[0053] In step 320, the fabric mapper 125 determines an appropriate set of interconnection channels in the low latency switch fabric 120 to meet the needs of the aggregate set of component modules 105, and then executes instructions to set up (activate) the interconnection channels. Each BBU module 155 requires multiple interconnection channels (e.g., one for each bearer in some cases) through each interface / router module, i.e., through the S1Mux 140 for communication to the MME 180, the M2Mux 135 for communication to the MCE 182, the GTPMux 145 for communication to the MBMS GW 187 and SGW 185, and the X2Mux 150 for communication to one or more external eNodeBs 190. In the case of the X2 interface, it may be known how many X2 connections are established for each BBU 155. It can be assumed that each BBU module 155 has an X2 connection to each of its corresponding peer BBU modules 155, which provides a baseline number of X2 interconnect channels for the X2 Mux 150. However, it may be unknown how many interconnect channels are needed to one or more external eNodeBs 190. Thus, the fabric mapper 125 can dynamically create and terminate interconnect channels between a BBU module 155 and an external eNodeB 190 as operational traffic demands change. Furthermore, based on how each BBU 155 is coupled to its corresponding CPRI connection 160, each BBU can have an interconnect channel to a CPRI circuit board via the low latency switch fabric 120.

[0054] Continuing with step 320, the fabric mapper 125 can allocate an array of dedicated memory for each interconnect channel, one for each direction of communication between two component modules. These can be static shared data structures that can be implemented as shared memory within a processor circuit board or as data structures to be used for RDMA communication over InfiniBand links. Given the static nature of the fixed arrays, the fabric mapper 125 can allocate them to have an appropriate size that cannot be excessive, but not so large as to preclude the flexibility to dynamically allocate additional arrays for additional interconnect channels that may be required as the orchestrator instantiates additional component modules 105 as needs grow.

[0055] In an embodiment using InfiniBand, the fabric mapper 125 can create a set of interconnect channels by defining multiple QPs (queue pairs), where each QP defines a pair of endpoints of the interconnect channel (e.g., BBU1 / S1Mux, BBU2 / S1Mux, BBU1 / GTPMux, etc.) and determines the allocation of service type (e.g., connection-oriented vs. datagram, reliable vs. unreliable, etc.), service level, link bit rate (e.g., x1, x4, and x12, (i.e., Infection Rate Control)), and fabric partitioning for the corresponding transmit and receive queues. This may depend on the device category and QCI corresponding to a certain bearer for which the fabric mapper 12 is creating and assigning the QP. Furthermore, based on the nature of how each BBU 155 is coupled to the corresponding CPRI connection 160, the fabric mapper 125 can define multiple QPs for each BBU and either dedicated port on the integrated CPRI circuit board. This remains true with the configuration of the system 100 .

[0056] Fabric mapper 125 establishes the appropriate QPs by executing instructions that invoke appropriate InfiniBand-defined verbs over low latency switch fabric 120 (InfiniBand switches in this embodiment). More specifically, fabric mapper 125 invokes appropriate verbs over each InfiniBand host channel adapter and sets up each component module 105 that is an InfiniBand consumer, where each consumer has multiple QPs.

[0057] After the fabric mapper 125 determines the appropriate set of interconnect channels, it then executes instructions to create those interconnect channels through the low latency switch fabric 120 and allocate the interconnect channels to corresponding pairs of component modules 105 .

[0058] In step 325, the orchestrator 130 executes instructions to instantiate the component modules 105 required by the system 100 to meet the estimated demand. In doing so, the orchestrator 130 may instantiate the appropriate interface / router components (M2Mux, S1Mux, GTPMux, and X2Mux) and one or more BBU modules 155 based on the resulting configuration data from step 310 described above with respect to FIG.

[0059] A factor to consider when allocating and instantiating BBU modules 155 involves a balance between the complexity of instantiation of each BBU module 155 and the multiple threading opportunities in the hardware computing environment of the system 100. For example, there may be an opportunity to cover a given area in the system 100 with a single large cell group 175 or two or more smaller cell groups 175. In the former case, the computing resources required for a larger and more complex BBU module 155 may be so large that the operating system of the hardware computing environment of the system 100 cannot adequately coordinate the processing of its own thread with other competing threads in other component modules 105. In other words, the computing efficiency of a multi-processor circuit board (having a multi-core processor) may be diminished by the presence of a large number of overlaying threads of a single BBU module 155. In this case, a smaller, more numerous, and more manageable (from an OS multitasking perspective) BBU module 155 may be provided to produce more efficient computing.

[0060] Further describing step 325, each instantiated BBU 155 can have a dedicated CPRI connection 162, whereby each instantiation is configured to be coupled to one or more port addresses to a hardware CPRI circuit board that is part of the hardware computing environment of system 100. Alternatively, each BBU 155 can be instantiated with one or more dedicated interconnection channels to an integrated CPRI circuit board through low latency switch fabric 120.

[0061] In a modification to the system 100, the orchestrator 130 can instantiate each BBU 155 in the form of a separate component module for each layer in the BBU protocol stack. Thus, instead of instantiating a BBU 155, the orchestrator can instantiate one PDCP module 210, one RLC module 220, one scheduler 230 and MAC module 240, and one PHY module 250 with multiple threads or with more than one PHY module 250. In this modification, the fabric mapper 125 can determine, create, and allocate interconnection channels for communication between them when communicating in an integrated manner as BBUs. In this case, the BBU 155 is a standard BBU with InfiniBand interconnection channels between each layer of the protocol stack.

[0062] Continuing with step 325, orchestrator 130 may instantiate different component modules 105 that share a hardware node (on the same server circuit board) or on different server circuit boards based on the computational power demands of a component module 105 and the latency and bandwidth stringency for communication between two or more component modules 105. For example, orchestrator 130 may execute instructions to instantiate POI / DAS modules 160 on the same server circuit board that places a PCIe CPRI card to minimize latency between the POI / DAS modules 160 and the remote unit 170.

[0063] Continuing with step 325, each instantiated component module 105 may register itself with the fabric mapper 125, which may provide each component module 105 with designated addresses and configuration information for the interconnect channels. Each component module 105 may communicate with the fabric mapper over its corresponding Ethernet connection 177 and the Ethernet switch 110. This allows each component module 105 to establish connections with the other component modules 105 via the Ethernet switch 110. For example, each BBU module 155 may execute instructions to register itself with the interface / router component modules, namely, S1Mux 140, GTPMux 145, X2Mux 150, and M2Mux 135. Each interface / router component module may then respectively configure itself to store associated cell IDs and other configuration information corresponding to each BBU module 155 for the interconnect channel corresponding to that BBU module 155 in association with the appropriate port addresses and configuration information (obtained by the fabric mapper 125).

[0064] At the end of step 325, all component modules 105 have been instantiated with memory space reserved for UE context data 191 and all required interconnection channels across the low latency switch fabric 120 have been established.

[0065] At step 330, each POI / DAS module 160 executes instructions to establish communication with a designated remote unit 170. In an exemplary embodiment in which each POI / DAS module 160 is coupled to a corresponding remote unit 170 through an intervening CPRI driver, each POI / DAS module 160 can execute instructions to activate and operate a data port for bidirectional communication with the remote unit 170. This can be done with a separate standalone CPRI connection between each POI / DAS-remote unit pair. Alternatively, system 100 can include a CPRI switch (not shown) that can act as a router between each POI / DAS module 160 and each remote unit 170. According to other exemplary embodiments, communication between each POI / DAS module 160 and its designated remote unit 170 may occur over a low-latency switch fabric 120 (e.g., an InfiniBand switch), and each POI / DAS module 160 and corresponding remote unit 170 may transmit I / Q (in-phase / quadrature) data bidirectionally over a dedicated interconnect channel, rather than providing a fiber optic connection from the low-latency switch fabric 120 to the remote unit 170. This may be done using the CPRI standard or a packet-based Ethernet protocol. In a variation in which one or more BBU modules 155 are directly coupled to one or more remote units 170, in step 330, each of these BBU modules 155 may establish a connection with its designated remote unit 170, as described above.

[0066] In step 335, each BBU module 155 sets up communication with its corresponding external network component 115 and the larger core network. For example, each BBU module 155 may implement the LTE-specific functionality of establishing S1 connections with one or more MMEs 180 to establish control plane communication with the MME 180 by the S1 Mux 140 over the interconnection channel set up in the low-latency switch fabric 120. Depending on how the S1 Mux 140 is configured, each BBU module 155 may be unaware of the presence of the S1 Mux 140, which acts transparently as an intermediary between it and the MME 180. After the S1 control plane communication is established between each BBU module 155 and the MME 180, the MME 180 and a BBU module 155 may continue the process of connecting each BBU module 155 to the external network component 115 via the GTPMux 145 and the M2 Mux 135 in cooperation with each other.

[0067] Further explaining step 335, the X2Mux module 150 can execute instructions to establish an X2 connection between each BBU module 155. This can occur dynamically during operation of the system 100 as UEs connect to each BBU module 155 to become aware of one or more other BBU modules 155 in the system 100 for potential handover. Alternatively, the X2 interface can be established at power-up, which allows each BBU module 155 to become aware of the other BBU modules 155 and obtain the IP addresses of the other BBU modules 155 when it connects to an operations support system (OSS) (not shown) in the core network and receives configuration data from the OSS. It will be understood that such variations are possible and are within the scope of this disclosure.

[0068] Further to step 335, each BBU module 155 may execute instructions to establish an X2 connection with one or more external eNodeBs 190. This may be performed automatically at power up, whereby the orchestrator 130 issues a command to each BBU 155 to execute instructions to establish automatic neighbor relations (as described above for the other BBU modules 155). Alternatively, this may occur later during operation of the system 100, whereby each BBU module 155 may facilitate the establishment of an X2 interface with an external eNodeB 190 by a connected UE, where the connected UE indicates a strong signal from the external eNodeB 190. This may be done in accordance with 3GPP TS 36.300.

[0069] In step 340, each BBU module 155 establishes a connection with UEs within the coverage area of ​​the corresponding cell group 175. With the S1 interface established between each BBU 155 and the MME 180 via the S1 Mux 140, each BBU module 155 executes instructions to generate MIB (Master Information Block) and SIB (System Information Block) information, which is formatted for broadcast signals. Each BBU 155 then transmits corresponding I / Q data via the low latency switch fabric 120 to the corresponding POI / DAS module 160, which transmits the I / Q data to the appropriate remote unit 170, which broadcasts the MIB and SIB information to UEs within the coverage area of ​​the cell group 175.

[0070] Continuing with step 340, each in-range UE receives the MIB and SIB information broadcast by a certain BBU module 155 and responds with an RRC connection request to the BBU module 155, which then responds with an RRC connection setup message, and so on, resulting in distinct UEs being in connection with each BBU module 155. As each UE connects and establishes services via the external network component 115, all corresponding communications between the UEs (through their BBU modules 155) occur via the appropriate interface / router component.

[0071] For example, communication between a BBU module 155 and an SGW 185 occurs via the GTPMux 145. In an embodiment where the low latency switch fabric 120 is an InfiniBand switch, data for a bearer can be relayed via a reliable RDMA QP, whereby the BBU module 155 and the GTPMux 145 can write (read) data from their respective memory sectors to the other allocated by the fabric mapper 125 via an InfiniBand channel adapter. The major difference between this and conventional LTE communication is that there is a single GTP interface (GTPMux 145) that acts like a router for multiple BBU modules 155.

[0072] When each UE itself connects to a BBU module 155, the scheduler module 230 in the BBU module 155 can retrieve the UE context data 191 obtained from the UE and add the retrieved data to the UE context data template. As a result, the UE context data shared memory object becomes an array of added UE context data templates, one for each connected UE, so that each BBU module subcomponent, i.e., Packet Data Convergence Protocol (PDCP) component 210, Radio Link Control (RLC) component 220, Media Access Control (MAC) component 230, and Physical layer (PHY) component 250, can access the corresponding required data in the UE context data template. The access is via a dedicated interconnection channel in the low latency switch fabric 120, but this is not necessary if the latency requirement for this type of data access is not too strict.

[0073] operation FIG. 5 illustrates an exemplary process 500 by which the system 100 operates, where the system performs the functions of a virtual base station and dynamically predicts and responds to connectivity demand changes.

[0074] In step 505, the orchestrator 130 executes instructions to determine the current and near future demands for connectivity of the system 100. Step 505 may be generally similar to step 305 of the exemplary process 300. Thus, assuming the system 100 is already running at any given iteration of step 505, the orchestrator 130 may have accumulated historical data regarding the connectivity demands of the system 100 since starting (from execution of the startup process 300) and the corresponding capacity and performance. The orchestrator 130 may further periodically monitor the performance of each BBU module 155. Given this history and performance data, the orchestrator 130 may perform a look-ahead algorithm to determine the near future demands and volatility and compare it to the current capacity of the system 100.

[0075] In step 510, the orchestrator 130 can execute instructions to monitor the performance of the system 100 (and each BBU module 155) as follows: Each BBU module 155 provides performance measurements (PMs) corresponding to each cell in the system 100. The PMs include low level counter information, such as the number of UE connection attempts and successes, the number of attempted and successful initial context setup requests and responses. Each BBU module 155 can execute instructions to provide these PMs to the orchestrator 130.

[0076] The orchestrator 130 can use the reported PM data to identify any BBU modules 155 that are becoming overloaded, as well as any BBU modules 155 that are exhibiting a consistent pattern in PM, have excess capacity, and can be shut down. If the latter and there are instances of excess capacity, the process 500 proceeds to branch 515, where the orchestrator can decide to shut down the underutilized BBU modules 155, and possibly the corresponding POI / DAS modules 160 and remote units 170 that collectively make up the cell group 175.

[0077] In step 520, the orchestrator 130 executes instructions to determine which baseband modules 155 and corresponding cell groups 175 should be shut down by determining which cell groups 175 are underutilized. This may include determining whether UEs currently being served by the underutilized cell groups 175 are close enough to be within range of other cell groups 175 to allow handoff without loss of coverage. If this is the case, the process 500 proceeds to step 525.

[0078] In step 525, the BBU module 115 of the cell group 175 to be terminated executes instructions to perform intra-LTE handover to the remaining cell group 175. It can do so using the X2 handover process. In this case, the source BBU module 155 sends an appropriate message to the destination BBU module 155 via the X2 Mux 150. After this handover is coordinated between the source (to be stopped) BBU module 155 and the destination (remaining) BBU module 155, the destination BBU module 155 executes instructions to complete the handover by appropriate signaling exchanges between it and the MME 180 via the S1 Mux 140 and between it and the SGW 185 via the GTPMux 145.

[0079] In step 530, the orchestrator 130 can execute commands to boost the power of the remaining cell group 175 that received the UEs taken over from the underutilized cell group 175 to be shut down. This may be necessary in a venue such as a stadium where the UEs connected to the remaining cell group 175 may be widely physically distributed. Alternatively, the remote unit 170 of the cell group 175 to be shut down can be synchronously taken over by the remaining cell group 175, and the remaining BBU module 155 can control both the original remote unit 170 and the remote unit 170 in the cell group 175 to be shut down. This can result in a larger cell group 175 being scheduled with a single BBU module 155 powered by two or more remote units 170.

[0080] FIG. 7 illustrates a reduced capacity configuration 700 of the system 100. Only the relevant component modules are illustrated in FIG. 7, it being understood that there are interface / router component modules and external components that are not illustrated. In the reduced capacity configuration 700, several cell groups 175 are combined into a larger cell group 705, which is driven by a single BBU module 155 and POI / DAS module 160. A high demand cell group 175 is illustrated, which itself has a dedicated BBU module 155 and POI / DAS module 160. Although not illustrated in this manner, the larger cell group 705 can be driven by a single (or a few) remote units 170 that now have increased power to compensate for the larger coverage area. It will be understood that such variations are possible and within the scope of the present disclosure. In this case, the remote units 170 that are not used can be shut down. It will be apparent that further power savings can be achieved by reducing the number and complexity of the component modules 105 of the system 100.

[0081] Returning to FIG. 5, if the orchestrator 130 determines in step 510 that the system 100 has sufficient capacity for current and near-future demand, then the process 500 proceeds to branch 540, where the orchestrator 130 executes instructions to add capacity to the system 100.

[0082] In step 545, orchestrator 130 executes instructions to determine where the connectivity demand surge is occurring. In doing so, orchestrator 130 identifies overloaded cell groups 175 by identifying BBU modules 155 reporting excess demand from reported performance measurements. With this information, orchestrator 130 identifies potential additional cell groups 175 by searching for unused remote units 170 within its coverage area that can then be shut down. Orchestrator 130 may do so by executing instructions to query a database or configuration file that lists available remote units 170, their corresponding antennas (not shown), and the coverage areas of these antennas. Additionally, this is true even when system 100 is operating in reduced capacity configuration 700, where two or more remote units 170 within a coverage area in an area experiencing increased demand may be operating redundantly as shown in FIG. 7. In this case, one or more of these remote units 170 may be available to form one or more new cell groups 175 in areas experiencing increased demand.

[0083] In step 550, the orchestrator 130 executes instructions to instantiate one or more BBU modules 155 and corresponding POI / DAS modules 160 to form one or more new cell groups 175. The process for doing this may be generally similar to steps 320-335 described above. As a result of step 550, the system 100 has one or more BBU modules 155 and POI / DAS modules 160, a shared memory array for UE context data 191 for the expected number of additional UEs, and interconnection channels between the new BBU modules 155, POI / DAS modules 160, and interface / router modules, as well as X2 connections to other BBU modules 155 via the X2 Mux 150.

[0084] In step 555, the orchestrator 130 can execute instructions for the BBU module 155 experiencing increased demand to have one or more new BBU modules 155 and corresponding cell groups 175 have a predetermined number of UEs performing X2 handover, thereby distributing the load among eligible cell groups 175.

[0085] 8 illustrates a maximum capacity configuration 800 of the system 100 in which all cell groups 175 are distributed such that each cell group 175 has a minimum coverage area due to high demand levels experienced by each. Each cell group 175 has a dedicated remote unit 170, which has a dedicated POI / DAS module 160 and a corresponding dedicated BBU module 155.

[0086] 5, at step 560, orchestrator 130 can send a signal to the core network at the appropriate network operator that system 100 has either increased or decreased its capacity in response to the change in demand. This can be useful information in that the network operator can then have the opportunity to appropriately reallocate network resources to system 100 in response to the change.

[0087] Interference Mitigation Inter-cell interference may occur between UEs located within the coverage area of ​​two cell groups 175. Considering the implementation of the virtualized BBU module 155 with the capacity to communicate with extremely low latency, the system 100 provides an opportunity for Inter-Cell Interference Coordination (ICIC).

[0088] FIG. 6 illustrates an example process 600 by which the orchestrator 130 can provide improved ICIC for adjacent cell groups 175, each of which transmits its own data frame at a Transmit Time Interval (TTI) on the same frequency.

[0089] In step 605, the orchestrator 130 executes instructions to identify two cells that are interfering with each other on a serving frequency. This can be done by two or more BBU modules 155 taking performance measurements (as in step 510), or by the orchestrator 130 receiving an indication from one or more BBU modules 155 that they are experiencing inter-cell interference on a frequency. In step 610, the orchestrator identifies which cell groups 175 are experiencing interference and in which frequency bands.

[0090] In step 615, the orchestrator 130 determines the number of connected UEs that collectively communicate with two or more interfering cells. If the total number of connected UEs is sufficiently below the number that would overload the aggregate cell group, the orchestrator 130 may execute instructions to merge the interfering cell groups 175 into a larger master cell group 175. This may be done in a similar manner as described for steps 520-530 of process 500. This results in a single BBU module 155 (and thus a single scheduler) that handles all of the UEs experiencing inter-cell interference.

[0091] Returning to step 615, if the total number of connected UEs in the interfering cell group 175 is such that they cannot be coalesced into a single cell group 175, the process 600 proceeds to step 625, where the orchestrator 130 executes instructions to instantiate one or more coordinator modules 157. Each coordinator module 157 is assigned to a serving frequency that is causing inter-cell interference. Additionally, each coordinator module 157 has a set of machine-readable instructions that, when executed by one or more processors, provides coordination between two or more interfering BBU modules 155 on the serving frequency identified by the orchestrator 130 in step 610, and performs one of two forms of coordination: inter-cell coordination and intra-frame coordination. In an exemplary embodiment, each coordinator module 157 can be implemented using a COE such as Kubernetes. Alternatively, a single Kubernetes instance can orchestrate the creation, operation, and shutdown of a set of coordinator modules 157.

[0092] In inter-cell coordination, the two BBU modules 155 are located both relatively close to the antennas of one cell group 175 and relatively far from the antennas of the second cell group 175, so that the UEs are close to each other and can coordinate communication with UEs in overlapping coverage service areas. Under inter-cell interference coordination, both of these UEs can use the same resource blocks at the same frequency. In this case, one UE is in communication with a nearby cell group 175, and the other UE is in communication with a relatively distant cell group 175. They can use the same resource blocks at the same frequency, i.e., because they have a difference in delay time between them. Neither UE is looking for the same resource blocks at the same time. Therefore, there is no inter-cell interference. In this case, the two BBU modules 155 can coordinate the handling of UEs in overlapping coverage service areas. In this scenario, the coordinator module 157 can act as a go-between to specify which UEs communicate with which cell group 175 to achieve this coordination. Additionally, the coordinator module 157 can allocate resource blocks between two interfering cell groups 175 such that one cell group is allocated time-dependent coverage at the outer edge of its coverage service area and the other cell group 175 is preferentially allocated inside its coverage service area.

[0093] Step 630 is an alternative to step 625, in which the coordinator module 157 executes instructions to perform intra-frame reconciliation. In this case, the coordinator module 157 communicates with the scheduler 230 of each interfering BBU module 155 via an interconnection channel. Using the interconnection channel in the low latency switch fabric 120 may allow two different schedulers 230 to share a given frame on a TTI basis for UEs that would otherwise cause inter-cell interference. This can be done independently as a function of the serving frequency (e.g., a 10 MHz spectrum chunk), so that there can be several coordinator modules 157, each handling a distinct 10 MHz band. In a variation of step 630, two different schedulers (BBU modules 155) can share a given frame, whereby one BBU module 155 can implement a standard LTE protocol stack and the other BBU module can be a special purpose scheduler for IoT devices such as CatM or NB-IoT. In this case, the coordinator module 157 can reserve a special set of resource blocks for the special purpose scheduler, whereby the relative priority of UEs served by the two schedulers (standard LTE BBU module 155 and special purpose IoT BBU module 155) is maintained. This avoids a situation where the IoT-specific scheduler might reserve resource blocks over a sequence of subframes that would otherwise enable improved coverage use, whereby a single IoT UE with poor coverage is given a large number of resource blocks to preemptively enable HARQ-related retransmissions (e.g., improved coverage). In this case, the coordinator 157 may further act as an arbitrator, ensuring that resource blocks of a common LTE frame are properly provided between the LTE BBU module 155 and the special purpose BBU module 155.

[0094] While the above embodiments have described system 100 as using a low latency switch fabric 120 to communicate between component modules 105, it will be understood that this is by way of example only and that other inter-process communication techniques, standards or technologies may be employed provided they provide sufficient bandwidth, sufficiently low latency, and do not involve a kernel, and are therefore possible and within the scope of the present disclosure.

Claims

1. a persistent machine-readable memory encoded with instructions that, when executed by one or more processors, cause the one or more processors to implement a virtual base station and a process, the process comprising the steps of: determining a user equipment demand for connectivity to a wireless communication access network; determining a combination of component modules based on the connectivity needs of user equipment, the combination of component modules including at least one baseband module and at least one functionally distinct interface / router component, each configured to communicate with a corresponding one of a plurality of external core network components; determining a number of interconnection channels between the at least one baseband module and the at least one interface / router component to meet the needs of the combination of the component modules; instantiating in memory each component module within the combination of component modules; connecting a POI / DAS module (Point of Interconnect / Distributed Antenna System module) or said at least one baseband module to a remote unit; connecting said at least one baseband module to at least one external core network component via said at least one interface / router component corresponding to said at least one external core network component; connecting said at least one baseband module to at least one UE (User Equipment) within coverage of a cell group of said at least one baseband module; Equipped with Each component module has a connection to the low latency switch fabric. Persistent machine readable memory.

2. 10. The persistent machine readable memory of claim 1, wherein each component module has a connection to an Ethernet switch.

3. 10. The persistent machine readable memory of claim 1, wherein the low latency switch fabric comprises an InfiniBand switch.

4. 4. The persistent machine-readable memory of claim 3, wherein the interface / router components are coupled to the at least one baseband module via the low latency switch fabric, and each of the interface / router components is coupled to the corresponding one of the external network components via an Ethernet switch.

5. 5. The persistent machine-readable memory of claim 4, wherein the step of connecting the baseband module to at least one UE comprises: establishing a connection between the POI / DAS (Point of Interconnect / Distributed Antenna System) module and the baseband module over the low latency switch fabric; and and c) establishing a connection between the POI / DAS module and the remote unit.

6. 6. The non-transitory machine readable memory of claim 5, wherein the connection between the POI / DAS module and the remote unit comprises a CPRI connection (Common Public Radio Interface connection).

7. 7. The persistent machine-readable memory of claim 4, wherein the step of connecting the baseband module to at least one UE comprises establishing a connection between the at least one baseband module and the remote unit.

8. 8. The non-transitory machine readable memory of claim 7, wherein the connection between the at least one baseband module and the remote unit comprises a Common Public Radio Interface (CPRI) connection.

9. 10. The non-transitory machine readable memory of claim 8, wherein the CPRI connection comprises a CPRI switch coupled between the at least one baseband module and the remote unit.

10. 10. The non-transitory machine readable memory of any one of claims 2 to 9, further comprising a plurality of baseband modules.

11. 11. The persistent machine readable memory of claim 10, wherein instantiating each of the combinations of component modules comprises allocating a shared memory for a plurality of UE context data.

12. 12. The persistent machine-readable memory of claim 10 or 11, wherein connecting the baseband module to at least one UE comprises writing UE context data corresponding to the at least one UE to a shared memory.

13. 13. The persistent machine readable memory of claim 10, wherein at least one of the plurality of baseband modules comprises an Internet of Things (IoT) scheduler component.

14. 14. The persistent machine readable memory of claim 2, wherein the at least one baseband module comprises: a Packet Data Convergence Protocol (PDCP) component; a Radio Link Control (RLC) component; a scheduler component including a Media Access Control (MAC) component; and PHY (Physical layer) components wherein each of the PDCP component and the PHY component is coupled to the low latency switch fabric.

15. 15. A persistent machine readable memory according to any one of claims 1 to 14, comprising: the determined combination of component modules includes at least one of a plurality of functionally distinct interface / router components, each of which is configured to communicate with a corresponding one of a plurality of external core network components; the determined plurality of interconnection channels are between the at least one baseband module and at least one of the interface / router components; the at least one baseband module is connected to at least one of the external core network components via at least one of the interface / router components, each of the interface / router components being configured to communicate with a corresponding one of a plurality of external network components to which the at least one external core network component corresponds; Persistent machine readable memory.

16. 16. The persistent machine-readable memory of claim 15, wherein at least one of the interface / router components includes an S1, X2, GTP (General Packet Radio Service Tunneling Protocol), or M2M (Machine-to-Machine) interface.

17. 17. A persistent machine readable memory according to claim 15 or 16, wherein the step of determining the demand for connectivity to the wireless communication access network comprises: The persistent machine readable memory includes steps for determining current demand, future demand, and demand volatility.

18. 20. The non-transitory machine readable memory of claim 17, wherein determining the future demand comprises at least one of extrapolating trend data and estimating based on historical data.

19. 20. The non-volatile machine readable memory of claim 17, wherein determining the volatility of demand comprises calculating an estimated deviation of demand over a limited period of time.

20. determining a user equipment demand for connectivity to a wireless communication access network; determining a combination of component modules based on the connectivity needs of user equipment, the combination of component modules including at least one baseband module and at least one functionally distinct interface / router component, each configured to communicate with a corresponding one of a plurality of external core network components; determining a number of interconnection channels between the at least one baseband module and the at least one interface / router component to meet the needs of the combination of the component modules; instantiating in memory each component module within the combination of component modules; connecting a POI / DAS module (Point of Interconnect / Distributed Antenna System module) or said at least one baseband module to a remote unit; connecting said at least one baseband module to at least one external core network component via said at least one interface / router component corresponding to said at least one external core network component; and connecting said at least one baseband module to at least one UE (User Equipment) within coverage of a cell group of said at least one baseband module; Including, Each component module has a connection to the low latency switch fabric. method.

21. 21. The method of claim 20, wherein the step of connecting the baseband module to at least one UE comprises: establishing a connection between the POI / DAS module and the baseband module over a low latency switch fabric of the component module; and establishing a connection between the POI / DAS module and the remote unit.

22. 21. The method of claim 20, wherein connecting the baseband module to at least one UE comprises establishing a connection between the at least one baseband module and the remote unit.

23. 21. The method of claim 20, wherein instantiating each of the combinations of component modules comprises allocating a shared memory for a plurality of UE context data.

24. 24. The method of claim 23, wherein coupling the baseband module to at least one UE comprises writing UE context data corresponding to the at least one UE to the shared memory.

25. 21. The method of claim 20, the determined combination of component modules includes at least one of a plurality of functionally distinct interface / router components, each of which is configured to communicate with a corresponding one of a plurality of external core network components; the determined plurality of interconnection channels are between the at least one baseband module and at least one of the interface / router components; the at least one baseband module is connected to at least one of the external core network components via at least one of the interface / router components, each of the interface / router components being configured to communicate with a corresponding one of a plurality of external network components to which the at least one external core network component corresponds; method.

26. 26. The method of claim 25, wherein the step of determining the demand for connectivity to the wireless communication access network comprises: The method includes determining a current demand, a future demand, and a volatility of the demand.

27. 27. The method of claim 26, wherein determining the future demand comprises at least one of extrapolating trend data and estimating based on historical data.

28. 27. The method of claim 26, wherein the step of determining the volatility of demand comprises calculating an estimated deviation in demand over a limited period of time.

Images