Dynamic configuration of uplink configured grants

Abstract

A method includes collecting performance-related information relating to performance of a first network node, collecting data traffic-related information relating to transmission of data by a user equipment (UE), to the first network node according to a configured grant (CG) generating a prediction, based on the performance-related information and the data traffic- related information, whether a property of the CG would be improved by either performing a handover of one or more UEs to a second network node or, alternatively, by changing a parameter of the CG, and based on the prediction, instructing the network node to perform a handover of the one or more UEs to the second network node or to change the parameter of the CG.

Description

DYNAMIC CONFIGURATION OF UPLINK CONFIGURED GRANTSFIELD

[0001] The present disclosure relates to wireless communication networks, and in particular to the use of configured grants for uplink communications in wireless communication networks.BACKGROUND

[0002] One goal of Fifth Generation (5G) mobile networks is to provide networks with the capability to support ultra reliable low-latency communications (URLLC). URLLC has strict requirements on certain communication parameters, such as packet error loss rate and latency. The third generation partnership project (3GPP) has described that a general URLLC reliability requirement for one transmission of a packet is a block error rate of 10-5 for 32 bytes with a user plane latency of 1 ms.

[0003] To achieve the strict URLLC requirements, one of the strategies proposed is to configure a User Equipment (UE) to transmit without the needing to first send a scheduling request. Specifically, a UE may be configured to periodically transmit uplink communications in what is known as uplink (UL) configured grant (CG) transmission. Uplink CG is illustrated, for example, in Figure 1, which illustrates a conventional dynamic grant transmission (left), versus a configured grant transmission (right) on an uplink interface. In a dynamic grant transmission, the UE must request for a grant from the base station using a scheduling request (SR) message. The grant is communicated back to the UE using a downlink control information (DCI) message, after which the UE transmits a UL signal on a channel, such as the physical uplink shared channel (PUSCH). In contrast, in a CG transmission, the UE can send data to a radio base station (RBS), such as a gNB in a New Radio (NR) communication system, on the PUSCH at predetermined intervals without needing to send a SR or receive a grant for each transmission. Accordingly, latency may be reduced through the use of CG.

[0004] A UE may be provided with a configured grant, which specifies a periodicity P of transmissions, via radio resource control (RRC) signalling. The UE can be configured to automatically transmit multiple repetitions (e.g., K repetitions) of data in grant-free (GF) resources within a given periodicity P to increase reliability. The repetitions can only happenwithin the interval specified by P, as shown in Figure 2, where each period P corresponds to one Hybrid Automated Repeat Request (HARQ) interval.

[0005] An issue arises from the fact that packets for transmission may arrive at the UE’s media access control (MAC) layer for transmission at any point within the period P. Thus, there is a chance that the UE may not be able to use all possible transmission opportunities during the period. For example, as shown in Figure 2, a UE is configured with a CG that specifies K = 4 GF resources per period P. In a first period, the UE receives, at the MAC layer, a packet for transmission prior to the first GF resource of the period. Thus, the UE is able to transmit the packet in all four configured GF resources. In the second period, however, the packet does not arrive at the MAC layer until after the first GF resource. Thus, the UE can only transmit the packet in the last three GF resources. Likewise, in the third period, the packet does not arrive at the MAC layer until after the second GF resource. Thus, the UE can only transmit the packet in the last two GF resources, and so on.

[0006] When the number of actual transmissions are smaller than the configured number available, the reliability of transmission may decrease and latency may increase, as the gNB may need to wait for next period to receive a packet.

[0007] Several solutions to this problem have been suggested. For example, some approaches that have been suggested include transmitting the data with increased power, or using shared resources, or waiting until the next P interval to transmit the data. However, transmitting the data with increased power has a disadvantage in that energy consumption and risk for interference will be increased. In addition, the use of shared resources may pose a risk to reliability, and may also lead to increased latency and resource consumption.

[0008] Another suggested approach is to schedule multiple configurations, and to allow the UE to choose the configuration with closest starting point to transmit all K repetitions. However, this approach may require a significant overhead in network signaling to communicate multiple configurations to the UE. This approach may also increase the computational and storage resource consumption by the UE.

[0009] Another suggested approach is to transmit all repetitions across P-intervals. One issue with this approach is that each period P corresponds to a different HARQ interval. Transmitting over multiple such HARQ intervals would mean that the gNB will not know the HARQ process identifier (i.e., ID) of the original transmission, as this ID changes for every interval. This means that it will be hard then for the gNB to know the sequence of retransmissions or issue an UL grant.

[0010] Yet another suggested approach involves using reserved resources, i.e., preallocated resources that allow transmission outside of HARQ interval. However, this also has the disadvantage that resources are reserved that could otherwise be used for other types of transmission.

[0011] One approach is described in Yan Liu, Yansha Deng, Maged Elkashlan, Arumugam Nallanathan, and George K. Karagiannidis, “Optimization of Grant-Free NOMA with Multiple Configured-Grants for mURLLC,” IEEE Journal on Selected Areas in Communications, Vol. 40, Issue 4 (April 2022). In this approach, the authors use a multi-agent deep reinforcement learning approach in which every agent, for every UE in a gNB configures the number of repetitions and starting slot of each CG assuming a multiple CG configuration.SUMMARY

[0012] A method according to some embodiments includes collecting performance- related information relating to performance of a first network node. Data traffic-related information relating to transmission of data by a user equipment (UE), to the first network node according to a configured grant (CG), is collected. A prediction is generated, based on the performance-related information and the data traffic-related information, of whether a property of the CG would be improved by either performing a handover of one or more UEs to a second network node or, alternatively, by changing a parameter of the CG. Based the prediction, the network node is instructed to perform a handover of the one or more UEs to the second network node or to change the parameter of the CG.

[0013] The parameter of the CG may include a periodicity (P) of the CG or a number of repetitions (K) per period of the CG.

[0014] The performance-related information may include one or more performance metrics collected by the first network node.

[0015] The data traffic-related information may include one or more of latency and / or throughput of the CG connection.

[0016] The property of the CG may include a robustness and / or a latency of data transmitted using the CG.

[0017] In some embodiments, generating the prediction may be performed in response to a trigger event at the network node. The trigger may be based on expiration of a time period. In some embodiments, the trigger event may be based occurrence of an event, wherein the event may include one or more of a packet drop rate, a number of dropped calls, acomparison of throughput to a throughput threshold, a number of active UEs connected to the network node, and / or a resource utilization at the network node exceeding a resource utilization threshold.

[0018] The prediction may be generated according to a policy that takes into account one or more of a state of the UE, a priority of the UE, a level of mobility of the UE between network cells, a movement speed of the UE and a location of the UE.

[0019] The prediction may be generated according to a policy that may be generated using reinforcement learning.

[0020] The policy may be implemented by a neural network.

[0021] The method may further include training the policy according to a reward function that is a function of one or more key performance indicators associated with the first network node and / or the user equipment. The key performance indicators may include, for example, include served traffic, total traffic and latency.

[0022] The reward function may have the form:where:1 if CLV giatency< O-PPiatency latencytaPPlatency avif Ctv giatenCy > O-PPiatency 9 latency avgiatency is an average latency of transmissions using the CG and appiatency is a maximum tolerable latency for transmissions using the CG.

[0023] In some embodiments, the key performance indicators include one or more of RRC setup success rate, ERAB setup success rate, Call Setup Success Rate, Call drop rate, Service Call drop rate, Intra-Frequency Handover Out Success Rate, Inter-Frequency Handover Out Success Rate, Inter-RAT Handover Out Success Rate, E-UTRAN IPThroughput, IP Throughput in DL, E-UTRAN IP Latency, E-UTRAN Cell Availability, Partial cell availability, and / or Mean Active Dedicated EPS Bearer Utilization. More generally, the key performance indicators may include one or more of the those mentioned in 3GPP TS 28.554 V18.3.1 (2023-09) or 3GPP TS 32.451 V17.0.0 (2022-04).

[0024] The policy may be trained using multi-agent reinforcement learning. The training may be performed using collaborative multi-agent deep reinforcement learning in which multiple agents try to optimize against a global reward. The policy may be trained by optimization against a global reward.

[0025] Each agent may be associated with a different radio base station.

[0026] Some embodiments provide an uplink configured grant configuration device adapted to perform the operations of collecting performance-related information relating to performance of a first network node, collecting data traffic-related information relating to transmission of data by a UE, to the first network node according to a CG, generating a prediction, based on the performance-related information and the data traffic-related information, whether a property of the CG would be improved by either performing a handover of one or more UEs to a second network node or, alternatively, by changing a parameter of the CG, and based on the prediction, instructing the network node to perform a handover of the one or more UEs to the second network node or to change the parameter of the CG.

[0027] Some embodiments provide an uplink configured grant configuration device including a processing circuitry, and a memory coupled to the processing circuitry. The memory includes computer-readable instructions that, when executed by the processing circuitry, cause the uplink configured grant configuration device to perform the operations of collecting performance-related information relating to performance of a first network node, and collecting data traffic-related information relating to transmission of data by a UE, to the first network node according to a CG. The operations further include generating a prediction, based on the performance-related information and the data traffic-related information, whether a property of the CG would be improved by either performing a handover of one or more UEs to a second network node or, alternatively, by changing a parameter of the CG, and based on the prediction, instructing the network node to perform a handover of the one or more UEs to the second network node or to change the parameter of the CG.

[0028] Some embodiments provide a computer program product including a non- transitory medium that stores computer-readable instructions that, when executed by processing circuitry, perform the operations of collecting performance-related information relating to performance of a first network node, and collecting data traffic-related information relating to transmission of data by a UE, to the first network node according to a CG. The operations further include generating a prediction, based on the performance-related information and the data traffic-related information, whether a property of the CG would be improved by either performing a handover of one or more UEs to a second network node or, alternatively, by changing a parameter of the CG, and based on the prediction, instructing the network node to perform a handover of the one or more UEs to the second network node or to change the parameter of the CG.

[0029] A system according to some embodiments includes a user equipment and an uplink configured grant configuration device, UCC . The UCC collects performance-related information relating to performance of a first network node and collects data traffic-related information relating to transmission of data by the UE, to the first network node according to a configured grant, CG. The UCC generates a prediction, based on the performance-related information and the data traffic-related information, whether a property of the CG would be improved by performing a handover of the UE to a second network node; and based on the prediction, instructs the network node to perform the handover of the UE to the second network node. The UE receives a handover command to perform the handover to the second network node, and performs the handover in response to the handover command.BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 illustrates a conventional dynamic grant transmission versus a configured grant transmission on an uplink interface.

[0031] Figure 2 illustrates an example of transmission using a configured grant with four grant-free transmission resources per period.

[0032] Figure 3 is a block diagram that illustrates components and messaging flows according to some embodiments.

[0033] Figures 4A and 4B are sequence diagrams that illustrates operations according to some embodiments.

[0034] Figure 5 illustrates a multi-agent deep reinforcement learning system that may be used for training the policy applied by the UCC.

[0035] Figure 6 is a graph that illustrates average reward rate as a function of training episodes for different embodiments.

[0036] Figure 7 is a graph of performance of a trained agent in terms of reward r for various embodiments.

[0037] Figure 8 illustrates a UCC implemented as a standalone system according to some embodiments..

[0038] Figure 9 illustrates operations of a UCC according to some embodiments.

[0039]

[0040] Figure 10 shows an example of a communication system in accordance with some embodiments.

[0041] Figure 11 shows a UE in accordance with some embodiments.

[0042] Figure 12 shows a network node in accordance with some embodiments.

[0043] Figure 13 is a block diagram of a host in accordance with various aspects described herein.

[0044] Figure 14 is a block diagram illustrating a virtualization environment in which functions implemented by some embodiments may be virtualized.

[0045] Figure 15 shows a communication diagram of a host communicating via a network node with a UE over a partially wireless connection in accordance with some embodiments.DETAILED DESCRIPTION

[0046] As described above, configured grants may be used to reduce latency of uplink transmissions by a UE. However, there currently exist certain challenges. For example, some transmission resources that are configured using a configured grant may be unused depending on the timing of data packets at the UE.

[0047] Previous approaches to address this problem focus on increasing the chances that a UE will transmit each packet in a given number of repetitions, using different mechanisms such as shared or reserved resources, cross-interval transmissions, and multiple CG configurations. However, such approaches may require pre-allocation of spectrum resources, and may increase reliability at the expense of latency.

[0048] For multiple CG configurations, some approaches uses deep reinforcement learning to configure one or more retransmission parameters, such as the number of repetitions and / or the HARQ slot for starting the transmission. Some approaches use a cooperative multiagent approach in which each agent is responsible for configuration of repetitions and start slot for one CG, and all agents are centralized in one RBS. However, this approach may consume a large amount of computational resources, as one agent must be trained for every CG configuration.

[0049] Conventional approaches may assume that spectrum and computational resources are always available at a RBS. This is not generally the case in real environments, as RBSs can become overloaded for various reasons.

[0050] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges. For example, some embodiments described herein may address one or more of these issues by providing a method for adjustment of periodicity of aCG in combination with initiating a handover to meet throughput and latency requirements for uplink URLLC traffic.

[0051] In particular, some embodiments provide systems and / or methods for configuring periodicity intervals of UL CG transmissions for a given number of repetitions. The systems / methods may use a multi-agent deep reinforcement learning (MARL) approach in which each agent is responsible for configuring the CG periodicity P of UEs served by a corresponding RBS (i.e. , one agent per RBS).

[0052] Some embodiments may be particularly useful in private dedicated networks, such as private 5G networks, because the approach optimizes for a complete network rather than individual RBSs as previously done. This is achieved by also enabling handovers of UEs from one cell with fewer resources to another neighboring cell has more resources available that can provide connectivity to the UEs. At the same time, some embodiments may only require a reasonable, typically smaller, amount of computational resources.

[0053] Certain embodiments may provide one or more technical advantages. For example, some embodiments described herein may improve resource utilization across multiple cells by scheduling resources via CG based on predicted UE traffic patterns.

[0054] Some embodiments may enhance delay performance for UEs by finding an optimal periodicity at the moment and adapting quickly to traffic demand changes.

[0055] Some embodiments may have the additional benefit of requiring reduced training time for agents in a RL system, as the solution uses a common experience pool.

[0056] Additionally, some embodiments can be used for offloading cells in addition to configuring uplink capacity.

[0057] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

[0058] Figure 3 is a block diagram that illustrates components and messaging flows according to some embodiments. In particular, some operations described herein are performed by a UL CG Configuration unit (UCC) 300, which may be implemented within a corresponding RBS. Alternatively, the UCC 300 may be implemented as a standalone unit that communicates with an RBS, or may be implemented as a network function that resides in a core network and that communicates with the RBS over a communication link.

[0059] Two data collection components provide data to the UCC 300, namely, a UE data collector unit 304 that collects UE-specific data and an RBS data collector unit 302 thatcollects RBS-specific data. The UE data collector unit 304 and the RBS data collector unit 302 may be provided as standard observability components within a communication network.

[0060] The UE data collector unit 304 may collect data traffic-related information, such as latency and throughput on uplink and downlink connections. The RBS data collector unit 302 may provide RBS-level data, such as total load of the RBS, which may be expressed in terms of the number of attached UEs, CPU, and memory utilization, etc. The RBS data collector unit 302 may additionally provide information such as performance monitor (PM) counters or alarms collected by a baseband processor of an RBS

[0061] Both UE and RBS related data is input to the UCC unit 300. Based on the provided data, the UCC unit 300 applies or executes a policy to decide whether to reconfigure a parameter of an UL CG that has been granted to a UE served by the RBS from a currently set value to a new value. The parameter may include, for example, a periodicity P of the UL CG, a number of repetitions K per period for the UL CG, a starting resource for the UL CG, or other parameter of the CG.

[0062] The policy may be implemented, for example, by a machine learning model, such as a deep neural network, that may be trained, for example, using reinforcement learning, to determine an action that can be taken by the RBS that is expected to produce a highest reward based on observations provided by the UE data collector unit 304 and the RBS data collector unit 302. The reward may be based on an improvement to a metric associated with the network, such as a key performance indicator (KPI), a metric associated with the CG, such as reliability and / or latency, or other metric. In some embodiments, the KPI is an “energy efficiency” metric, as defined section 5.6.1.2 of 3GPP TS 32.451 V17.0.0 (2022-04).

[0063] In some embodiments, the key performance indicators may include one or more of RRC setup success rate, ERAB setup success rate, Call Setup Success Rate, Call drop rate, Service Call drop rate, Intra-Frequency Handover Out Success Rate, Inter-Frequency Handover Out Success Rate, Inter-RAT Handover Out Success Rate, E-UTRAN IP Throughput, IP Throughput in DL, E-UTRAN IP Latency, E-UTRAN Cell Availability, Partial cell availability, and / or Mean Active Dedicated EPS Bearer Utilization. In some embodiments, the key performance indicators comprise one or more of the those mentioned in 3GPP TS 28.554 V18.3.1 (2023-09) or 3GPP TS 32.451 V17.0.0 (2022-04).In particular, the UCC unit 300 may process the data retrieved from the UE data collector unit 304 and the RBS data collector unit 302 as input. If the UCC 300 determines that a property associated with the CG may be improved by taking an action, such as changingthe configured grant and / or performing handover (HO) of one or more UEs to another cell, the UCC 300 may output a suggestion to the RBS to perform the action. The property of the CG may, for example, be a robustness of communications using the CG to channel conditions, a throughput, a latency of data transmitted using the CG, or other property. In this context, "robustness" covers completeness of data that may be checked or measured using known techniques such as CRC check or those known from the transmission control protocol (TCP).

[0064] If the UCC 300 decides based on the policy that a parameter, such the periodicity P, of the CG should be changed, the decision is then relayed to the transmitter component 308 of the RBS, which decides when to transmit this information to the UE. To reconfigure the CG, the RBS may need to reconfigure the RRC session with the UE via an RRC connection setup or reconfiguration message. This message may be transmitted sooner for an inactive UE than for active UE to avoid interfering with traffic requirements of the active UE.

[0065] If the UCC 300 decides based on the policy that one or more UEs should be handed over to another cell, the decision is related to a handover component 306 of the RBS, which may in response initiate a handover of the one or more UEs in accordance with the recommendation from the UCC 300.

[0066] Figure 4A is a sequence diagram that illustrates operations according to some embodiments. As shown therein, the UE data collector (UEDC) 304 provides UE traffic- related data 402 to the UCC, while the RBS data collector (RBSDC) 302 provides RBS-related data 404 to the UCC 300. The UCC 300 continuously observes the UE traffic-related data 402 and RBS-related data 404 provided by the UEDC 304 and the RBSDC 302. Based on the data provided, the UCC 300 triggers 406 a policy to adjust a parameter of the CG and / or to initiate a handover of one or more UEs.

[0067] The trigger that causes the UCC 300 to execute the policy may be based on expiration of a time period and / or occurrence of an event. In some embodiments, the trigger that causes the UCC 300 to execute the policy may occur when one or more observations exceed a predefined limit. For example, execution of the policy may be triggered when the packet drop rate on the UL interface exceeds a predetermined amount, the number of dropped calls (for voice sessions) exceeds a predetermined amount, the aggregate throughput on the UL exceeds a threshold level, the number of attached UE that are active exceeds a predetermined number, the load on the RBS (e.g., in terms of CPU utilization or memory utilization, or related to the ratio of UEs to schedulable resources in the RBS) exceeds a threshold, etc.

[0068] The policy of the UCC 300 may be implemented using a deep neural network that takes as input a combination of UE traffic data 402 and RBS load data 404 and generates a prediction of whether a property of the CG would be improved by performing a handover of one or more UEs to another RBS and / or by changing a parameter of the CG. . That is, the output of the policy may include (a) a recommendation to handover one or more UEs, (b) a recommendation to change a parameter of a CG, (c) a recommendation both to handover one or more UEs and a recommendation to change a parameter of a CG, or (d) no recommendation, based on the prediction of whether such action would improve a property of the CG.

[0069] As illustrated in Figure 4A, the UCC 300 may determine based on application of the policy that one or more UEs should be handed over from the RBS to a neighboring cell. In that case, the UCC 300 sends a message 408 to the HO unit 306 recommending that the HO unit select one or more UEs for handover. In some embodiments, the message 408 may indicate a particular UE to handover. In other embodiments, the message 408 may indicate a number of UEs to handover or a percentage of UEs connected to the RBS to handover. The HO unit 306 then performs the handover 410.

[0070] The HO unit 306 may decide which UE(s) to handover based one or more criteria. For example, the decision of which UE(s) to handover may be based on an indication of historical mobility of the UE, i.e., how fast did UE historically handover from one cell to another. If a UE performs handover faster or more often than another UE, that may indicate that the UE is more mobile and therefore may be preferred for handover.

[0071] Another indication of mobility can be retrieved from a beamforming manager in case of multiple input multiple output (MIMO) connectivity in which UEs are tracked by narrow beams. The location and direction of movement of a UE can be identified by the beam.

[0072] Yet another indicator, especially in ultra-high frequency bands may be obtained based on Joint Communication and Sensing (JCAS) technology, which uses radarlike technology to locate objects.

[0073] Accordingly, a system according to some embodiments includes a UE 100 and an uplink configured grant configuration device, UCC 300. The UCC 300 collects performance-related information 404 relating to performance of a first network node and collects data traffic-related information relating 402 to transmission of data by the UE 100, to the first network node according to a CG. The UCC 300 generates a prediction, based on the performance-related information and the data traffic-related information, whether a property of the CG would be improved by performing a handover of the UE 100 to a second networknode, and based on the prediction, instructs (408) the network node to perform the handover of the UE 100 to the second network node. The UE 100 receives a handover command to perform the handover to the second network node, and performs the handover in response to the handover command.

[0074] Figure 4B is a sequence diagram that illustrates operations according to further embodiments. As shown therein, the UE data collector (UEDC) 304 provides UE traffic-related data 402 to the UCC, while the RBS data collector (RBSDC) 302 provides RBS- related data 404 to the UCC 300. The UCC 300 continuously observes the UE traffic-related data 402 and RBS-related data 404 provided by the UEDC 304 and the RBSDC 302. Based on the data provided, the UCC 300 triggers 406 a policy that generates a recommendation to adjust a parameter of the CG.

[0075] In particular, the policy may output a new parameter 412, such as a new periodicity (P new) that is to be applied to a given CG that has been configured at a UE. The parameter is communicated to the transmitter (TX) 308 of the RBS for configuration. The transmitter decides (414), based on which CG is being updated, when to apply the new parameter within a time threshold. This decision can be made on an individual UE basis and may be based on factors such as an activity of the UE. For example, if the state of the UE is IDLE, the CG update may be immediate as it may not affect data traffic. On the other hand, if the UE is in an ACTIVE state, the CG update may be delayed, such as based on a prediction on when data traffic from UE will stop.

[0076] In some embodiments, the TX 308 may decide when to update the parameter of the CG based on a policy of the UE. For example, for a UE with low priority best-effort type of traffic (e.g., those with flows that have a QoS Class Identifier 9, or those using besteffort enhanced mobile broadband type of network slices), a CG parameter update could be transferred immediately. On the other hand, for a mission-critical UE (e.g., those that have flows with QoS Class Identifier less than 9, or for example 65, 66, 67 or those using mission- critical, ultra-reliable low latency communications network slices), the CG parameter update may be delayed until it is determined that the update will not affect the performance of the UE.

[0077] The TX 308 transmits the new parameter to the UE 100, for example, in an RRC reconfiguration message 416 that includes the updated parameter P new in its payload. The UE 100 then applies the new parameter to the CG configuration (block 418).

[0078] Figure 5 illustrates a multi-agent deep reinforcement learning system that may be used for training the policy it applied by the UCC 300 of each RBS. A set of agents502-1, 502-X is provided, where each agent corresponds to a single cell in the network. Each of the agents 502-1, 502-X receives observations o from an environment 510, and based on the agent’s policy it. takes an action a on the environment 510. The observed state of the environment in response to the action is updated, and the agent 502-1, 502-X then receives a reward r in response to the updated state. The agent 502-1, 502-X updates its policy abased on the reward received. The agents 502-1, 502-X may store and pull experiences from an experience pool 504.

[0079] The experience pool stores “experiences” from different agents, where an “experience” is represented as a 4-tuple comprising <state, action, new state, reward>. An experience represents the “learning” of an agent that given a state of the environment, when the agent took a specified action, it received a reward and observed a new state. As the experience pool contains contributions from many agents, learnings from one agent can be used to train another. Thus, an agent may not have to take a random action / explore every state it encounters.

[0080] That is, in each state of the environment 510, an agent 502-1, 502-X takes an action a and receives a reward r for the optimality of the action as well as a new state. Over time, the agent 502-1, 502-X leams a policy that it believes will take the best action for the highest reward given the observed state of the environment 510.

[0081] It is assumed that the agents 502-1, 502-X have partial observability of the environment 510, i.e., they can observe only a subset of the complete state. At the same time, the agents are collaborative, in that they try to optimize the same global reward.

[0082] As part of the “observation” space, the system considers the aggregate UL throughput of UE in a cell, as well as the average latency. Additionally, from the RBS side, an indication of load of the RBS is provided, which is itself an average of the ratio of active UE to total supported UE, CPU and memory utilization.

[0083] As an “action” space, the system consider first a parameter of the CG such as periodicity P: P 6 [1... 8], which indicates the interval in time of PUSCH resources available for UL transmission by the UE. The system may also consider a percentage of UEs for handover such that HO: HOG [[0,5), [5, 10),... ,[95,100]], therefore A = [P, HO],

[0084] Based on the definition of “observation” and “action”, the following “reward” may be used for learning:where latency t is defined as:

[0085] In the formulas above, appiatency indicates the maximum tolerable latency for URLLC application. It may be set to a constant, such as 15ms. This value may be determined at implementation by a designer. The value avgiatency is an average latency of transmissions using the CG.

[0086] Figure 6 is a graph that illustrates average reward rate as a function of training episodes for a first embodiment that only adjusts the periodicity P of a CG and a second embodiment in which both the periodicity P of a CG is adjusted (curve 602) and a number of repetitions (repK) of the CG is adjusted (curve 604). As illustrated in Figure 6, the average reward rate tends to be higher when both the periodicity P and the repetition rate can be adjusted based on application of the policy it.

[0087] Figure 7 is a graph of performance of a trained agent in terms of reward r for various embodiments. In a baseline graph (curve 702), the system is static in that the CG parameters are fixed and no handover is performed. In a first embodiment (curve 704), the CG parameters may be adjusted based on the applied policy, but no handovers are performed, and in a second embodiment (curve 706), both the CG parameters may be adjusted and handovers may be performed based on the applied policy. As seen in Figure 7, the best performance is obtained using the second embodiment.

[0088] Figure 8 illustrates a UCC 300 implemented as a standalone system including a processing circuitry 312, a memory 314 coupled to the processing circuitry or that stores computer-readable instructions executable by the processing circuitry 312 for performing the operations described above, and a transceiver 316 coupled to the processing circuitry for communicating with an RBS, the UEDC 304 and / or the UEDC 302.

[0089] The UCC 300 is configured to perform the operations described above. In particular, the UCC is configured to perform the operations of collecting performance-related information relating to performance of a first network node, and collecting data traffic-related information relating to transmission of data by a UE, to the first network node according to a CG. The UCC 300 generates a prediction, based on the performance-related information and the data traffic-related information, whether a property of the CG would be improved by either performing a handover of one or more UEs to a second network node or, alternatively, by changing a parameter of the CG, and based on the prediction, instructs the network node to perform a handover of the one or more UEs to the second network node or to change the parameter of the CG.

[0090] Figure 9 illustrates operations of a UCC 300 according to some embodiments. Referring to Figure 9, a method according to some embodiments includes collecting performance-related information relating to performance of a first network node (block 902), and collecting data traffic-related information relating to transmission of data by a UE to the first network node according to a configured grant (CG) (block 904).

[0091] A prediction is generated (block 906), based on the performance-related information and the data traffic-related information, of whether a property of the CG would be improved by either performing a handover of one or more UEs to a second network node or, alternatively, by changing a parameter of the CG. Based on the prediction, the network node is instructed (block 908) to perform a handover of the one or more UEs to the second network node or to change the parameter of the CG.

[0092] Figure 10 shows an example of a communication system 1000 in accordance with some embodiments.

[0093] In the example, the communication system 1000 includes a telecommunication network 1002 that includes an access network 1004, such as a radio access network (RAN), and a core network 1006, which includes one or more core network nodes 1008. The access network 1004 includes one or more access network nodes, such as network nodes 1010a and 1010b (one or more of which may be generally referred to as network nodes 1010), or any other similar 3rdGeneration Partnership Project (3GPP) access nodes or non- 3GPP access points. Moreover, as will be appreciated by those of skill in the art, a network node is not necessarily limited to an implementation in which a radio portion and a baseband portion are supplied and integrated by a single vendor. Thus, it will be understood that network nodes include disaggregated implementations or portions thereof. For example, in someembodiments, the telecommunication network 1002 includes one or more Open-RAN (ORAN) network nodes. An ORAN network node is a node in the telecommunication network 1002 that supports an ORAN specification (e.g., a specification published by the O-RAN Alliance, or any similar organization) and may operate alone or together with other nodes to implement one or more functionalities of any node in the telecommunication network 1002, including one or more network nodes 1010 and / or core network nodes 1008.

[0094] Examples of an ORAN network node include an open radio unit (O-RU), an open distributed unit (O-DU), an open central unit (O-CU), including an O-CU control plane (O-CU-CP) or an O-CU user plane (O-CU-UP), a RAN intelligent controller (near-real time or non-real time) hosting software or software plug-ins, such as a near-real time control application (e.g., xApp) or anon-real time control application (e.g., rApp), or any combination thereof (the adjective “open” designating support of an ORAN specification). The network node may support a specification by, for example, supporting an interface defined by the ORAN specification, such as an Al, Fl, Wl, El, E2, X2, Xn interface, an open fronthaul user plane interface, or an open fronthaul management plane interface. Moreover, an ORAN access node may be a logical node in a physical node. Furthermore, an ORAN network node may be implemented in a virtualization environment (described further below) in which one or more network functions are virtualized. For example, the virtualization environment may include an O-Cloud computing platform orchestrated by a Service Management and Orchestration Framework via an O-2 interface defined by the O-RAN Alliance or comparable technologies. The network nodes 1010 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 1012a, 1012b, 1012c, and 1012d (one or more of which may be generally referred to as UEs 1012) to the core network 1006 over one or more wireless connections.

[0095] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 1000 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 1000 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0096] The UEs 1012 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 1010 and other communication devices. Similarly, the network nodes 1010 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 1012 and / or with other network nodes or equipment in the telecommunication network 1002 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 1002.

[0097] In the depicted example, the core network 1006 connects the network nodes 1010 to one or more hosts, such as host 1016. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 1006 includes one more core network nodes (e.g., core network node 1008) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 1008. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0098] The host 1016 may be under the ownership or control of a service provider other than an operator or provider of the access network 1004 and / or the telecommunication network 1002, and may be operated by the service provider or on behalf of the service provider. The host 1016 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0099] As a whole, the communication system 1000 of Figure 10 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such asspecific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0100] In some examples, the telecommunication network 1002 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 1002 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 1002. For example, the telecommunications network 1002 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC) / Massive loT services to yet further UEs.

[0101] In some examples, the UEs 1012 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 1004 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 1004. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi -radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN- DC).

[0102] In the example, the hub 1014 communicates with the access network 1004 to facilitate indirect communication between one or more UEs (e.g., UE 1012c and / or 1012d) and network nodes (e.g., network node 1010b). In some examples, the hub 1014 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 1014 may be a broadband router enabling access to the core network 1006 for the UEs. As another example, the hub 1014 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 1010, or byexecutable code, script, process, or other instructions in the hub 1014. As another example, the hub 1014 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 1014 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 1014 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 1014 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 1014 acts as a proxy server or orchestrator for the UEs, in particular if one or more of the UEs are low energy loT devices.

[0103] The hub 1014 may have a constant / persistent or intermittent connection to the network node 1010b. The hub 1014 may also allow for a different communication scheme and / or schedule between the hub 1014 and UEs (e.g., UE 1012c and / or 1012d), and between the hub 1014 and the core network 1006. In other examples, the hub 1014 is connected to the core network 1006 and / or one or more UEs via a wired connection. Moreover, the hub 1014 may be configured to connect to an M2M service provider over the access network 1004 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 1010 while still connected via the hub 1014 via a wired or wireless connection. In some embodiments, the hub 1014 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 1010b. In other embodiments, the hub 1014 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 1010b, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0104] Figure 11 shows a UE 1100 in accordance with some embodiments. As used herein, a UE refers to a device capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle, vehicle-mounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3 GPP), including a narrow bandinternet of things (NB-IoT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE.

[0105] A UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle- to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

[0106] The UE 1100 includes processing circuitry 1102 that is operatively coupled via a bus 1104 to an input / output interface 1106, a power source 1108, a memory 1110, a communication interface 1112, and / or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in Figure 11. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0107] The processing circuitry 1102 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 1110. The processing circuitry 1102 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 1102 may include multiple central processing units (CPUs).

[0108] In the example, the input / output interface 1106 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or anycombination thereof. An input device may allow a user to capture information into the UE 1100. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

[0109] In some embodiments, the power source 1108 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 1108 may further include power circuitry for delivering power from the power source 1108 itself, and / or an external power source, to the various parts of the UE 1100 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 1108. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 1108 to make the power suitable for the respective components of the UE 1100 to which power is supplied.

[0110] The memory 1110 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 1110 includes one or more application programs 1114, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 1116. The memory 1110 may store, for use by the UE 1100, any of a variety of various operating systems or combinations of operating systems.[oni] The memory 1110 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module(DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUICC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 1110 may allow the UE 1100 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 1110, which may be or comprise a device-readable storage medium.

[0112] The processing circuitry 1102 may be configured to communicate with an access network or other network using the communication interface 1112. The communication interface 1112 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 1122. The communication interface 1112 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network). Each transceiver may include a transmitter 1118 and / or a receiver 1120 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 1118 and receiver 1120 may be coupled to one or more antennas (e.g., antenna 1122) and may share circuit components, software or firmware, or alternatively be implemented separately.

[0113] In the illustrated embodiment, communication functions of the communication interface 1112 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / intemet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

[0114] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 1112, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).

[0115] As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.

[0116] A UE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and / or software in dependence of the intended application of the loT device in addition to other components as described in relation to the UE 1100 shown in Figure 11.

[0117] As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements, and transmits theresults of such monitoring and / or measurements to another UE and / or a network node. The UE may in this case be an M2M device, which may in a 3 GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-IoT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.

[0118] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.

[0119] Figure 12 shows a network node 1200 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with a UE and / or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)), O-RAN nodes or components of an O-RAN node (e.g., O-RU, O-DU, O-CU).

[0120] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units, distributed units (e.g., in an O-RAN access node) and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0121] Other examples of network nodes include multiple transmission point (multi - TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, networkcontrollers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).

[0122] The network node 1200 includes a processing circuitry 1202, a memory 1204, a communication interface 1206, and a power source 1208. The network node 1200 may be composed of multiple physically separate components (e.g., aNodeB component and aRNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 1200 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 1200 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1204 for different RATs) and some components may be reused (e.g., a same antenna 1210 may be shared by different RATs). The network node 1200 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1200, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1200.

[0123] The processing circuitry 1202 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 1200 components, such as the memory 1204, to provide network node 1200 functionality.

[0124] In some embodiments, the processing circuitry 1202 includes a system on a chip (SOC). In some embodiments, the processing circuitry 1202 includes one or more of radio frequency (RF) transceiver circuitry 1212 and baseband processing circuitry 1214. In some embodiments, the radio frequency (RF) transceiver circuitry 1212 and the baseband processingcircuitry 1214 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 1212 and baseband processing circuitry 1214 may be on the same chip or set of chips, boards, or units.

[0125] The memory 1204 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processing circuitry 1202. The memory 1204 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions capable of being executed by the processing circuitry 1202 and utilized by the network node 1200. The memory 1204 may be used to store any calculations made by the processing circuitry 1202 and / or any data received via the communication interface 1206. In some embodiments, the processing circuitry 1202 and memory 1204 is integrated.

[0126] The communication interface 1206 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, the communication interface 1206 comprises port(s) / terminal(s) 1216 to send and receive data, for example to and from a network over a wired connection. The communication interface 1206 also includes radio front-end circuitry 1218 that may be coupled to, or in certain embodiments a part of, the antenna 1210. Radio front-end circuitry 1218 comprises filters 1220 and amplifiers 1222. The radio front-end circuitry 1218 may be connected to an antenna 1210 and processing circuitry 1202. The radio front-end circuitry may be configured to condition signals communicated between antenna 1210 and processing circuitry 1202. The radio front-end circuitry 1218 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 1218 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 1220 and / or amplifiers 1222. The radio signal may then be transmitted via the antenna 1210. Similarly, when receiving data, the antenna 1210 may collect radio signals which are then converted into digital data by the radio front-end circuitry 1218. The digital data may be passed to the processing circuitry 1202. In other embodiments,the communication interface may comprise different components and / or different combinations of components.

[0127] In certain alternative embodiments, the network node 1200 does not include separate radio front-end circuitry 1218, instead, the processing circuitry 1202 includes radio front-end circuitry and is connected to the antenna 1210. Similarly, in some embodiments, all or some of the RF transceiver circuitry 1212 is part of the communication interface 1206. In still other embodiments, the communication interface 1206 includes one or more ports or terminals 1216, the radio front-end circuitry 1218, and the RF transceiver circuitry 1212, as part of a radio unit (not shown), and the communication interface 1206 communicates with the baseband processing circuitry 1214, which is part of a digital unit (not shown).

[0128] The antenna 1210 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. The antenna 1210 may be coupled to the radio front-end circuitry 1218 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, the antenna 1210 is separate from the network node 1200 and connectable to the network node 1200 through an interface or port.

[0129] The antenna 1210, communication interface 1206, and / or the processing circuitry 1202 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a UE, another network node and / or any other network equipment. Similarly, the antenna 1210, the communication interface 1206, and / or the processing circuitry 1202 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.

[0130] The power source 1208 provides power to the various components of network node 1200 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 1208 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 1200 with power for performing the functionality described herein. For example, the network node 1200 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 1208. As a further example, the power source 1208 may comprise a source of power in the form of a battery orbattery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0131] Embodiments of the network node 1200 may include additional components beyond those shown in Figure 12 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, the network node 1200 may include user interface equipment to allow input of information into the network node 1200 and to allow output of information from the network node 1200. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 1200.

[0132] Figure 13 is a block diagram of a host 1300, which may be an embodiment of the host 1016 of Figure 10, in accordance with various aspects described herein. As used herein, the host 1300 may be or comprise various combinations hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 1300 may provide one or more services to one or more UEs.

[0133] The host 1300 includes processing circuitry 1302 that is operatively coupled via a bus 1304 to an input / output interface 1306, a network interface 1308, a power source 1310, and a memory 1312. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 11 and 12, such that the descriptions thereof are generally applicable to the corresponding components of host 1300.

[0134] The memory 1312 may include one or more computer programs including one or more host application programs 1314 and data 1316, which may include user data, e.g., data generated by a UE for the host 1300 or data generated by the host 1300 for a UE. Embodiments of the host 1300 may utilize only a subset or all of the components shown. The host application programs 1314 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 1314 may also provide for user authentication and licensing checks and may periodically report health,routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 1300 may select and / or indicate a different host for over-the- top services for aUE. The host application programs 1314 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real- Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0135] Figure 14 is a block diagram illustrating a virtualization environment 1400 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1400 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized. In some embodiments, the virtualization environment 1400 includes components defined by the O-RAN Alliance, such as an O-Cloud environment orchestrated by a Service Management and Orchestration Framework via an O-2 interface.

[0136] Applications 1402 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment Q400 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0137] Hardware 1404 includes processing circuitry, memory that stores software and / or instructions executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1406 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1408a and 1408b (one or more of which may be generally referred to as VMs 1408), and / or perform any of the functions, features and / or benefits described in relation with someembodiments described herein. The virtualization layer 1406 may present a virtual operating platform that appears like networking hardware to the VMs 1408.

[0138] The VMs 1408 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1406. Different embodiments of the instance of a virtual appliance 1402 may be implemented on one or more of VMs 1408, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[0139] In the context of NFV, a VM 1408 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each of the VMs 1408, and that part of hardware 1404 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1408 on top of the hardware 1404 and corresponds to the application 1402.

[0140] Hardware 1404 may be implemented in a standalone network node with generic or specific components. Hardware 1404 may implement some functions via virtualization. Alternatively, hardware 1404 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1410, which, among others, oversees lifecycle management of applications 1402. In some embodiments, hardware 1404 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 1412 which may alternatively be used for communication between hardware nodes and radio units.

[0141] Figure 15 shows a communication diagram of a host 1502 communicating via a network node 1504 with a UE 1506 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments,of the UE (such as a UE 1012a of Figure 10 and / or UE 1100 of Figure 11), network node (such as network node 1010a of Figure 10 and / or network node 1200 of Figure 12), and host (such as host 1016 of Figure 10 and / or host 1300 of Figure 13) discussed in the preceding paragraphs will now be described with reference to Figure 15.

[0142] Like host 1300, embodiments of host 1502 include hardware, such as a communication interface, processing circuitry, and memory. The host 1502 also includes software, which is stored in or accessible by the host 1502 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 1506 connecting via an over-the-top (OTT) connection 1550 extending between the UE 1506 and host 1502. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 1550.

[0143] The network node 1504 includes hardware enabling it to communicate with the host 1502 and UE 1506. The connection 1560 may be direct or pass through a core network (like core network 1006 of Figure 10) and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.

[0144] The UE 1506 includes hardware and software, which is stored in or accessible by UE 1506 and executable by the UE’s processing circuitry. The software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 1506 with the support of the host 1502. In the host 1502, an executing host application may communicate with the executing client application via the OTT connection 1550 terminating at the UE 1506 and host 1502. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 1550 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 1550.

[0145] The OTT connection 1550 may extend via a connection 1560 between the host 1502 and the network node 1504 and via a wireless connection 1570 between the network node 1504 and the UE 1506 to provide the connection between the host 1502 and the UE 1506. The connection 1560 and wireless connection 1570, over which the OTT connection 1550 may be provided, have been drawn abstractly to illustrate the communication between the host 1502and the UE 1506 via the network node 1504, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

[0146] As an example of transmitting data via the OTT connection 1550, in step 1508, the host 1502 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 1506. In other embodiments, the user data is associated with a UE 1506 that shares data with the host 1502 without explicit human interaction. In step 1510, the host 1502 initiates a transmission carrying the user data towards the UE 1506. The host 1502 may initiate the transmission responsive to a request transmitted by the UE 1506. The request may be caused by human interaction with the UE 1506 or by operation of the client application executing on the UE 1506. The transmission may pass via the network node 1504, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1512, the network node 1504 transmits to the UE 1506 the user data that was carried in the transmission that the host 1502 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1514, the UE 1506 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 1506 associated with the host application executed by the host 1502.

[0147] In some examples, the UE 1506 executes a client application which provides user data to the host 1502. The user data may be provided in reaction or response to the data received from the host 1502. Accordingly, in step 1516, the UE 1506 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input / output interface of the UE 1506. Regardless of the specific manner in which the user data was provided, the UE 1506 initiates, in step 1518, transmission of the user data towards the host 1502 via the network node 1504. In step 1520, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 1504 receives user data from the UE 1506 and initiates transmission of the received user data towards the host 1502. In step 1522, the host 1502 receives the user data carried in the transmission initiated by the UE 1506.

[0148] One or more of the various embodiments improve the performance of OTT services provided to the UE 1506 using the OTT connection 1550, in which the wireless connection 1570 forms the last segment.

[0149] In an example scenario, factory status information may be collected and analyzed by the host 1502. As another example, the host 1502 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 1502 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 1502 may store surveillance video uploaded by a UE. As another example, the host 1502 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 1502 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and / or transmitting data.

[0150] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 1550 between the host 1502 and UE 1506, in response to variations in the measurement results. The measurement procedure and / or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 1502 and / or UE 1506. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 1550 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 1550 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 1504. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 1502. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 1550 while monitoring propagation times, errors, etc.

[0151] Although the computing devices described herein (e.g., UEs, network nodes, hosts) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to beunderstood that these computing devices may comprise any suitable combination of hardware and / or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

[0152] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer-readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.

Claims

Claims:

1. A method, comprising: collecting (902) performance-related information relating to performance of a first network node; collecting (904) data traffic-related information relating to transmission of data by a user equipment, UE, to the first network node according to a configured grant, CG; generating (906) a prediction, based on the performance-related information and the data traffic-related information, whether a property of the CG would be improved by either performing a handover of one or more UEs to a second network node or, alternatively, by changing a parameter of the CG; and based on the prediction, instructing (908) the network node to perform a handover of the one or more UEs to the second network node or to change the parameter of the CG.

2. The method of Claim 1, wherein the parameter of the CG comprises a periodicity, P, of the CG or a number of repetitions, K, per period of the CG.

3. The method of any previous Claim, wherein the performance-related information comprises one or more performance metrics collected by the first network node.

4. The method of any previous Claim, wherein the data traffic-related information comprises one or more of latency and / or throughput of the CG connection.

5. The method of any previous Claim, wherein the property of the CG comprises a robustness and / or a latency of data transmitted using the CG.

6. The method of any previous Claim, wherein generating the prediction is performed in response to a trigger event at the network node.

7. The method of Claim 6, wherein the trigger is based on expiration of a time period.

8. The method of Claim 6, wherein the trigger event is based occurrence of an event, wherein the event comprises one or more of: a packet drop rate, a number of dropped calls, a comparison of throughput to a throughput threshold, a number of active UEs connected to thenetwork node, and / or a resource utilization at the network node exceeding a resource utilization threshold.

9. The method of any previous Claim, wherein the prediction is generated according to a policy that takes into account one or more of: a state of the UE, a priority of the UE, a level of mobility of the UE between network cells, a movement speed of the UE and a location of the UE.

10. The method of any previous Claim, wherein the prediction is generated according to a policy that is generated using reinforcement learning.

11. The method of Claim 10, wherein the policy is implemented by a neural network.

12. The method of Claim 10, wherein the performance-related information comprises one or more key performance indicators, KPIs, associated with the first network node and the data traffic-related information comprises one or more KPIs associated with the user equipment, the method further comprising training the policy according to a reward function that is a function of one or more of the KPIs associated with the first network node and / or the KPIs associated with the user equipment.

13. The method of Claim 12, wherein the key performance indicators comprise at least one traffic indicator, and at least one latency indicator.

14. The method of Claim 13, wherein the reward function has the form:where:avgiatency is an average latency of transmissions using the CG and appiatency is a maximum tolerable latency for transmissions using the CG.

15. The method of Claim 12, wherein the key performance indicators comprise at leastone energy usage indicator associated with the UE or the first network node.

16. The method of Claim 12, wherein the policy is trained using multi-agent reinforcement learning.

17. The method of Claim 16, wherein the training is performed using collaborative multi-agent deep reinforcement learning in which a plurality of agents try to optimize against a global reward.

18. The method of Claim 16 or 17, wherein the policy is trained by optimization against a global reward.

19. The method of Claim 17, wherein each agent of the plurality of agents is associated with a different radio base station.

20. An uplink configured grant configuration device adapted to perform the operations of any of Claims 1 to 19.

21. An uplink configured grant configuration device (300)comprising: a processing circuitry (312); and a memory (314) coupled to the processing circuitry, wherein the memory comprises computer-readable instructions that, when executed by the processing circuitry, cause the uplink configured grant configuration device to perform the operations of any of Claims 1 to 19.

22. A computer program product comprising a non-transitory medium that stores computer-readable instructions that, when executed by processing circuitry, perform the operations of any of Claims 1 to 19.

23. A system, comprising: a user equipment (100); and an uplink configured grant configuration device, UCC (300); wherein the UCC collects (902) performance-related information relating toperformance of a first network node and collects (904) data traffic-related information relating to transmission of data by the UE, to the first network node according to a configured grant, CG; wherein the UCC generates (906) a prediction, based on the performance-related information and the data traffic-related information, whether a property of the CG would be improved by performing a handover of the UE to a second network node; and based on the prediction, instructs (908) the network node to perform the handover of the UE to the second network node; wherein the UE receives a handover command to perform the handover to the second network node, and performs the handover in response to the handover command.

Images