Technique for autonomously operating a wireless telecommunications infrastructure

Abstract

A technique for autonomously operating a wireless telecommunications infrastructure according to an intent is described. As to a method aspect of the technique, the method performs at least two child processes and a parent process. Each of the child process is associated to one of the at least two KPIs of the wireless telecommunication infrastructure and predicts a reward value for each of the at least one action based on a rate at which the associated KPI fulfills the intent, sampled over a random distribution of other actions taken by the other child processes. The parent process determines for each of the at least one action of each of the at least two child processes, an uncertainty value (U), and selects a subset of the actions based on the reward values predicted by the child processes and their respective U determined by the parent process.

Description

[0001] Technique for autonomously operating a wireless telecommunications infrastructure

[0002] Technical Field

[0003] The present disclosure relates to a technique for autonomously operating a wireless telecommunications infrastructure. More specifically, and without limitation, methods and devices are provided for autonomously operating a wireless telecommunications infrastructure according to an intent.

[0004] Background

[0005] Wireless telecommunications infrastructures such as wireless networks have to maintain an increasing number of diverse quality requirements in a very dynamic environment. The operation and management of these requirements are beyond human capacity. Therefore, wireless telecommunications infrastructures are controlled autonomously according to intents.

[0006] There are several techniques for intent-based management of wireless networks, but each has its own drawbacks, such as high complexity due to multiple intents, low capacity and slow computational resources due to complexity, bias towards one or more intents, inability to adapt to the dynamic environment of the network, and so on.

[0007] Summary

[0008] Accordingly, there is a need for a practical and reliable technique than can autonomously operate dynamic telecommunication networks by taking an increasing number of quality requirements into account.

[0009] As to a method aspect, a method of autonomously operating a wireless telecommunications infrastructure according to an intent is provided. The method comprises performing at least two child processes, each associated to one of at least two key performance indicators (KPI) of the wireless telecommunications infrastructure. Each of the child processes predicts a reward value for each of at least one action based on a rate at which the associated KPI fulfills the intent, sampled over a random distribution of other actions taken by the child processes other than the respective child process. The method further comprises performing a parent process. The parent process determines, for each of the at least one action of each of the at least two child processes, an uncertainty value (U) based on a number of prior uses of the respective action. The parent process further selects a subset of the actions based on the reward values predicted by the child processes and their respective uncertainty values (U) determined by the parent process. The method further comprises performing the selected subset of the actions.

[0010] Embodiments of the technique autonomously operate a wireless telecommunications infrastructure according to the (at least one) intent by performing multiple child processes, each associated with a specific key performance indicator (KPI) of the infrastructure. The child processes predict a reward value for each action based on how well the associated KPI fulfills the intent, taking into account actions taken by other child processes. The child process may have knowledge of the impact of its actions with some uncertainty. A parent process determines an uncertainty value for each action based on the number of prior uses of that action. The parent process selects the subset out of the actions for which the child processes predicted reward values. The subset of actions is selected based on maximizing the reward values predicted by the child processes and taking their respective uncertainty values into account. Finally, the selected subset of actions is performed or initiated by the parent process.

[0011] Same or further embodiments of the technique autonomously operate a wireless telecommunications infrastructure guided by an intent. A minimum of two child processes is executed, each pertinent to a KPI of the infrastructure. Each child process anticipates a reward value for each of at least one action. This anticipation is based on the success rate of the KPI that it is associated with, in relation to meeting the intent in the context of the random distribution of various other actions carried out by child processes other than the considered child process. The parent process determines an uncertainty value for the at least one action from each child process. This is based on the frequency of prior uses of said action. The parent further selects a subset of actions based on the anticipated reward values dispensed by the child processes and their respective determined uncertainty values. The parent process performs or initiates the selected subset of actions. The reward value may be understood as a negative, zero or a positive number, corresponding to a decrease, neutral or an improvement in the KPI, respectively. In same or further embodiments, the child processes may not be aware of the assessed (e.g., proposed) other actions of the other child processes. The child processes only observe the outcomes of these actions in relation to their own, and consider the other actions of the other child processes as a random noise.

[0012] Same or further embodiments of the technique can perform the child processes independently and / or in parallel (e.g., simultaneously without mutual dependencies) by virtue of the definition of the reward value as the rate of success of the respective action under the random distribution of the actions (i.e., the outcomes) of the other child processes. Hence, the technique can be scalable by parallelization of the multiple child processes with the reward value as a well- defined target within each of the child processes.

[0013] Embodiments of the method can control the operation of the wireless telecommunications infrastructure for as many services as requested given the (e.g., compute and / or network and / or radio) resources of the wireless telecommunications infrastructure, while avoiding conflicts. The available resources may change over time (e.g., according to an incident).

[0014] In at least some embodiments of the technique, each child process implements a unique KPI and the parent process, after inferring the uncertainty, selects the one or more actions which provide the best reward values, thereby maximizing the KPIs' fulfilment of the intent. The selected subset of actions, after analysis and selection, are then performed (e.g. initiated or distributed). This reward value prediction may include a machine learning (ML) model (also referred to as artificial intelligence model). Furthermore, this may be a process of reinforcement learning for autonomous network operation. For example, the method may be iterative, i.e. may be repeated (e.g. periodically or in response to changing conditions in the wireless telecommunications infrastructure). The iteration of the method may be used for iterative learning of the predicting of the reward values at the child processes and / or the determining of the uncertainty value at the parent process. The child processes may continue to predict reward values based on the evolving performance of the KPIs, and / or the parent process may adjust the subset of actions based on the updated predictions and uncertainty values. By utilizing this form of reinforcement learning, the autonomous operation of the wireless telecommunications infrastructure can optimize its performance according to the specified intent and adapt to changing conditions. Alternatively or in addition, the learning and exploring may be part of operating the wireless telecommunications infrastructure, which can avoid a simulation of a complex wireless telecommunications infrastructure and enable more realistic predictions.

[0015] Herein, when specifying each of the at least two child processes, the "respective child process" may refer to the one of the at least two child processes that is respectively considered under the meaning of "each". When specifying each of the at least one action of each of the at least two child processes, the "respective action" may refer to the one of the at least one action of the respective child process, which action is respectively considered under the meaning of "each".

[0016] Herein, when specifying one or each of the child processes, "respective KPI" or "associated KPI" or "respectively associated KPI" may refer to the one of at least two KPIs that is associated to the respective child process among the at least two child processes.

[0017] Herein, actions of the child processes (e.g., taken or assessed by the child processes) may collectively refer to the at least one action of each of the at least two child processes. Accordingly, when specifying one or each of the child processes, other actions taken by the child processes other than the respective child process may collectively refer to the at least one action taken by each of the at least two child processes other than the respective child process. The at least two child processes other than the respective child process may be briefly referred to the other child processes (e.g., although there may be only one other child process in case of two child processes).

[0018] Predicting the reward value for each of at least one action based on a rate at which the associated KPI fulfills the intent may mean predicting the reward value for each of the at least one action based on a rate at which the associated KPI is predicted to fulfill the intent as a result of the respective action.

[0019] The intent may specify one or more criteria (e.g. requirements) for the at least two KPIs. The associated KPI may fulfil the intent if the associated KPI meets the one or more criteria.

[0020] The intent may specify a joint criterion for the at least two KPIs. For example, the utility function may be a function of the at least two KPIs and the joint criterion may require that the utility function equals or exceeds a target utility value. The wireless telecommunications infrastructure may be or may comprise a radio access network (RAN). Performing the selected subset of actions may mean that the parent process implements (e.g., configures) these actions within the wireless telecommunications infrastructure (e.g., in the RAN). The RAN may be a fourth generation (e.g., 4G LTE) RAN or a fifth generation (e.g., 5G NR) RAN.

[0021] Performing the selected one or more actions encompass all the options such as the initiating or distributing the one or more actions.

[0022] The at least two KPIs (e.g., according to the method aspect) may comprise at least one of latency, delay bound, jitter, reliability, data rate, Quality of Service, QoS, and Quality of Experience, QoE.

[0023] Each of the KPIs may be indicative of the performance of a technical aspect of the wireless telecommunications infrastructure, e.g. at least one of latency, delay bound, jitter, reliability, data rate, Quality of Service (QoS), and Quality of Experience (QoE).

[0024] The at least one action of each of the at least two child processes (e.g., according to the method aspect) may change at least one operating parameter of the wireless telecommunications infrastructure, which influences the KPI associated to the respective child process. Alternatively or in addition, the at least one action of each of the at least two child processes comprises at least one of: changing an allocation of radio resources; changing a frequency reuse; changing a modulation scheme; changing a coding scheme; changing a maximum bit rate (MBR) of a user equipment (UE); and changing a transmit power level.

[0025] Each of the least one action may be an action of operating the wireless telecommunications infrastructure. Alternatively or in addition, each of the at least one action may influence the KPI. For example, each of the actions may comprise or trigger changing one or more operating parameters of the wireless telecommunications infrastructure.

[0026] For example, the at least one action may comprise (e.g., as the one or more operating parameters) at least one of allocating radio resources (e.g., among cells), configuring a frequency reuse (e.g., between neighboring cells), configuring a modulation scheme, configuring a coding scheme, changing a maximum bit rate (MBR) of a user equipment (UE), changing a transmit power level, and changing spectral efficiency. The radio resources may comprise at least one of frequencies resources (e.g., resource blocks), time resources (e.g., slots), and spatial resources (e.g., beams). In a variant of any embodiment, the action may comprise changing priorities associated to the radio resources or the operating parameters.

[0027] The changing of the MBR of a UE may comprise changing the MBR of one or each UE using a service associated to the respective child process (e.g., using a service associated to the KPI associated to the respective child process).

[0028] Herein, for a child process associated to a KPI, the KPI may also be referred to as being associated to the child process.

[0029] The rate at which the associated KPI fulfils the intent (e.g., according to the method aspect) may be a conditional probability given the random distribution of the other actions taken by the other child processes.

[0030] The rate at which the associated KPI fulfils the intent may be a (e.g. conditional) probability (or an estimate value for the probability) given the random distribution of the other actions taken by the other child processes (e.g., based on a finite number of samples over the random distribution of actions). The random distribution of actions may be an equal or uniform probability distribution of the actions. Alternatively or in addition, the random distribution may correspond to independent and identically distributed actions.

[0031] Herein, the other actions being taken by the other child processes may refer to the other actions assessed by the other child processes by predicting a reward value for the other actions. "Being taken" may or may not require that the action is actually performed in the wireless telecommunications infrastructure as this decision is reserved to the parent process.

[0032] One or each of the at least two child processes (e.g., according to the method aspect) may predicts the reward value for each of the at least one action based on the rate of the associated KPI fulfilling the intent given a configuration of the wireless telecommunications infrastructure. Alternatively or in addition, one or each of the at least two child processes may predict the reward value for each of the at least one action based on measuring the rate at which the associated KPI fulfills the intent for multiple UEs. The results of measuring whether the associated KPI fulfils the intent may be based on observing different UEs. Such results may be referred to as meta-data (e.g., in the sense that these data are abstracted from the behavior of induvial UEs and / or relate to the state of the wireless telecommunications infrastructure). The meta-data may be used for training a neural network system of the respective child process (e.g., a network system of a child agent implementing the child process).

[0033] The child processes may utilize a form of reinforcement learning to predict the reward values for various actions based on the degree of achievement of specified KPIs and / or by simulating the wireless telecommunications infrastructure. Alternatively or in addition, the relationship between the KPI fulfilment, the prediction of the reward value, and the random distribution of the other actions may be as follows. The respective child process may assess (e.g., by means of simulation of the wireless telecommunications infrastructure) if the associated KPI fulfils the intent as a result of the at least one action of the child process given a random sample of the other actions (i.e., actions of other child processes). The respective child process may determine the rate (e.g., the reward value) as the ratio of how many times the associated KPI fulfils the intent divided by the number of assessed random samples.

[0034] The wireless telecommunications infrastructure configuration may comprise priority of each KPI or actions, and / or a number of user equipment expecting a service (e.g., intent), and / or an extremum (e.g., maximum or minimum) of a KPI to be fulfilled, and etc. Alternatively or in addition, the wireless telecommunications infrastructure configuration may change over time.

[0035] The parent process (e.g., according to the method aspect) may determine the uncertainty value (U) based on a number of times the respective action has been performed in a given timestep and / or the given timesteps.

[0036] Each timestep may be a time interval. The timesteps may be a (optionally periodic) partitioning of time in consecutive time intervals.

[0037] The parent process may determine the uncertainty as the inverse of the number of prior uses of the respective action (e.g., how many times the respective action has been previously performed), optionally limited to cases in which the action was successful in fulfilling the associated KPI. Alternatively or in addition, the parent process may determine the uncertainty as the square root of the inverse of the number of prior uses of the respective action in a given timestep and the square root of the logarithm of the given timestep (e.g. the logarithm of time), e.g.,

[0038] The parent process (e.g., according to the method aspect) may select the subset of actions based on maximizing the determined uncertainty value (U) and / or maximizing the reward values predicted by the child processes, and / or maximizing both the reward values predicted by the child processes and the determined uncertainty value (U).

[0039] The parent process (e.g., evaluation agent) may manage the uncertainty (e.g., based on the uncertainty value) in the reward value and the at least one action. Based on an expected utility (e.g., based on predicted KPIs resulting from the subset of the actions) and / or the determined uncertainty values (as amount of uncertainty) and / or the predicted reward values by the child processes, the parent process may select (e.g., propose) the subset of the actions (e.g., one or more actions) that is to be applied to the wireless telecommunications infrastructure (e.g., a network). Optionally, the parent process may select the subset of the actions by a combination of maximizing the reward values and minimizing the uncertainty values.

[0040] The subset of the actions may be smaller than (i.e., may be a proper subset of) the set of all actions for which the at least two child processes predicted reward values. The subset of actions may comprise zero action and / or one action and / or multiple actions, optionally one action per child process.

[0041] Alternatively or in addition, the parent process may select an action that has never been tried before or has been performed less frequently. In this way, the parent process and / or the child processes may learn more about possible actions (e.g., explore the possible actions by selecting an action based on the maximum uncertainty value), and optionally may update the uncertainty value and / or the predicting of the reward value for said action. Alternatively or in addition, the parent process may exploit the possible actions by selecting an action based on the maximum predicted reward values. Alternatively or in addition, the parent process may exploit and explore by selecting an action based on the maximum of both of the predicted reward value and the determined uncertainty value (e.g., by maximizing a weighted linear combination of the predicted reward value and the determined uncertainty value).

[0042] Herein, any child process may also be referred to as child agent, e.g. if the child process includes a neural network system and / or if machine learning is applied to the prediction of the reward values.

[0043] Furthermore, any child process may be referred to as a proposal agent, e.g., as a KPI proposal agent if the child process is responsible for a single KPI or as an action proposal agent, since the child process more or less recommends the at least one action by means of the reward value.

[0044] The intent or each of the intent and one or more further intents (e.g., according to the method aspect) may further correspond to a service provided by the wireless telecommunications infrastructure. Alternatively or in addition, the one or more services provided by the wireless telecommunications infrastructure may comprise at least one of: ultra-reliable and low-latency communication (URLLC); enhanced mobile broadband (eMBB) service; web real-time communication (WebRTC); massive machine-type communication (mMTC); and massive internet of things (mloT) service; and industrial internet of things (I loT) service.

[0045] The intent or each of the intents (e.g., the intent and one or more further intents) of the method may correspond to (e.g., may define) a service provided by the wireless telecommunications infrastructure. The method may comprise multiple intents (e.g., for multiple services, respectively). In other words, the method may be a method of autonomously operating a wireless telecommunications infrastructure according to at least one intent or at least two intents. For example, the at least two intends may specify the at least two KPIs, respectively.

[0046] The intent (e.g. a service) may comprise at least one of: ultra-reliable and low- latency communication (URLLC), enhanced mobile broadband (eMBB), web realtime communication (WebRTC), massive machine-type communication (mMTC), massive internet of things (mloT) services, and industrial internet of things (lloT).

[0047] The autonomously operated wireless telecommunications infrastructure may manage multiple intents according to different requirements (e.g., in terms of the KPIs) and / or different regions (e.g., different cells or different tracking areas of the RAN). The autonomously operated wireless telecommunications infrastructure may be a cognitive network. Alternatively or in addition, the autonomous operation of the wireless telecommunications infrastructure may be a distributed intent-based (e.g., service-based) management.

[0048] The method for the autonomous operation may comprise at least one closed loop (CL), e.g. for each intent. Alternatively or in addition, each CL may control the wireless telecommunications infrastructure according to its associated at least one intent according to the method to fulfill the associated different requirements (e.g., in terms of the at least two KPIs).

[0049] The at least one intent (e.g., the at least one service) provided or to be provided by the wireless telecommunications infrastructure may dynamically change over time (e.g., priorities of the services may change). Therefore, the autonomously operated wireless telecommunications infrastructure may be responsive to changing the at least one intent, e.g. in terms of the associated requirements (e.g., criteria in terms of the KPIs).

[0050] The autonomously operated wireless telecommunications infrastructure (e.g., a cognitive network) may reduce or avoid human intervention. Embodiments of the method can create a dynamic autonomous systems that can self-monitor, selfevaluate and / or self-heal by selecting the actions to fulfill all specified requirements (e.g., in terms of the KPIs) according to the at least one intent.

[0051] The intent-driven CLs may be used at different places for network management (at vertical layers and / or different domains) to create an end-to-end (E2E) automation framework that is flexible to deal with dynamic requirements (e.g., in terms of the KPIs) and provide operational efficiency with highly autonomous network management.

[0052] In an autonomous approach, the requirement (e.g., in terms of the KPIs) comes as a high-level intent, which is a declarative information object that defines the requirements (e.g., KPIs) of a service that the network is expected to meet. Each requirement may be understood as a key performance indicator (KPI).

[0053] Each intent (e.g., service) may have different requirements in terms or the KPIs. For example, each intent (e.g., service) may relate to different KPIs. For example, an ultra-reliable and low-latency communication (URLLC) service may for example require its latency to be less than 1 ms (e.g., involving the latency as one of the KPIs) and packet loss to be less than 0.1 % (e.g., involving packet loss as another one of the KPIs). Different intents may therefore have different KPIs in terms of number and type.

[0054] Providing a service (e.g., meeting an intent) may be understood as successfully meeting all the KPIs of the intent (e.g., service). Alternatively or in addition, providing a service may be understood as meet all the KPIs of the intent for 95% of the incidents or more. In other words, meeting the KPI may be understood as being above or below a threshold and / or above or below a threshold with a 98% Cl, optionally 95% or 90%.

[0055] One or each of the at least two child processes (e.g., according to the method aspect) may predict the reward value for each of at least two actions.

[0056] One or each of the at least two child processes may predict the reward value for each of at least two actions. The actions of the respective child process may also be referred to as arms of the child process. The autonomously operating of the wireless telecommunications infrastructure according to an embodiment of the method may predict the reward value for each of multiple actions per child process, e.g. utilizing a multi-armed bandit (MAB) algorithm for each child process, to ensure that the operating parameters of the wireless telecommunications infrastructure (e.g., priorities for RAN resources) of different services (such as WebRTC and URLLC) are in alignment and may do not cause conflicts.

[0057] The parent process may apply some specific methods (e.g., algorithms) to determine the uncertainty value of each action. Alternatively or in addition, the parent process may use an Upper Confidence Bound (UCB) algorithm.

[0058] A combination of the MAB algorithm and the UCB algorithm may measure how good an action is and how much uncertainty it may contain. The parent process may select an action that maximize the predicted (e.g., expected) reward and minimizes the uncertainty value of the action. The child process may apply MAB to learn the predicting of the expected reward values of their actions. The parent process may apply UCB to determine the uncertainty value of each or at least one of the actions.

[0059] Each of the at least two child processes (e.g., according to the method aspect) may be uniquely associated to a different one of the at least two KPIs. Alternatively or in addition, each of the at least two child processes may be exclusively associated to a single one of the at least two KPIs.

[0060] Each of the at least two child processes may be (e.g., uniquely) associated to a different one of the at least two KPIs (i.e., a unique KPI). Alternatively or in addition, each of the at least two KPIs may be associated to one the at least two child processes. Embodiments of the method may increase the scalability by requiring only one KPI per child process.

[0061] Unlike a prior art proposal agent, which has to solve multiple problems in terms of multiple KPIs with different requirements and therefore solves a complex entity since it has to optimize multiple objectives simultaneously, each of the at least two child processes may be (e.g., exclusively) associated with (e.g., deal with) a single one (i.e., only one) KPI (e.g., at a time). For example, each of the child processes may only need to deal with a single objective. In other words, each child process may deal with a single KPI and / or may not accept multiple KPI issues. Hence, each of the child processes may be more efficient in predicting the reward value and / or learning to predict the reward value (for each of the at least one action). Alternatively or in addition, the selecting of the subset of actions may be more efficient at the patent process, since the predicted reward values are uniquely associated to one of the KPIs, which can prevent conflicts and / or allow for weighting the predicted reward values and / or enable resolving KPI conflicts.

[0062] The method may utilize a hierarchical algorithm, where each child process (e.g., KPI proposal agent) may correspond to a specific service, such as WebRTC or URLLC.

[0063] Each child process (e.g., KPI proposal agent) may have a set of arms representing a set of different actions (being the at least one action of the respective child process). For example, the different action may give different priorities to resources of the wireless telecommunications infrastructure, e.g. to provide alternative measures to fulfill the intent (e.g. a service) or to fulfill different intents (e.g., different services).

[0064] Alternatively or in addition, each child process may have a set of arms. Each arm corresponds to an action. Each arm may predict the reward value of an action at a given time. Alternatively or in addition, each arm of the child process may predict the reward value of an action while assuming at least one random action taken by the one or more other child processes other than the respective child process. Alternatively or in addition, each child process may perform the prediction for each arm multiple times, as the random actions of other child processes may be averaged out. Each child process then may determine the expected reward (e.g., predict a reward value) of its individual actions at that point in time.

[0065] Examples of possible actions that the wireless telecommunications infrastructure may take comprise changing maximum bit rate (MBR) of a user equipment (UE) registered with the service, and / or changing the priority level of that service and / or moving the service from the cloud to the edge. An action that is selected to improve a KPI of one service may degrade the KPI of another service (or its service). In other words, actions may conflict with another action. For instance, the wireless telecommunications infrastructure may increase the priority of the WebRTC service to improve its QoE, but this may negatively impact the packet loss of the URLLC service. This situation may lead to a conflict, which is resolved at the parent process (e.g., evaluation agent) after receiving all the predicted KPIs for each proposed action. Therefore, a good level of prediction is crucial for intentbased automation (IBA) to operate efficiently.

[0066] The intent (e.g., according to the method aspect) may comprise a target KPI (KPI_t) for each of the at least two KPIs of the wireless telecommunications infrastructure. The associated KPI may fulfill the intent if the associated KPI meets or exceeds the KPI_t of the intent.

[0067] The intent may comprise (i.e., may be indicative of, e.g., imply or expressly specify) a target KPI (KPI_t) for each of the at least two KPIs of the wireless telecommunications infrastructure. For example, the intent may specify a utility function of the at least two KPIs. The KPI_t may be a threshold value or an inflection point of the utility function. Alternatively or in addition, the method autonomously operates the wireless telecommunications infrastructure according to at least two intents. Each of the at least two intents may comprise a target KPI for a different one of the at least two KPIs.

[0068] Each KPI may have a corresponding a child process. Each child process may be configured to assess (e.g. by predicting the reward value) of predefined multiple actions (e.g., implemented as arms) that may be taken to resolve its KPI issue. The rewards value for each action (e.g., arm) may be predicted (e.g., determined) by how well the service targets (i.e., KPI_t), such as QoE for WebRTC and delay bound for URLLC, are met. There may be a reward value for every action assessed by the child process. In other words, each child process may have a set of arms (i.e. actions), and each arm may output a reward value that depends on the given targets (e.g., KPI_t) for each KPI of the intent.

[0069] The child processes reduce the complexity of the problem to be solved and increase the scalability, for example by predicting the reward values of the actions for one KPI at a time and averaging the other actions of the other child processes.

[0070] There may be multiple child processes performed in the method for autonomously operating the wireless telecommunications infrastructure, optionally as many as the number of KPIs. The responsibility of each child process (e.g., predict agent) may be to help their service get enough resources of the wireless telecommunications infrastructure (e.g., RAN resources) to fulfill its requirements (e.g., target KPIs). To do this, each child process may have a set of arms (i.e., actions), and each arm may have a reward that depends on the given target values (KPI_t) for each service KPI.

[0071] The parent process may or may not have prior knowledge of the impact of any of the actions on the KPIs associated to the child processes (e.g., which KPIs are to fulfil the target KPIs, KPI_t). If the parent process has prior knowledge of the impact of any of the actions (which it may select or even perform or initiate) on the at least two KPIs (e.g. whether or not the target KPI is fulfilled), the level of uncertainty about the impact of the action is less than if the parent process has no (or less) knowledge of the impact of its actions on the at least two KPIs. Alternatively or in addition, the more often an action is taken (i.e., performed or initiated), the less the uncertainties remains in the decision whether or not to select the action.

[0072] The method (e.g., according to the method aspect) may further comprise the step of providing data for at least two child processes. The provided data may be at least one of a historical data of the wireless telecommunications infrastructure; resources of the wireless telecommunications infrastructure; configurations of the wireless telecommunications infrastructure; and a simulated data of the wireless telecommunications infrastructure.

[0073] The child process may be provided with prior information (e.g., data) about the wireless telecommunications infrastructure (e.g., for training a neural network system or child agent of the respective child process). The prior information may be indicative of capabilities and / or capacities and / or network resources of the wireless telecommunications infrastructure. Alternatively or in addition, the prior information may be historical information of the wireless telecommunications infrastructure in use. Alternatively or in addition, the information may be simulated data of the wireless telecommunications infrastructure in use.

[0074] Before the prediction is performed by the child processes, the information provided may be updated, optionally before each prediction.

[0075] The predicted reward value (e.g., according to the method aspect) may comprise at least one reward value for each of the at least one action at a certain time.

[0076] Each child process may predict the reward value multiple times (e.g., at each of the timesteps) for each arm (e.g., for each action assessed by the respective child process), optionally as the random actions of other child processes may be averaged. Then, each child process may determine the expected reward value of its individual actions at that time instant (e.g., at the timestep). Let Rkltbe the predicted (e.g., expected) reward values for the 7-th child process with k = 1 to K arms at a time t (e.g., a discretization of time and / or an index of the timestep). Each child process may output a vector comprising all the predicted (e.g., expected) reward values of each arm (e.g., action) as its elements. For example, let Rtl— ] be the vector of reward value predicted (e.g., expected) by the 7-th child process. This vector may correspond to a number of K arms with their predicted (e.g., expected) reward values at time t.

[0077] The performing of the at least two child process (e.g., according to the method aspect) may predict the reward values periodically and / or upon a triggering event.

[0078] Each child process may update the reward vector periodically or at a time when the network (e.g., wireless telecommunications infrastructure) configuration changes. For example, any change in the network configuration may trigger the child processes to perform the prediction.

[0079] The predicted reward value (e.g., according to the method aspect) at a certain time may be saved. The saved predicted reward value at the certain time may be the provided data for at least two child processes. Alternatively or in addition, the saved predicted reward values may be used as historical information, e.g. prior to the subsequent prediction performed by the child process, optionally for training purpose. Alternatively or in addition, the saved predicted reward values may be used by the parent process, e.g., for training and / or as a backup information (e.g., in case the respective child process fails or stops predicting at a later time).

[0080] The method (e.g., according to the method aspect) may further comprise the step of combining actions of the child processes and the corresponding reward values.

[0081] The parent process may receive the predicted reward values of the actions from each child process. The parent process may receive the reward values (e.g., as a vector) from each child process and determine a combination set from the different actions coming from the child processes. The parent process may combine the information from the different child processes to determine which subset of the actions is best (e.g., to fulfill an intent that is currently observed to be violated or any other given issues).

[0082] The parent process may combine the reward vectors of the child processes in a way that contains all the possible combinations of the elements of the vectors. For example if there is a child process i=l with a vector of 5 elements and a child process i=2 with a vector of 7 elements, the parent process may combine the reward values in a matrix of 5x7 elements. Alternatively or in addition, the parent process may combine the rewards according to a function. Alternatively or in addition, the parent process may combine the reward values by assigning a weight to each element and / or summing the reward values (e.g., the reward values corresponding to the same action assess by different child processes). Alternatively or in addition, the weight may be determined based on the network resources and / or the priority of the services to be delivered.

[0083] The method (e.g., according to the method aspect) may further comprise the step of updating at least one of the predicted reward values for each of the at least one action; the uncertainty value of the at least one action; and a number of actions per child processes.

[0084] After the selected subset of the actions has been performed, the parent process (e.g., or a monitoring agent) may collect actual measurements (e.g. representing an impact of the performed subset of the action) and may send them to one or more of the child processes (e.g., the proposal agent), e.g. to the one child process that had assessed the selected action. The child process may repeat the prediction step (e.g., based on the data received from the parent process) until a final stable action is performed.

[0085] The parent process (e.g., proposal agent) may eliminate some of the actions that have small impact on the KPI_t and / or cannot improve the KPI with a good certainty. Alternatively or in addition, each child process may learn the reward values of its actions, and optionally may eliminate one or more actions with very low predicted reward value (e.g., eliminated from a list of actions assessed by the respective child process).

[0086] Since there may be many possible actions that the child processes may propose (e.g., assess by predicting reward values), the child process may eliminate one or more of the actions that cannot improve the KPI with a good degree of certainty or prospect. For example the child may eliminate the actions for which their predicted reward values are lower than a predefined threshold value, optionally close to zero. In other words, the child process may be able to filter out neutral or ineffective actions (e.g., in the step of predicting reward values for or before the parent process).

[0087] The parent process may be able to balance a strike between a current (e.g., expected) utility and a possible improved utility, optionally by exploring the less tried and / or untried actions. The possible improved utility may come from performing less-known or least-known actions (i.e., actions with high or highest uncertainty). Embodiments of the method can improve the capability of the parent process in the selecting of the subset of the actions. Thus, an intent-based management framework performing an embodiment of the method may act more intelligently and robust.

[0088] The parent process (e.g., according to the method aspect) may further determine a priority of the intent and / or a priority of the target KPIs (KPI_t).

[0089] Alternatively or in addition, the determined priority of the intent and / or the priority of the target KPI may be used by the child process as the provided information prior to performing a prediction. The parent process (e.g., according to the method aspect) may select the subset of the actions based on a priority of the intent and / or a priority of the target KPIs (KPI_t). Alternatively or in addition, the parent process (e.g., according to the method aspect) may select the subset of actions based on resources of the wireless telecommunications infrastructure.

[0090] The parent process may have prior knowledge of the network resource. Alternatively or in addition, the parent process may select the subset of the actions based on the network resource and / or the given network configuration. Alternatively or in addition, the parent process may select a subset of actions which comprises an action of a change in network configuration based on the network resource.

[0091] The method (e.g., according to the method aspect) may be performed according to at least two intents.

[0092] The at least one of the child processes (e.g., according to the method aspect) and the parent process may be performed according to at least one sort of artificial intelligence (Al). Any one of the steps of the method may be performed by an artificial intelligence (Al) agent (briefly: agent). The Al may be understood as (or the Al agent may be understood to implement) a machine learning algorithm, e.g. MAB, reinforcement learning, federated bandit, Thompson sampling, MAB plus upper confidence bound, etc. The Al agent (e.g., the child agents and / or parent agent) may comprise a neural network system.

[0093] As to a device aspect a computer program product is provided. The computer program product comprises program code portions for performing any one of the steps of the method aspect when the computer program product is executed on one or more computing devices, optionally stored on a computer-readable recording medium.

[0094] As to a device aspect, a node of a wireless telecommunications infrastructure is provided. The node comprises memory operable to store instructions and processing circuitry operable to execute the instructions, such that the node is operable to perform at least two child processes, each associated to one of at least two key performance indicators (KPI) of the wireless telecommunications infrastructure. Each of the child processes predicts a reward value for each of at least one action based on a rate at which the associated KPI fulfills the intent, sampled over a random distribution of other actions taken by the child processes other than the respective child process. The node is further operable to perform a parent process. The parent process determines, for each of the at least one action of each of the at least two child processes, an uncertainty value (U) based on a number of prior uses of the respective action. The parent process further selects a subset of the actions based on the reward values predicted by the child processes and their respective uncertainty values (U) determined by the parent process. The node is further operable to perform the selected subset of the actions.

[0095] The node (e.g., according to the device aspect) may be further operable to perform any one of the steps of the method aspect.

[0096] As to another device aspect, a node supporting a radio access network (RAN) in a wireless telecommunications infrastructure is provided. The node is configured to perform at least two child processes, each associated to one of at least two key performance indicators (KPI) of the wireless telecommunications infrastructure. Each of the child processes predicts a reward value for each of at least one action based on a rate at which the associated KPI fulfills the intent, sampled over a random distribution of other actions taken by the child processes other than the respective child process. The node is further configured to perform a parent process. The parent process determines, for each of the at least one action of each of the at least two child processes, an uncertainty value (U) based on a number of prior uses of the respective action. The parent process further selects a subset of the actions based on the reward values predicted by the child processes and their respective uncertainty values (U) determined by the parent process. The node is further configured to perform the selected subset of the actions.

[0097] The node (e.g., according to the other device aspect) may be further configured to perform any one of the steps of method aspect.

[0098] The node (e.g., according to the device aspect or the other device aspect) may be a core node of a core network (CN) of the wireless telecommunications infrastructure. Alternatively or in addition, the node may be a network node of a radio access network (RAN) of the wireless telecommunications infrastructure, optionally a base station or a central unit of the base station.

[0099] As to a system aspect, a communication system is provided. The communication system includes a host computer comprising processing circuitry configured to provide user data. The host computer further comprises a communication interface configured to forward user data to a (e.g., cellular or ad hoc) radio network of a wireless telecommunications infrastructure for transmission to a user equipment (UE). The wireless telecommunications infrastructure comprises a radio interface and processing circuitry, the processing circuitry of the wireless telecommunications infrastructure being configured to execute any one of the steps of the method aspect.

[0100] The communication system (e.g., according to the system aspect) may further including a user equipment (UE). The UE may comprise a radio interface and processing circuitry, the processing circuitry of the UE being configured for radio access to the wireless telecommunications infrastructure.

[0101] The communication system (e.g., according to the system aspect), wherein the wireless telecommunications infrastructure may comprise one or more nodes configured to execute any one of the steps of the method aspect.

[0102] The radio network of the communication system (e.g., according to the system aspect) may further comprise a base station, or a radio device functioning as a gateway, which may be configured to communicate with the UE and / or may be configured to execute any one of the steps of the method aspect.

[0103] In the communication system (e.g., according to the system aspect), the processing circuitry of the host computer may be configured to execute a host application, thereby providing the user data; and a processing circuitry of the UE may be configured to execute a client application associated with the host application.

[0104] Without limitation, for example in a 3GPP implementation, the user equipment (UE) may be any radio device configured for radio access to the wireless telecommunications infrastructure, e.g., according to a 3GPP specification.

[0105] The technique may be applied in the context of 3GPP New Radio (NR), e.g. as a fifth generation radio access technology of the wireless telecommunications infrastructure.

[0106] The technique may be implemented in accordance with a 3GPP specification, e.g., for 3GPP release 17 or 18. The technique may be implemented for 3GPP LTE or 3GPP NR, e.g. using proximity services according to a modification of the 3GPP document TS 23.303, version 17.1.0, or for 3GPP NR, e.g. using proximity services according to a modification of the 3GPP document TS 33.303, version 17.2.0.

[0107] The wireless telecommunications infrastructure (e.g., the RAN) may comprise one or more base stations, e.g., performing the third method aspect.

[0108] Any of the radio devices may be a 3GPP user equipment (UE) or a Wi-Fi station (STA). The radio device may be a mobile or portable station, a device for machinetype communication (MTC), a device for narrowband Internet of Things (NB-loT) or a combination thereof. Examples for the UE and the mobile station include a mobile phone, a tablet computer and a self-driving vehicle. Examples for the portable station include a laptop computer and a television set. Examples for the MTC device or the NB-loT device include robots, sensors and / or actuators, e.g., in manufacturing, automotive communication and home automation. The MTC device or the NB-loT device may be implemented in a manufacturing plant, household appliances and consumer electronics.

[0109] The wireless telecommunications infrastructure (e.g., the RAN) may be implemented by one or more base stations. The base station may encompass any station that is configured to provide radio access to any of the radio devices (UEs). The base stations may also be referred to as cell, transmission and reception point (TRP), radio access node or access point (AP). The base station and / or the relay radio device may provide a data link to a host computer providing the user data to the remote radio device or gathering user data from the remote radio device. Examples for the base stations may include a 3G base station or Node B, 4G base station or eNodeB, a 5G base station or gNodeB, a Wi-Fi AP and a network controller (e.g., according to Bluetooth, ZigBee or Z-Wave).

[0110] The RAN may be implemented according to the Global System for Mobile Communications (GSM), the Universal Mobile Telecommunications System (UMTS), 3GPP Long Term Evolution (LTE) and / or 3GPP New Radio (NR).

[0111] Brief Description of the Drawings

[0112] Further details of embodiments of the technique are described with reference to the enclosed drawings, wherein: Fig. 1 shows a schematic block diagram of a first embodiment of a device for autonomously operating a wireless telecommunication infrastructure according to at least one intent;

[0113] Fig. 2 shows a flowchart for a first embodiment of a method of autonomously operating a wireless telecommunication infrastructure according to at least one intent;

[0114] Fig. 3 shows a schematic diagram of an intent management function according to the prior art;

[0115] Fig. 4 shows a schematic block diagram of a second embodiment of the device of Fig. 1, which can include the features and steps of the first embodiment;

[0116] Fig. 5 shows a schematic block diagram of a first implementation of the child process in any embodiment of the device of Fig. 1;

[0117] Fig. 6 shows a schematic block diagram of a first implementation of the parent process in any embodiment of the device of Fig. 1;

[0118] Fig. 7 shows a schematic block diagram of a third embodiment of the device of Fig. 1, which can include the features and steps of the first or second embodiment;

[0119] Fig. 8 shows a schematic block diagram of a second implementation of the child process of any embodiment of the device of Fig. 1;

[0120] Fig. 9 shows a schematic block diagram of a third implementation of the child process of any embodiment of the device of Fig. 1;

[0121] Fig. 10 shows a schematic block diagram of a fourth embodiment of the device of Fig. 1, which can include the features and steps of the first, second or third embodiment;

[0122] Figs. 11A and 11B show an experimental result of autonomously operating a wireless telecommunication infrastructure according to an embodiment of the method of Fig. 2, in comparison with two prior art methods; Figs. 12A and 12B show an experimental result of autonomously operating a wireless telecommunication infrastructure according to an embodiment of the method of Fig. 2, in comparison with two prior art methods;

[0123] Figs. 13A and 13B show an experimental result of autonomously operating a wireless telecommunication infrastructure according to an embodiment of the method of Fig. 2, in comparison with two prior art methods;

[0124] Fig. 14 shows a schematic block diagram of a network node embodying the device of Fig. 1;

[0125] Fig. 15 shows a schematic block diagram of a core node embodying the device of Fig. 1;

[0126] Fig. 16 schematically illustrates an example telecommunication network connected via an intermediate network to a host computer;

[0127] Fig. 17 shows a generalized block diagram of a host computer communicating via a base station or radio device functioning as a gateway with a user equipment over a partially wireless connection; and

[0128] Figs. 18 and 19 show flowcharts for methods implemented in a communication system including a host computer, a base station or radio device functioning as a gateway and a user equipment.

[0129] Detailed Description

[0130] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as a specific network environment in order to provide a thorough understanding of the technique disclosed herein. It will be apparent to one skilled in the art that the technique may be practiced in other embodiments that depart from these specific details. Moreover, while the following embodiments are primarily described for a New Radio (NR) or 5G implementation, it is readily apparent that the technique described herein may also be implemented for any other radio communication technique, including a Wireless Local Area Network (WLAN) implementation according to the standard family IEEE 802.11, 3GPP LTE (e.g., LTE-Advanced or a related radio access technique such as MulteFire), for Bluetooth according to the Bluetooth Special Interest Group (SIG), particularly Bluetooth Low Energy, Bluetooth Mesh Networking and Bluetooth broadcasting, for Z-Wave according to the Z-Wave Alliance or for ZigBee based on IEEE 802.15.4.

[0131] Moreover, those skilled in the art will appreciate that the functions, steps, units and modules explained herein may be implemented using software functioning in conjunction with a programmed microprocessor, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Digital Signal Processor (DSP) or a general purpose computer, e.g., including an Advanced RISC Machine (ARM). It will also be appreciated that, while the following embodiments are primarily described in context with methods and devices, the invention may also be embodied in a computer program product as well as in a system comprising at least one computer processor and memory coupled to the at least one processor, wherein the memory is encoded with one or more programs that may perform the functions and steps or implement the units and modules disclosed herein.

[0132] Fig. 1 schematically illustrates a block diagram of an embodiment of a device for autonomously operating a wireless telecommunication infrastructure according to at least one intent. The device is generally referred to by the reference sign 100.

[0133] The device 100 comprises at least one child process module 104 (hereinafter referred to as a child process or child agent). Each child process module 104 may be associated with one of at least two KPIs of the wireless telecommunication infrastructure. Each of the child process modules 104 may predict a reward value for each of at least one action. Each of the child process modules 104 may assume multiple possible actions according to one KPI, optionally to improve the KPI so as fulfil the intent (e.g. as far as the respective KPI is concerned). Each of the child process modules 104 may predict a reward value for each of the at least one action based on a rate at which the associated KPI fulfils the intent, sampled over a random distribution of other actions taken by (e.g., assessed by or associated with) the child processes other than the respective child process. The prediction under the random distribution of the other actions may be implemented using a stochastic sampling algorithm such as a Monte Carlo method.

[0134] Each of the actions may be referred to as an arm. The prediction of the reward values for the actions may be referred to as a multi-arm prediction process. Each of the child processes 104 may predict the reward value for each of the at least one actions based on the rate of successful fulfilment of the associated KPI given a wireless telecommunications infrastructure configuration. One or each of the at least two child processes 104 may predict the reward value for each of at least two actions. One or each of the at least two child processes 104 may predict the reward value for each of the at least one action at a particular time, and / or periodically and / or upon being triggered. One or each of the at least two child processes 104 may save (e.g., record or store) the predicted reward values, optionally periodically.

[0135] The child process 104 may have knowledge of the impacts (e.g., effects) of its actions with some uncertainty. The child processes 104 may independently perform child process in parallel. The at least one action of each of the two child processes 104 may change at least one operating parameter of the wireless telecommunications infrastructure that affects the KPI associated with the respective child process 104. Alternatively or in addition, the at least one action of each of the at least two child processes 104 may comprise at least one of changing an allocation of radio resources; changing a frequency reuse; changing a modulation scheme; changing a coding scheme; changing a maximum bit rate, MBR, of a UE; and changing a transmit power level.

[0136] The child process 104 may be provided with data prior to the prediction or prior to each time of the predictions. The provided data may be a historical data of the wireless telecommunications infrastructure (e.g., saved predicted reward values from previous iterations); resources of the wireless telecommunications infrastructure; configurations of the wireless telecommunications infrastructure; and a simulated data of the wireless telecommunications infrastructure.

[0137] The device 100 further comprises a parent process module 108 or 110 (referred to herein as parent process for short). The parent process module 108 or 110 may be in communication with the child process module 104. The child process module 104 may provide the predicted reward values and actions to the parent process module 108 or 110, optionally as a vector in a given time step. The child process module 104 may further provide the parent process module 108 or 110 with the uncertainties corresponding to each reward values. The parent process module 108 or 110 may have an overview knowledge of the at least two child processes. The parent process module 108 or 110 may provide messages to the at least one child process 104. The message may be indicative of an update of the wireless telecommunication infrastructure resources and / or configurations, and / or indicative of the removal (e.g., elimination) and / or provision of an action.

[0138] The parent process module 108 or 110 determines, for each of the at least one action of each of the at least two child processes 104, an uncertainty value (U) based on a number of prior uses of the respective action.

[0139] The parent process module 108 or 110 may further select a subset of the actions based on the reward values predicted by the child processes and their respective uncertainty values (U) determined by the parent process 108 or 110. The subset of the actions may be no action and / or one or more of the actions of the one or more of the child processes 104.

[0140] The parent process module 108 or 110 may further combine the actions of the child process and the corresponding reward values. The combination may be understood as a tensor product of the predicted reward values (e.g., vectors).

[0141] The parent process module 108, 110 may further select a subset of actions based on maximizing the determined uncertainty value (U) and / or maximizing the reward values predicted by the child processes, and / or maximizing the reward values predicted by the child processes and the determined 208 U. The parent process module 108 or 110 may further save the determined U and the predicted reward values, optionally periodically.

[0142] The parent process module 108, 110 may further determine a priority of the intent and / or a priority of the target KPI (KPI_t). The parent process module 108, 110 may further select a subset of actions based on the priority of the intent and / or the priority of the target KPIs, KPI_t. The parent process module 108, 110 may further select a subset of actions based on the on resources of the wireless telecommunications infrastructure.

[0143] The device 100 further comprises an action module 114. The action module 114 may perform or initiate performing of the selected subset of the actions. The action module 114 may be in communication with the child process module 104 and / or the parent process module 108, 110. The parent process may further determine the uncertainty value (U) based on the number of the performed respective actions until the time t, e.g. in a given timestep and / or the number of the given timesteps.

[0144] Any of the modules of the device 100 may be implemented by units configured to provide the corresponding functionality.

[0145] The device 100 may be a computer program product comprising program code portion for performing the method of autonomously operating a wireless telecommunications infrastructure according to the at least one intent.

[0146] The device 100 may be implemented in a node of the wireless telecommunications infrastructure. The node and the components of the wireless telecommunications infrastructure controlled according to the action may be in communication (e.g., via a backhaul network), e.g. for performing the action and / or for obtaining the information after the action. The node may be a network node and / or core node of a wireless telecommunication infrastructure, for example as described below.

[0147] Fig. 2 shows an example flowchart of a method 200 for autonomously operating a wireless telecommunication infrastructure according to at least one intent. Optionally, the method 200 may be performed according to more than one intent. The intent may have at least one KPI, optionally two or more. The wireless telecommunication infrastructure may be provided with a child process module 104 corresponding exclusively to one KPI. The wireless telecommunication infrastructure may be provided with as many child process modules 104 as necessary to correspond exclusively to one KPI.

[0148] In a step 204, each of the child processes 104 predicts a reward value for each of at least one action based on a rate at which the associated KPI fulfils the intent, sampled over a random distribution of other actions taken by the child processes 104 other than the respective child process 104.

[0149] Optionally, in a step 206, the parent process module 108, 110 may combine (e.g., for each of the actions of the child processes) the corresponding reward values.

[0150] In a step 208, the parent process module 108, 110 may determine for each of the at least one action of each of the at least two child processes 104, an uncertainty value (U) based on a number of prior uses of the respective action. In a step 210, the parent process module 108, 110 may select a subset of the actions based on the reward values predicted 204 by the child processes 104 and their respective uncertainty values (U) determined 208 by the parent process 108, 110. The parent process may further select 210 a subset of actions based on resources of the wireless telecommunications infrastructure and / or configurations of the wireless telecommunications infrastructure.

[0151] In a step 214, the method 200 performs or initiates performing the selected 210 subset of the actions.

[0152] Optionally, in a step 202, the method 200 provides data to at least two child processes. The provided data is at least one of a historical data of the wireless telecommunications infrastructure; resources of the wireless telecommunications infrastructure; configurations of the wireless telecommunications infrastructure; and a simulated data of the wireless telecommunications infrastructure.

[0153] Optionally, in a step 212, the parent process module 108, 110 further determines a priority of the intent and / or a priority of the target KPIs (KPI_t). The parent process module 108, 110 may further select 210 a subset of actions based on the priority of the intent and / or the priority of the KPI_t.

[0154] Optionally, in a step 216, the method 200 updates at least one of the predicted reward values for each of the at least one action; the uncertainty value of the at least one action; and a number of actions per child processes.

[0155] The method 200 may be understood as an algorithm for resolving conflicts in autonomous cellular networks without human intervention, as contemplated in 6G. The method (e.g., algorithm) may use a multi-armed bandit (MAB) procedure to ensure that the priorities of the different services, such as WebRTC and URLLC, are aligned and do not cause conflicts.

[0156] The method 200 may further use a hierarchical architecture, wherein each of the child processes 104 (e.g., KPI proposal agent) corresponds to a specific KPI of a service to be provided, such as WebRTC or URLLC. Each of the child processes 104 (e.g., KPI proposal agent) may have a set of arms representing the different priorities that can be given to the service. Each of the arms may correspond to a single action. The reward value for each action (e.g., arm) may be determined by whether or not the service targets, such as QoE for WebRTC and delay bound for URLLC, are met.

[0157] The method 200 allows scalable implementation where multiple services can be managed autonomously in a practical way. Furthermore, it allows online exploration that is critical in cases where the network has no or very little information.

[0158] The method 200 employs a new intent management framework (IMN, or intentbased management framework) comprising different modules (e.g., phases), such as the child process module (e.g., "proposal agent"), responsible for proposing an action (i.e. a new network configuration) to solve a problem on a network metric such as increasing latency.

[0159] Autonomously operating wireless infrastructures are expected to be a key feature of 6G, enabling self-organizing, self-optimizing and self-healing networks without human intervention. The method 200 may use a multi-armed bandit procedure to ensure that the priorities of the different services, such as WebRTC and URLLC, are in alignment and do not cause conflicts. The proposed method 200 may be used to efficiently allocate resources and prioritize services in real time, resulting in improved performance and user experience.

[0160] The method 200 may propose a new algorithmic perspective to the current intentbased management framework (IMF), which may be more robust to unknown environments and it is scalable, as we have briefly explained above.

[0161] The method 200 may be performed by the device 100. For example, the module 104 may perform the step 204, etc.

[0162] Fig. 3 shows a schematic diagram of an intent management function according to the prior art. The intent management function can be used by the telecommunication infrastructures, such as the 5th generation (5G) network. A 5G network is required to support many different types of services, such as ultrareliable low latency communication (URLLC) (e.g. robot control), enhanced mobile broadband (eMBB) (e.g. webRTC), and mMTC (massive machine type communication) (e.g. massive loT of mloT) services. Each service may have different requirements. For example, a URLLC service may require its latency to be less than 1 ms and packet loss to be less than 0.1%. In another example, a WebRTC service may require its QoE to be greater than 4, or mloT may require its energy consumption to less than 1 joule per day. These requirements can be also known as KPIs, so different services may have different KPIs in terms of numbers and types. Fulfilment of a service means that all KPIs of these services are successfully met. The goal of a network is to support as many services as requested, given the network resources. At the same time, as the number of different requirements increases, it is necessary to take decisions in an autonomous way to fulfil all these KPIs. In an autonomous approach, the requirement comes as a high-level intent, which is a declarative information object that defines the requirements of a service that the network is expected to fulfil.

[0163] To manage the intents, networks use closed loops (CL) in different domains, as shown in Fig. 3. A closed loop is a management structure where the system works on a managed entity with a specific goal and is able to monitor and take action on it through a feedback loop. Each closed loop consists of several agents. Fig. 3 shows the architecture of Intent-based Automation (IBA) architecture that supports the implementation of closed loops. Data grounding agents can collect raw data describing the state of the managed environment. Once problems are identified and stored in the knowledge base, the cognitive core can create goals and select agents to propose solutions. Prediction agents estimate the impact of proposals on the system state before the proposed actions are recommended and executed in the managed environment. When predictions are available, the system estimates their impact on all active expectations. The evaluation agent receives the estimated impact of all proposed actions on all active expectations and selects the best action in terms of global network utility or cost. In the process, this component must identify conflicting actions - actions that improve some expectations (unmet expectations become met expectations) but degrade other expectations (met expectations become unmet expectations).

[0164] As example of a possible action that the network can take is to change the maximum bit rate (MBR) of the UEs registered with the service, or to change the priority level of these services, or to move the service from the cloud to the edge. An action that is selected to improve a KPI of one service may degrade the KPI of another service (or its service). For instance, the network may increase the priority of the WebRTC service to improve its QoE, but this action may negatively impact the packet loss of URLLC service. This situation leads to a conflict, which is resolved at the evaluation agent after receiving all the predicted KPIs for each proposed action. Hence, a good level of prediction is crucial for the IBA to operate efficiently.

[0165] In the current IBA framework, when a problem is detected (i.e., a degradation of the latency KPI), the corresponding proposal agent acts, and it may know what to propose to solve the problem. It is assumed that the proposal agent is well trained before proposing an action, and it is aware of the impact of different actions on the KPI. It is also assumed that no new KPI enters the control of the network, and that proposal agents are provided with the information about all possible KPIs and actions. On the contrary, in practice, a proposal agent cannot be aware of the impact of all its actions on the KPI and a proposal agent has to learn the consequences of some (or all) of its actions during the operation, online. This brings a new challenge and now an online (e.g., real time) learning mechanism is needed for the system for the proposal agents that are not well informed about their actions. Since a proposal agent cannot know the impact of a proposed action, some uncertainty is introduced into the system and needs to be managed effectively.

[0166] The prior art intent management frameworks do not have a good solution for the design of the proposal agent. In other words, the current proposal agents' lack of a good algorithmic point of view, and the agent cannot handle complex scenarios where it needs to propose appropriate actions.

[0167] Fig. 4 shows a schematic block diagram of a second embodiment of the device 100 for performing the method 200. The method 200 provides a solution for redesigning the proposal agent of the prior art. When a prior art proposal agent has to solve multiple problems (multiple KPIs with different requirements), it becomes a complex entity because it has to optimize multiple objectives simultaneously. The important criterion is to have a scalable proposal agent where scalability means that when the number of issues is high, the proposal agent can still propose good actions. The method 200 is scalable as explained below. Scalability may be understood as the capability of the child process 104 and it measures how many issues it can solve at one time. The method 200 proposes a child process 105 that deals with only one KPI at a time, and it only needs to deal with a single objective that is easy to learn and optimize. The child process 104 deals with a single KPI, and we do not assign multiple KPI issues to a single KPI. The child process 104 may behave as a proposal agent according to the prior art. The child processes 104 propose actions, but they may also learn not to propose every action because some of those actions are not useful. Consequently, the parent process 108, 110 combines all the information provided by the child processes 104 and learns which actions are the best given the problems to be solved.

[0168] The parent process 108, 110 may use some specific methods (e.g., some machine learning algorithms) to learn the uncertainty of each action. For example, the parent process 108, 110 may use an upper confidence bound (UCB) to be used with MAB. MAB plus UCB may measure how good an action is and how much uncertainty it contains. The parent process 108, 110 may select an action that maximizes MAB plus UCB. If an action is new, its uncertainty must be high, and this fact causes UCB to be high, and the parent process 108, 110 may tend to select this action (e.g., explore). In this way, the parent process 108, 110 may learn (e.g., online). The role of the parent process 108, 110 in this sense is critical. Worth to mention that the parent process 108, 110 may act as an "evaluation agent" in our proposed framework, but the evaluation agent of the prior art lacks the capabilities of the parent process 108, 110.

[0169] In the multi-arm bandit usage, each child process 104 may decide on its action (e.g., arm), but the parent process 108, 110 may actually make the final decision as it behaves like a central authority. In this sense, the method 200 proposes a hierarchical MAB, which is different from the common MAB usage.

[0170] Therefore, the method 200 provides a new algorithmic perspective to the current intent-based management framework (IMF). For example, in the current IMF, the prediction agent is replaced by a much simpler child process module 104. From this perspective, the method 200 is also a different implementation. The parent process 108, 110 is also capable of exploring the unknown part of the networks and it can make decisions by taking into account the uncertainty coming from the less known actions. The parent process 108, 110 may be able to strike a balance between the current expected utility and the possible improved utility. The possible improved utility may come from by trying the less known actions. In summary, the parent process 108, 110 is more capable in the final action selection 210. These features will make the IMF of the method 200 more intelligent and robust. Each KPI may have a corresponding child process module 104, and each child process module 104 may have a number of actions (e.g., arms) that may be used to solve its KPI problem. Each action of the child module 104 has an associated reward value. The reward value may comprise of the reward values corresponding to each action (e.g., arm).

[0171] For example, a child process 104 may be designed to solve a QoE issue if any. This child process 104 may have a set of actions, and each action can have a different impact on the QoE. Some actions may significantly improve the QoE but some others may degrade it. A reward value may be defined as how much an action can improve the QoE at a given network condition (e.g., network configuration). For example, the reward value may be defined for an action as a rate of successfully fulfilling the intent, optionally for 90% and / or 95% and / or 100% of the time. For instance, as an action, a priority is set to 20, and as a result of this action the QoE is above a target (i.e., above 4 out of 5) for 80% of the time for more than %90 of the UEs.

[0172] In principle, the child process 104 cannot be aware of what other child processes 104 propose for their own KPI issue. Therefore, there may be conflict among the child processes 104.

[0173] Although the child process 104 may have some knowledge of the impact of its actions on the target KPI, this knowledge may not be certain, and there might be some level of uncertainty on the impact of these actions since these actions may not have been tried many times. Also, some actions also may not be tried ever before.

[0174] With this realistic assumption, there is also some uncertainty about the reward of actions; hence, the child process 104 has to learn this reward with a good level of certainty. Two challenges here are: a) The child process 104 are not fully knowledgeable about the impact of their actions, and b) The child processes 104 are not aware of what actions other child processes 104 will propose (e.g., what reward values may predict 204).

[0175] For the first challenge, the method 200 proposes an online learning strategy for the child process 104 to learn the impact of its actions accurately, for example, a combination of reinforcement learning and / or federated bandit. For the second challenge, the method 200 proposes a distributed mechanism as child processes 104 that cannot communicate with each other due to the scalability of the problem.

[0176] Fig. 4 shows that each child processes 104 associated with a KPI proposes at least one action to fulfill the intent (e.g., overcome a problem). Then, the child processes 104 may predict a reward value for each of the at least one action. The child process 104 may have some knowledge about the impact of its actions with some uncertainty. Some actions may not have been tried before. Each child process 104 may propose an action and predict 204 the reward value for each of at least one action based on prior knowledge, e.g., some offline information and data and / or trained some machine learning algorithms. Each Child process 104 may propose a set of actions (possibly all its actions). Each action has a reward value. The reward value may be predicted 204 via a prediction module within the child process 104. This prediction 204 mechanism may include an ML model.

[0177] The child process 104 may eliminate one or some of the actions that cannot improve the associated KPI and / or cannot improve the associated KPI with good certainty. Each child process 104 may learn the reward value of its actions, and may eliminate the actions with very low predicted 204 reward values.

[0178] The child processes 104 may be in communication with the parent process 108, 110 and may send the reward value to the parent process 108, 110. The parent process 108, 110 may be referred to as an evaluation agent.

[0179] The parent process 108, 110 may receive all the predicted 204 reward values of the actions from each child process 104. The parent process 108, 110 may combine the received reward values, optionally based on a utility function and / or add weight coefficients to the combined reward values. The parent process 108, 110 may further find the best configuration to maximize the utility function and uncertainty.

[0180] The parent process 108 may further manage the uncertainty in the reward values and actions, and based on the expected utility and the amount of uncertainty, it proposes actions that are applied to the real network.

[0181] The parent process 108, 110 may further monitor the actual measurements, optionally after performing 214 the selected 210 subset of actions. The parent process 108, 110 may send the actual measurements to the child process 104 for the next prediction 204. The parent process 108, 110 and the child process 104 may be in communication and iterating the steps of the method 200 until a final stable action is performed. The child process 104 and the parent process 108, 110 may continuously learn in each iteration of the steps of method 200. According to some embodiments, the method 200 proposes an improved hierarchical and federated multi-armed bandit structure, through meta-learning. Multi-armed bandits may be understood as an online learning paradigm based on the explore-exploit dilemma with limited resources to maximize the expected gain. In the standard stochastic Multi-Armed Bandit (MAB) settings, a single player plays K arms directly, and each arm has its own reward value. In the context of proposed method 200, the "arm" may be defined as each action per service, such as priority in the network. MAB approach may be understood as a solution for the challenge a) explained above, where online learning is required.

[0182] The setting that may fit a federated MAB problem is one where the actions taken by worker MABs are used by a master to resolve conflicts and arrive at a final decision. In dynamic environments with limited observations, federated bandit methods may be typically more suitable than traditional reinforcement learning methods, especially if the reward function is sparse or noisy. This is because federated bandit methods focus on optimizing the expected reward given the available context and do not rely on building a complete model of the environment, whereas traditional reinforcement learning methods aim to maximize the long-term expected reward and require a complete model of the environment.

[0183] Both Reinforcement Learning (RL) and Federated Bandit (FB) may be suitable for a dynamically changing but observable environment, but the choice may depend on the specific requirements and characteristics of the problem.

[0184] RL is generally more suitable when the child process 104 and / or the parent process 108, 110 needs to learn a policy for sequential decision-making tasks in which the actions can affect future observations and rewards. In this case, the child process 104 and / or the parent process 108, 110 may learn from trial and error by interacting with the environment (e.g., optionally the child process and the parent process may be understood as the environment for each other) and receiving feedback in the form of reward values. The parent process 108, 110 goal may be to maximize the cumulative reward values over time.

[0185] FB is more suitable when the child process 104 and / or the parent process 108, 110 needs to make a single decision or a sequence of independent decisions based on the current context, which may include observable features or attributes of the environment. In this case, the child process 104 and / or the parent process 108, 110 may try to optimize the expected rewards for each decision (e.g., selection 210), but the decisions may not be sequential, and the outcomes of one decision may not affect future observations or reward values.

[0186] In a dynamically changing but observable environment (e.g., in a wireless telecommunication infrastructure), both RL and FB may adapt to changes in the environment by updating their policies based on the new observations. However, RL may be more suitable when the changes in the environment affect the sequence of decisions (e.g., selection 210) or the long-term reward values. Meanwhile, FB may be more suitable when the changes in the environment affect the immediate reward values or the context in which the decisions (e.g., selection 210) are made. However, from an implementation perspective, according to some embodiments, FB may be more practical as the state definition of RL is not straightforward, and in a non-stationary environment, instead of looking for longterm reward values focusing on immediate reward value may be more feasible and practical.

[0187] Each child process 104 may be associated with a specific KPI. For example, Fig. 4 shows two child processes 104. One child process-1 104, may be associated with the QoE KPI for the WebRTC service and child process-ll 104 may be associated with the Latency KPI for the URLLC service. There may be many child processes 104 in the network. The responsibility of each child process 104 may be helping their service get enough resource to fulfill its requirements. To do so, each child process 104 may have a set of arms (i.e., actions), and each arm may have a reward value that depends on the given target values for each service KPI. For example, when Child process-1 104 plays arm as changing its priority level from 30 to 10, a reward value may be defined as how often the QoE KPI is above its target.

[0188] Below Table 1 gives an example of reward values under different configurations. Table 1

[0189] Table 1 may be used as an example of the reward values and the data set for the child process 104 offline training.

[0190] According to some embodiments, an ML model may be used by the child process 104 to predict 204 the expected reward values. The input to this ML model may include which actions (e.g., arms) are taken. In the example in Table 1, there are two services. One service is, for example, a video service and its relevant KPI is QoE. Other service is, for example, the URLLC aiming to fulfill its expected KPI of Latency. Each of the services has actions (e.g., arms).

[0191] In this example, arm may be the service priority level, which takes values between 10 and 100. As an example, for the first row of the data set it means that when video and URLLC services want to set their priority level to 100 and 60, respectively and the number of UEs for those services is 2 and 4. Also, threshold 1 and threshold 2 correspond to the target KPIs values of those services. In other words, the Latency must be less than 40 ms for URLLC and QoE must be greater than 3.5 for video. As an exemplary calculation of the reward value, the configuration may be run M times, and the number of trials those target KPIs are met is obtained. For example, for the first configuration at the first row in Table 1, the Latency target is achieved 0.636 fraction of M trails whereas QoE KPIs is met 0.767 fraction of the time. The output may be defined as the expected reward for each KPIs, which mean, as an example, 0.636 is the expected reward for URLLC under the configuration defined at the first row of Table 1. Other rows of Table 1 show the expected reward values (outputs columns) with different network configurations (inputs columns).

[0192] Services may change their arms (i.e., priority levels) to achieve more resources but every service may require a good priority and there may be conflict and bad resource allocation. Therefore, it is important for those services to select their arms by considering their interests and also others' benefits. By generating the reward values as elucidated earlier, we can mitigate the individual effects of UE and create a more universally applicable set of meta-data for the initial training of the MAB (e.g., child process 104 and / or the parent process 108, 110). In the main process the MAB formulates priority tuples by using these reward values in the predictor (e.g., the child process 104), which makes partial observations of the system and utilizes the feedback generated throughout the meta-learning session. The offline data set shown in Table 1 may be generated with an ML sandbox or a simulation environment. Historical data may also be used as offline training data. In the historical data, some actions (e.g., arms) may have been taken under different network conditions without using any particular algorithm such as multiarm bandit. This data may still be useful and may be used as training data for the method 200.

[0193] However, this reward value may depend on many other factors such as what actions other child process 104 may propose and / or the network dynamics such as the number of UE for each service. However, the child process 104 does not know what actions other child processes 104 propose to maintain scalability. Hence, each child process 104 has to predict 204 the reward value of its actions. In Fig. 5, the predictor is located at each child process 104 and it may predict 204 the reward values of each action of the child process 104. The predictor may take some input parameters such as the number of UEs for each services, target KPI values, possible actions that the child process 104 can take. Since the child processes 104 do not know what other child processes 104 propose, this information is considered random and it is provided to the predictor as input. Then, the predictor may provide a predicted reward value for each action of the child process 104. Fig. 5 shows a schematic block diagram of a first embodiment of the child process 104. The input information may be history data of the network, the current dynamic of the network, the random actions from other child processes 104 and an action to improve the KPI. The output of the predictor may be the predicted 204 reward value for each of the at least one action.

[0194] Each child process 104 may make the prediction 204 for each arm multiple times, as the random actions from other child agents can be averaged out. Then, each child process 104 may determine the expected reward value (e.g., predict 204 the reward value) of its each actions at that time instant. Let Rkltbe the expected reward value for child process-1 104 with arm k at a time t. Each child process 104 may create a vector that contains all the expected reward values of each arm / action as its element. Let Rtl= [ 7?it, R2lt, ... , RKlt] the vector of expected reward of child process-1 and this vector includes K number of arms with their expected reward at time t. Each child process 104 may update this reward value vector periodically or upon receipt of a system update and / or a trigger message.

[0195] The method 200 may assume the following assumptions:

[0196] (i) Child processes 104 may be unaware of the proposed actions of other child processes 104. They only observe the outcomes of these actions in relation to their own and consider them random noise.

[0197] (ii) There may be a known functional mapping between resources allocated to a user equipment and its corresponding Quality of Experience (QoE) for each service in use. Note that such a mapping may be obtained offline based on subjective experiments.

[0198] (iii) The actual resource allocation of the network may be based on the outcome of the policy that is performed by the gNB according to the existing scheduling protocols.

[0199] The assumption (i) may result in a less than favorable setting for achieving the optimal solution, but this is considered to improve the scalability of the solution. The assumption (ii) may not be a strong assumption, since a quantitative QoE function is required for any new service. Finally, the assumption (iii) may be an assumption that allows for the decomposition of the problem into sub-problems, and allows for a modular implementation. The architecture proposed by the method 200 may have affinity to well-known DiffServ architecture on the Internet. In particular, the objective may be to autonomously assign weights to flows (i.e., services) to ensure statistical guarantees. The assigned weights may reflect the share of the resources among the services. Once the shares are determined, the gNB may be responsible for allocating the necessary PRB (and possibly with the aid of power control) to ensure each user equipment subscribing to the service receives the actual share in the end. The latter mentioned resource allocation mechanism is already available, and thus, the proposed method 200 works on top of this mechanism.

[0200] A child process 104 may create an active action (e.g., arm) set, where there may be possible good actions that can improve the target KPI. The child process 104 may observe that some actions cannot improve the target KPI under many different conditions (e.g., optionally including many different combinations of other child process 104 actions and also various dynamics network parameters). Then, a child process 104 may eliminate those actions from the active set. This process can be dynamic where an action may be eliminated from the active set at one time but later it may be included again.

[0201] Each child process 104 may create its active sets and the expected reward values (e.g., vector). As the resources of the network are shared, there may be a possibility that the actions of a child process 104 may conflict with one or more other child processes 104. This conflict may be handled at the parent process 108, 110 as shown in Fig. 4 that has a larger perspective on conflicts between services and acts as the final decision maker for the final actions (e.g., selecting a subset of actions) that will be applied to the real network.

[0202] Fig. 6 shows the parent process 108, 110 according to the method 200. The parent process 108, 110 may receive the reward vector from each child process 104 and may make a combination set from different actions coming from the child processes 104. For example, child process-1 104 and child process-ll 104 may send their reward vectors to the parent process 108, 110 / ?[ = > — > RK,C ] and / ?[ = [ Rit, R ,t respectively. Then the parent process 108, 110 may create a combination set that comprises a "parent reward" for each combination. As an example, the reward values R^tand / ?3tare combined, the effective parent reward may be given as <2 ([1,1], [2,3], t) = f (R^t, R^t) which means that if the first action from child process-1 104 and third actions from child process-ll 104 are taken, the parent reward is <2 ([1,1], [2,3], t). This parent reward may be simply a weighted average Q([l,l], [2,3], t) = wlR^t+ w2 Rltwhere wl and w2 may be the weights representing the importance of the QoE and the Latency KPIs for this example. In this example, the number of actions that the parent process 108, 110 can play is the number of each possible combinations. For instance, when child process-1 104 has an action set of priority ranging from 10 to 100 with a step of 10 (total 10 actions / arm) and if child process-ll 104 has also the same set, then the parent process 108, 110 has 10x10 = 100 numbers of actions / arms.

[0203] As another example assuming:

[0204] Child process- 1 104 action set is priority levels: [10, 20, 30,..., 100] -> 10 actions Child process- II 104 action set is priority levels: [10, 20, 30,..., 100] -> 10 actions

[0205] And the parent process 108, 110 has

[0206] [(10, 10), (10,20), (10,20), ...,(20, 10), (20,20), ...,(100, 10), ...,(100, 100)] ->100 actions. (10,30) means that when child process-1 104 takes 10 and child process-ll 104 takes 30 as their actions. Let each combined action at parent be denoted as a. As an example a can be a combined action from

[0207] [(10, 10), (10,20), (10,20), ...,(20, 10), (20,20), ...,(100, 10), ...,(100, 100)]. Therefore, Q(a, t) may be the parent reward value with the action a at time t. The parent process 108, 110 may determine the list containing the Q(a,t) for each combination.

[0208] Once the parent process 108, 110 receives all reward vector from the child processes 104 and determines Q(a,t), one simple decision making process for the parent process 108, 110 may be to find the final action a that will be applied to the real network with the maximum parent reward Q(a,t).

[0209] A simple solution performed by the parent process 108, 110 to find the best action based on: a* = argmax Q(a,t).

[0210] However, this solution may not yield an overall satisfactory result because of the fact that not all actions have been tried and taken many times and there might be some certain level of uncertainty with some actions a. Therefore, the parent process 108, 1110 may need to handle this uncertainty as well. The parent process 108, 110 may obtain a good level of certainty for the actions to make a good assessment of the impact of these actions. Therefore, in addition to the expected reward value of the actions, the parent process 108, 110 may consider the uncertainty level of each action a. Let U(a,t) be the amount of uncertainty with action a at time t. Then, the parent process 108, 110 according to the method 200 may select 210 the action a as the final action to be applied to the real network as follows: a* = argmax Q(a,t) + U(a,t)

[0211] This method 200 may try to get the maximum expected reward value but also tries to minimize the uncertainty. As an example for U(a,t), UCB algorithm can be used:

[0212] U(va, ty= & N(a)

[0213] Where N(a) is the number of times the action a has been applied to the real network.

[0214] Fig. 7 shows a schematic block diagram of a third embodiment of the device 100. In the wireless telecommunications infrastructure, problems (e.g., issues) to be solved may rise randomly and proposed actions by different child processes 104 may be asynchronous. In other words, two different child processes may propose their actions and the reward value for each of the at least one actions at different time instances. The device 100 performing the method 200 may allow this, and final actions are selected 210 at the parent process 108, 110 whenever it determines it is necessary to take an action, e.g., to fulfill the intent.

[0215] However, the child processes 104 may keep sending their predicted reward values to the parent process 108, 110 but it does not necessary mean that an action is always selected whenever the reward values has been received, e.g., selecting 210 no action may be considered as an action selection by the parent process 108, 110. Also, the final selected 210 action (e.g., a subset of the actions) may be determined according to a final utility function (optionally plus uncertainty). Therefore, if the contribution of each child process 104 to the total utility is different (i.e., the weight wl and w2), then the parent process 108, 110 may select 210 the action that will improve the utility function (e.g., KPI) the most. In case that the proposed actions and their reward values are similar (optionally equal) the parent process 108, 110 may randomly select 210 one of the actions since the total utility function may not change. Hence, this will prevent any ping-pong effect. Fig. 8 shows a schematic block diagram of a second embodiment of the child process 104 of an embodiment of the device 100. The child process 104 may utilize new network (NW) function such as network data analysis function NWDAF. As it shows in Fig. 8 the child process 104 may use an NWDAF service consumer, i.e. an NWDAF containing AnLF to request and get from another NWDAF, i.e. an NWDAF containing MTLF ML Model Information, using Nnwdaf_MLModellnfo services.

[0216] Fig. 9 shows a schematic block diagram of a third embodiment of the child process 104 of an embodiment of the device 100. As depicted in Fig. 9, a ML Model Information is used by an NWDAF containing AnLF to derive analytics. An NWDAF may be at the same time a consumer of this service provided by other NWDAF(s) and a provider of this service to other NWDAF(s).

[0217] In Figs 8 and 9, the NWDAF may be connected to the each child process 104 and it may help for the reward value prediction 204 of the child process 104. Alternatively or in addition, the NWDAF may be connected to the parent process 108, 110 and it may help for performing any steps of parent process, such as determining 208 the uncertainty value and / or selectin 210 a subset of the actions, etc.

[0218] Fig. 10 shows a schematic block diagram of a fourth embodiment of the device 100. As there may be many actions proposed by the child processes 104, the parent process 108, 110 may determine a huge number of combinations 206 of actions. Therefore, the actions with reward values lower than a predefined threshold and / or high certainty may be eliminated (e.g., by the parent process 108, 110 and / or by the child process 104) and the active set of actions may be updated 216. The active set of actions may be understood as a set of actions and their associated predicted 204 reward values that has been sent to the parent process 108, 110. For instance, the parent process 108, 110 may determine a function F(a,t) for each action a, and make a list of actions and sort the list of actions according to a descending reward value. The parent process 108, 110 may pick the M number of actions from the top of the list of actions, i.e., the actions with the highest reward values will be selected 210 and the actions with the lowest reward values will be eliminated.

[0219] We have implemented our algorithm in Python. In our scenario there are two serves, namely WebRTC and URLLC services. For each service, number of UEs can vary over time. For WebRTC service, the relevant KPI is QoE and for each WebRTC UE QoE should be higher than a threshold. For URLLC service, there are two relevant KPIs, that are the Latency and the Packet Loss, and for each of those KPIs there are associated thresholds. These thresholds value can be arbitrary and can take any value and comparisons will be based on their effectiveness in meeting service level agreements (SLAs) stated above in our experiments, we aim to compare the performance of three different approaches: the proposed CSO (Contextual Service-oriented)-MAB, the greedy child approach, and the CSO-MAB with Thompson sampling.

[0220] The network is started to simulate by maximizing the MBR to 3 Mbps and it takes the parameters of priorities, number of UEs and duration input for the services as input. In the network simulation, 10 Mbps as total bandwidth is allocated from core to edge, namely child's, depends on the priority per service proposed by algorithms at the parent. Our examples are consisting of three different scenario which provides an environment with dynamical changes on QoE, the Latency target and number of UEs as seen in the tables. All phases in scenarios are arranged to not exceed 10 Mbps in total. Our three basic scenario are selected to show effects of alteration on targets and UEs clearly. Dynamically changing parameters in the tables are Y, q, T are QoE target in Mbps, the Latency target in millisecond and duration of the phase in seconds respectively and the notion of durations in tables can be scaled to rounds equally, all durations are 20 for each patch as seen in the tables. The performance measures of the algorithms are cumulative number of SLA violations. The cumulative number of SLA represent the summation of all deviation from the service targets, when targets are not satisfied per round then metric of Euclidean distance, | Ytarget - Yproposed | or | qproposed - qtarget | , is used to determine amount of violation.

[0221] Figs. 11A and 11B shows an experimental result of an autonomously operating a wireless telecommunication infrastructure according to at least one intent according to the method 200 in comparison with two prior arts. The child process 104 may be scalable but may propose bad actions and the issue may not be solved.

[0222] Figs. 11A and 11B shows simulation results of the performance of the method 200 in comparison with some prior art methods. In a dynamic environment (e.g., dynamic network configurations) where number of UEs or the target of KPIs may change at any time, the method 200 may perform the best approach in comparison with prior art. The method 200 may additionally have the ability to learn new situations online, which is a major advantage of using multi-arm bandit (MAB). In particular, the child process 104 may lack the knowledge about the impact of some actions, e.g., due to those actions may not have tried before or those actions are newly instated. The child process 104 and / or the parent process 108, 110 according to the method 200 may learn the impacts of these actions over time and incorporate them to solve the issue.

[0223] The Table 2 shows the network configuration for as an exemplary Experiment-1.

[0224] Table 2

[0225] The Y and q refers to the target KPI values for the QoE and the Latency, respectively. The number of UEs for Video and URLLC services are equal to 5 which are represented by N-l and N-ll. The algorithms, i.e., Tompson sampling (TS), greedy child and CSO-MAB according to the method 200, are performed over different phases and each phase takes T seconds. For example, in Experiment-1 in total we have 5 phases and each phase is played 20 seconds. The Experiment-1 shows that the KPI_t of QoE may change in different phases while the KPI_t of the Latency is constant.

[0226] Fig. 11A shows the Latency violation with Experiment-1 configurations. It clearly demonstrate that the method 200 learns faster and performs significantly better than prior art algorithms as the number of the Latency violation the lowest. Fig. 11B shows the QoE Violation with Experiment-1 configurations. It shows that the method 200 learns faster and performs significantly better than prior art algorithms as the number of QoE violation the lowest.

[0227] The Table 3 shows the network configuration for as an exemplary Experiment-Il.

[0228] Table 3

[0229] The Y and q refers the target KPI values for the QoE and the Latency, respectively. The number of UEs for Video and URLLC services are equal to 5 which are represented by N-l and N-ll. The algorithms, i.e., Tompson sampling (TS), greedy child and CSO-MAB according to the method 200, are performed over different phases and each phase takes T seconds. For example, in this experiment in total we have 5 phases and each phase is played 20 seconds. The Experiment-ll shows that the KPI_t of the Latency may change in different phases while the KPI_t of the QoE is constant.

[0230] Fig. 12A shows the Latency violation with Experiment-ll configurations. It clearly demonstrate that the method 200 learns faster and performs significantly better than prior art algorithms as the number of the Latency violation the lowest.

[0231] Fig. 12B shows the QoE Violation with Experiment-ll configurations. It shows that the method 200 learns faster and performs significantly better than TS and similar to the Greedy children (as child processes). Nevertheless the method 200 demonstrate a more stable performance than the Greedy Childs algorithm.

[0232] The Table 4 shows the network configuration for as an exemplary Experiment-Ill.

[0233] Table 4

[0234] The Y and q refers the target KPI values for the QoE and the Latency, respectively. The number of UEs for Video and URLLC services change in different phase of the experiment. Also, in different phases, the target KPIs may also change. The algorithms, i.e., Tompson sampling (TS), greedy child and CSO-MAB according to the method 200, are performed over different round denoted by phases and each phase takes T rounds. For example, in this experiment in total we have 10 phases and each phase is played 10 seconds. The Experiment-lll has more dynamic configurations in comparison with the Experiment-1 and II. Fig. 13A shows the Latency violation with Experiment-Ill configurations. It clearly demonstrate that the method 200 learns faster and performs significantly better than prior art algorithms as the number of the Latency violation the lowest.

[0235] Fig. 13B shows the QoE Violation with Experiment-Ill configurations. It shows that the method 200 learns faster and performs significantly better than prior art algorithms as the number of QoE violation the lowest.

[0236] Figs 11A to 13B shows in the per-round average of 5 long run simulations for different phases in the Table 2 to Table 4.

[0237] As shown in the Figs. 11A to 13B and Experiments above, the algorithms are evaluated with respect to QoE KPI changes, which may become harder as rounds go on. At this case, the Latency may be more achievable KPI target while the algorithms try to satisfy QoE KPI target meanwhile. That's why, the algorithms compromise from achievable targets to handle this trade-off. From the Figs. 11A to 13B, this tradeoff is definitely set in a most efficient way by the algorithm of the method 200. For the QoE satisfaction, TS algorithm tries to compromise from the Latency which is not enough as seen and greedy approach violates unnecessarily vast amount of service level in the Latency due to its incapability about finding optimal tradeoff autonomously.

[0238] In overall, deviations of Greedy childs algorithm in Figs. 11A to 13B are slightly bigger than the other methods but they just focus on solely one target and give a full importance to that target regardless of trade-off between services. Therefore, it can be seen as more successful than TS, if we just take an evaluation from the perspective of one service rather than the overall cell of the network. On other hand, TS try to solve this trade-off but cannot succeed as method 200 as demonstrated in Figs. llA to 13B.

[0239] The numerical outcomes of the real experiments are mostly overlapping with what we expect from the algorithms selected as benchmarks in the simulated experiments. In the Greedy childs, the algorithm behaves quite greedy to satisfy all service level agreements without any authority like the method 200 defines as parent process 108, 110 or evaluator agent. On the other hand, the results for the CSO without UCB approach show that there is a need for acquisition function to explore environment for resolution of conflict in an optimal way. The technique may be applied to uplink (UL), downlink (DL) or direct communications between radio devices, e.g., device-to-device (D2D) communications or sidelink (SL) communications.

[0240] The device 100 may be part of a network node 1400 (e.g., transmitting station) or a core node 1500 (e.g., transmitting station) in the wireless telecommunication infrastructure. Alternatively or in addition, the device 100 may be part of a host computer 1710, 1630 and / or part of the cloud.

[0241] Each of the transmitting station 1400, 1500 may be a radio device or a base station. Herein, any radio device may be a mobile or portable station and / or any radio device wirelessly connectable to a base station or RAN, or to another radio device. For example, the radio device may be a user equipment (UE), a device for machine-type communication (MTC) or a device for (e.g., narrowband) Internet of Things (loT). Two or more radio devices may be configured to wirelessly connect to each other, e.g., in an ad hoc radio network or via a 3GPP SL connection. Furthermore, any base station may be a station providing radio access, may be part of a radio access network (RAN) and / or may be a node connected to the RAN for controlling the radio access. For example, the base station may be an access point, for example a Wi-Fi access point.

[0242] Herein, whenever referring to noise or a signal-to-noise ratio (SNR), a corresponding step, feature or effect is also disclosed for noise and / or interference or a signal-to-interference-and-noise ratio (SINR).

[0243] Fig. 14 shows a schematic block diagram for an embodiment of the device 100. The device 100 comprises processing circuitry, e.g., one or more processors 1404 for performing the method 200 and memory 1406 coupled to the processors 1404. For example, the memory 1406 may be encoded with instructions that implement at least one of the modules 102, 104 and 106.

[0244] The one or more processors 1404 may be a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, microcode and / or encoded logic operable to provide, either alone or in conjunction with other components of the device 100, such as the memory 1406, base station functionality. For example, the one or more processors 1404 may execute instructions stored in the memory 1406. Such functionality may include providing various features and steps discussed herein, including any of the benefits disclosed herein. The expression "the device being operative to perform an action" may denote the device 100 being configured to perform the action.

[0245] As schematically illustrated in Fig. 14, the device 100 may be embodied by a base station 1400, e.g., functioning as a gNB. The base station 1400 comprises a radio interface 1402 coupled to the device 100 for radio communication with one or more radio devices, e.g., functioning as a UE.

[0246] Fig. 15 shows a schematic block diagram for an embodiment of the device 100. The device 100 comprises processing circuitry, e.g., one or more processors 1504 for performing the method 200 and memory 1506 coupled to the processors 1504. For example, the memory 1506 may be encoded with instructions that implement at least one of the modules 202, 204 and 206.

[0247] The one or more processors 1504 may be a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, microcode and / or encoded logic operable to provide, either alone or in conjunction with other components of the device 100, such as the memory 1506, core node functionality. For example, the one or more processors 1504 may execute instructions stored in the memory 1506. Such functionality may include providing various features and steps discussed herein, including any of the benefits disclosed herein. The expression "the device being operative to perform an action" may denote the device 100 being configured to perform the action.

[0248] As schematically illustrated in Fig. 15, the device 200 may be embodied by a core node function 1500, e.g., functioning as a fifth generation core (5GC). The core node 1500 comprises an interface 1502 coupled to the device 100 for communication with one or more RAN nodes, e.g., functioning as a base station such as gNB.

[0249] With reference to Fig. 16, in accordance with an embodiment, a communication system 1600 includes a telecommunication network 1610, such as a 3GPP-type cellular network, which comprises an access network 1611, such as a radio access network, and a core network 1614. The access network 1611 comprises a plurality of base stations 1612a, 1612b, 1612c, such as NBs, eNBs, gNBs or other types of wireless access points, each defining a corresponding coverage area 1613a, 1613b, 1613c. Each base station 1612a, 1612b, 1612c is connectable to the core network 1614 over a wired or wireless connection 1615. A first user equipment (UE) 1691 located in coverage area 1613c is configured to wirelessly connect to, or be paged by, the corresponding base station 1612c. A second UE 1692 in coverage area 1613a is wirelessly connectable to the corresponding base station 1612a. While a plurality of UEs 1691, 1692 are illustrated in this example, the disclosed embodiments are equally applicable to a situation where a sole UE is in the coverage area or where a sole UE is connecting to the corresponding base station 1612.

[0250] Any of the base stations 1612 and the UEs 1691, 1692 may embody the device 100.

[0251] The telecommunication network 1610 is itself connected to a host computer 1630, which may be embodied in the hardware and / or software of a standalone server, a cloud-implemented server, a distributed server or as processing resources in a server farm. The host computer 1630 may be under the ownership or control of a service provider, or may be operated by the service provider or on behalf of the service provider. The connections 1621, 1622 between the telecommunication network 1610 and the host computer 1630 may extend directly from the core network 1614 to the host computer 1630 or may go via an optional intermediate network 1620. The intermediate network 1620 may be one of, or a combination of more than one of, a public, private or hosted network; the intermediate network 1620, if any, may be a backbone network or the Internet; in particular, the intermediate network 1620 may comprise two or more sub-networks (not shown).

[0252] The communication system 1600 of Fig. 16 as a whole enables connectivity between one of the connected UEs 1691, 1692 and the host computer 1630. The connectivity may be described as an over-the-top (OTT) connection 1650. The host computer 1630 and the connected UEs 1691, 1692 are configured to communicate data and / or signaling via the OTT connection 1650, using the access network 1611, the core network 1614, any intermediate network 1620 and possible further infrastructure (not shown) as intermediaries. The OTT connection 1650 may be transparent in the sense that the participating communication devices through which the OTT connection 1650 passes are unaware of routing of uplink and downlink communications. For example, a base station 1612 need not be informed about the past routing of an incoming downlink communication with data originating from a host computer 1630 to be forwarded (e.g., handed over) to a connected UE 1691. Similarly, the base station 1612 need not be aware of the future routing of an outgoing uplink communication originating from the UE 1691 towards the host computer 1630.

[0253] By virtue of the method 200 being performed by any one of the UEs 1691 or 1692 and / or any one of the base stations 1612, the performance or range of the OTT connection 1650 can be improved, e.g., in terms of increased throughput and / or reduced latency. More specifically, the host computer 1630 may indicate to the core node 1500 or the network node 1400 or the user equipment (e.g., on an application layer) the intent or the QoS of the traffic.

[0254] Example implementations, in accordance with an embodiment of the UE, base station and host computer discussed in the preceding paragraphs, will now be described with reference to Fig. 17. In a communication system 1700, a host computer 1710 comprises hardware 1715 including a communication interface 1716 configured to set up and maintain a wired or wireless connection with an interface of a different communication device of the communication system 1700. The host computer 1710 further comprises processing circuitry 1718, which may have storage and / or processing capabilities. In particular, the processing circuitry 1718 may comprise one or more programmable processors, application-specific integrated circuits, field programmable gate arrays or combinations of these (not shown) adapted to execute instructions. The host computer 1710 further comprises software 1711, which is stored in or accessible by the host computer 1710 and executable by the processing circuitry 1718. The software 1711 includes a host application 1712. The host application 1712 may be operable to provide a service to a remote user, such as a UE 1730 connecting via an OTT connection 1750 terminating at the UE 1730 and the host computer 1710. In providing the service to the remote user, the host application 1712 may provide user data, which is transmitted using the OTT connection 1750. The user data may depend on the location of the UE 1730. The user data may comprise auxiliary information or precision advertisements (also: ads) delivered to the UE 1730. The location may be reported by the UE 1730 to the host computer, e.g., using the OTT connection 1750, and / or by the base station 1720, e.g., using a connection 1760.

[0255] The communication system 1700 further includes a base station 1720 provided in a telecommunication system and comprising hardware 1725 enabling it to communicate with the host computer 1710 and with the UE 1730. The hardware 1725 may include a communication interface 1726 for setting up and maintaining a wired or wireless connection with an interface of a different communication device of the communication system 1700, as well as a radio interface 1727 for setting up and maintaining at least a wireless connection 1770 with a UE 1730 located in a coverage area (not shown in Fig. 17) served by the base station 1720. The communication interface 1726 may be configured to facilitate a connection 1760 to the host computer 1710. The connection 1760 may be direct, or it may pass through a core network (not shown in Fig. 17) of the telecommunication system and / or through one or more intermediate networks outside the telecommunication system. In the embodiment shown, the hardware 1725 of the base station 1720 further includes processing circuitry 1728, which may comprise one or more programmable processors, application-specific integrated circuits, field programmable gate arrays or combinations of these (not shown) adapted to execute instructions. The base station 1720 further has software 1721 stored internally or accessible via an external connection.

[0256] The communication system 1700 further includes the UE 1730 already referred to. Its hardware 1735 may include a radio interface 1737 configured to set up and maintain a wireless connection 1770 with a base station serving a coverage area in which the UE 1730 is currently located. The hardware 1735 of the UE 1730 further includes processing circuitry 1738, which may comprise one or more programmable processors, application-specific integrated circuits, field programmable gate arrays or combinations of these (not shown) adapted to execute instructions. The UE 1730 further comprises software 1731, which is stored in or accessible by the UE 1730 and executable by the processing circuitry 1738. The software 1731 includes a client application 1732. The client application 1732 may be operable to provide a service to a human or non-human user via the UE 1730, with the support of the host computer 1710. In the host computer 1710, an executing host application 1712 may communicate with the executing client application 1732 via the OTT connection 1750 terminating at the UE 1730 and the host computer 1710. In providing the service to the user, the client application 1732 may receive request data from the host application 1712 and provide user data in response to the request data. The OTT connection 1750 may transfer both the request data and the user data. The client application 1732 may interact with the user to generate the user data that it provides. It is noted that the host computer 1710, base station 1720 and UE 1730 illustrated in Fig. 17 may be identical to the host computer 1630, one of the base stations 1612a, 1612b, 1612c and one of the UEs 1691, 1692 of Fig. 16, respectively. This is to say, the inner workings of these entities may be as shown in Fig. 17, and, independently, the surrounding network topology may be that of Fig. 16.

[0257] In Fig. 17, the OTT connection 1750 has been drawn abstractly to illustrate the communication between the host computer 1710 and the UE 1730 via the base station 1720, without explicit reference to any intermediary devices and the precise routing of messages via these devices. Network infrastructure may determine the routing, which it may be configured to hide from the UE 1730 or from the service provider operating the host computer 1710, or both. While the OTT connection 1750 is active, the network infrastructure may further take decisions by which it dynamically changes the routing (e.g., on the basis of load balancing consideration or reconfiguration of the network).

[0258] The wireless connection 1770 between the UE 1730 and the base station 1720 is in accordance with the teachings of the embodiments described throughout this disclosure. One or more of the various embodiments improve the performance of OTT services provided to the UE 1730 using the OTT connection 1750, in which the wireless connection 1770 forms the last segment. More precisely, the teachings of these embodiments may reduce the latency and improve the data rate and thereby provide benefits such as better responsiveness and improved QoS.

[0259] A measurement procedure may be provided for the purpose of monitoring data rate, latency, QoS and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 1750 between the host computer 1710 and UE 1730, in response to variations in the measurement results. The measurement procedure and / or the network functionality for reconfiguring the OTT connection 1750 may be implemented in the software 1711 of the host computer 1710 or in the software 1731 of the UE 1730, or both. In embodiments, sensors (not shown) may be deployed in or in association with communication devices through which the OTT connection 1750 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software 1711, 1731 may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 1750 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not affect the base station 1720, and it may be unknown or imperceptible to the base station 1720. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling facilitating the host computer's 1710 measurements of throughput, propagation times, latency and the like. The measurements may be implemented in that the software 1711, 1731 causes messages to be transmitted, in particular empty or "dummy" messages, using the OTT connection 1750 while it monitors propagation times, errors etc.

[0260] Fig. 18 is a flowchart illustrating a method implemented in a communication system, in accordance with one embodiment. The communication system includes a host computer, a base station and a UE which may be those described with reference to Figs. 16 and 17. For simplicity of the present disclosure, only drawing references to Fig. 18 will be included in this paragraph. In a first step 1810 of the method, the host computer provides user data. In an optional substep 1811 of the first step 1810, the host computer provides the user data by executing a host application. In a second step 1820, the host computer initiates a transmission carrying the user data to the UE. In an optional third step 1830, the base station transmits to the UE the user data which was carried in the transmission that the host computer initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In an optional fourth step 1840, the UE executes a client application associated with the host application executed by the host computer.

[0261] Fig. 19 is a flowchart illustrating a method implemented in a communication system, in accordance with one embodiment. The communication system includes a host computer, a base station and a UE which may be those described with reference to Figs. 16 and 17. For simplicity of the present disclosure, only drawing references to Fig. 19 will be included in this paragraph. In a first step 1910 of the method, the host computer provides user data. In an optional substep (not shown) the host computer provides the user data by executing a host application. In a second step 1920, the host computer initiates a transmission carrying the user data to the UE. The transmission may pass via the base station, in accordance with the teachings of the embodiments described throughout this disclosure. In an optional third step 1930, the UE receives the user data carried in the transmission.

[0262] As has become apparent from above description, at least some embodiments of the technique allow for an improved selection of a relay radio device and / or an improved selection of a SL connection establishment. Same or further embodiments can ensure that the traffic relayed by the relay radio device is given the appropriate QoS treatment. Many advantages of the present invention will be fully understood from the foregoing description, and it will be apparent that various changes may be made in the form, construction and arrangement of the units and devices without departing from the scope of the invention and / or without sacrificing all of its advantages. Since the invention can be varied in many ways, it will be recognized that the invention should be limited only by the scope of the following claims.

Claims

Claims1. A method (200) of autonomously operating a wireless telecommunications infrastructure according to an intent, the method (200) comprising: performing at least two child processes (104), each associated to one of at least two key performance indicators, KPI, of the wireless telecommunications infrastructure, wherein each of the child processes (104) predicts (204) a reward value for each of at least one action based on a rate at which the associated KPI fulfills the intent, sampled over a random distribution of other actions taken by the child processes (104) other than the respective child process (104); performing a parent process (108, 110), wherein the parent process (108, 110)(i) determines (208), for each of the at least one action of each of the at least two child processes (104), an uncertainty value, U, based on a number of prior uses of the respective action, and(ii) selects (210) a subset of the actions based on the reward values predicted (204) by the child processes (104) and their respective uncertainty values, U, determined (208) by the parent process(108, 110); and performing (214) the selected (210) subset of the actions.

2. The method (200) of claim 1, wherein the at least two KPIs comprise at least one of latency, delay bound, jitter, reliability, data rate, Quality of Service, QoS, and Quality of Experience, QoE.

3. The method (200) of claim 1 or 2, wherein the at least one action of each of the at least two child processes (104) changes at least one operating parameter of the wireless telecommunications infrastructure, which influences the KPI associated to the respective child process (104), and / or wherein the at least one action of each of the at least two child processes (104) comprises at least one of: changing an allocation of radio resources; changing a frequency reuse; changing a modulation scheme; changing a coding scheme; changing a maximum bit rate, MBR, of a user equipment, UE; and changing a transmit power level.

4. The method (200) of any one of claims 1 to 3, wherein the rate at which the associated KPI fulfils the intent is a conditional probability given the random distribution of the other actions taken by the other child processes (104).

5. The method (200) of any one of claims 1 to 4, wherein one or each of the at least two child processes (104) predicts (204) the reward value for each of the at least one action based on the rate of the associated KPI fulfilling the intent given a configuration of the wireless telecommunications infrastructure, and / or wherein one or each of the at least two child processes (104) predicts (204) the reward value for each of the at least one action based on measuring the rate at which the associated KPI fulfills the intent for multiple UEs.

6. The method (200) of any one of claims 1 to 5, wherein the parent process (108, 110) determines (208) the uncertainty value, U, based on a number of times the respective action has been performed (214) in a given timestep and the given timesteps.

7. The method (200) of any one of claims 1 to 6, wherein the parent process (108, 110) selects (210) a subset of actions based on: maximizing the determined (208) uncertainty value, U, or maximizing the reward values predicted (204) by the child processes (104), or maximizing both the reward values predicted (204) by the child processes (104) and the determined (208) uncertainty value, U.

8. The method (200) of any one of claims 1 to 7, wherein the intent or each of the intent and one or more further intents corresponds to a service provided by the wireless telecommunications infrastructure, and / or wherein the one or more service provided by the wireless telecommunications infrastructure comprise at least one of: ultra-reliable and low- latency communication, URLLC; enhanced mobile broadband, eMBB, service; web real-time communication, WebRTC; massive machine-type communication, mMTC; and massive internet of things, mloT, service; and industrial internet of things, lloT, service.

9. The method (200) of any one of claims 1 to 8, wherein one or each of the at least two child processes (104) predicts (204) the reward value for each of at least two actions.

10. The method (200) of any one of claims 1 to 9, wherein each of the at least two child processes (104) is uniquely associated to a different one of the at least two KPIs; and / or wherein each of the at least two child processes (104) is exclusively associated to a single one of the at least two KPIs.

11. The method (200) of any one of claims 1 to 10, wherein the intent comprises a target KPI, KPI_t, for each of the at least two KPIs of the wireless telecommunications infrastructure, and wherein the associated KPI fulfills the intent if the associated KPI meets or exceeds the KPI_t of the intent.

12. The method (200) of any one of claims 1 to 11, wherein the method (200) further comprises the step of providing (202) data for at least two child processes (104), wherein the provided data is at least one of: a historical data of the wireless telecommunications infrastructure; resources of the wireless telecommunications infrastructure; configurations of the wireless telecommunications infrastructure; and a simulated data of the wireless telecommunications infrastructure.

13. The method (200) of any one of claims 1 to 12, wherein the predicted (204) reward value comprises at least one reward value for each of the at least one action at a certain time.

14. The method (200) of any one of claims 1 to 13, wherein the performing of the at least two child process (104) predicts (204) the reward values periodically and / or upon a triggering event.

15. The method (200) of any one of claims 1 to 14, wherein the predicted (204) reward value at the certain time, is saved, wherein the saved predicted (204) reward value at the certain time is provided (202) data for at least two child processes (104).

16. The method (200) of any one of claims 1 to 15, wherein the method (200) further comprises the step of: combining (206) actions of the child processes (104) and the corresponding reward values.

17. The method (200) of any one of claims 1 to 16, wherein the method (200) further comprises the step of updating (216) at least one of: the predicted reward values for each of the at least one action; the uncertainty value of the at least one action; and a number of actions per child processes (104).

18. The method (200) of any one of claims 1 to 17, wherein the parent process (108, 110) further determines (212) a priority of the intent and / or a priority of the target KPIs, KPI_t.

19. The method (200) of any one of claims 1 to 18, wherein the parent process (108, 110) selects (210) the subset of actions based on a priority of the intent and / or a priority of the target KPIs, KPI_t.

20. The method (200) of any one of claims 1 to 19, wherein the parent process (108, 110) selects (210) the subset of actions based on resources of the wireless telecommunications infrastructure.

21. The method (200) of any one of claims 1 to 20, wherein the method (200) is performed according to at least two intents.

22. The method (200) of any one of claims 1 to 21, wherein at least one of the child processes (104) and the parent process (108, 110) is performed according to at least one sort of artificial intelligence, Al.

23. A computer program product (100) comprising program code portions for performing the steps of any one of the claims 1 to 22 when the computer program product is executed on one or more computing devices (1404; 1504), optionally stored on a computer-readable recording medium (1406; 1506).

24. A node (1400; 1500) of a wireless telecommunications infrastructure, the node (1400; 1500) comprising memory operable to store instructions and processing circuitry operable to execute the instructions, such that the node (1400; 1500) is operable to: perform at least two child processes (104), each associated to one of at least two key performance indicators, KPI, of the wireless telecommunications infrastructure, wherein each of the child processes (104) predicts (204) a reward value for each of at least one action based on a rate at which the associated KPIfulfills the intent, sampled over a random distribution of other actions taken by the child processes (104) other than the respective child process (104); perform a parent process (108, 110), wherein the parent process (108, 110)(i) determines (208), for each of the at least one action of each of the at least two child processes (104), an uncertainty value, U, based on a number of prior uses of the respective action, and(ii) selects (210) a subset of the actions based on the reward values predicted (204) by the child processes (104) and their respective uncertainty values, U, determined (208) by the parent process (108, 110); and perform (214) the selected (210) subset of the actions.

25. The node (1400; 1500) of claim 24, further operable to perform any one of the steps of any one of claims 2 to 22.

26. A node (1400; 1500) supporting a radio access network, RAN, in a wireless telecommunications infrastructure, the node (1400; 1500) being configured to: perform at least two child processes (104), each associated to one of at least two key performance indicators, KPI, of the wireless telecommunications infrastructure, wherein each of the child processes (104) predicts (204) a reward value for each of at least one action based on a rate at which the associated KPI fulfills the intent, sampled over a random distribution of other actions taken by the child processes (104) other than the respective child process (104); perform a parent process (108, 110), wherein the parent process (108, 110)(i) determines (208), for each of the at least one action of each of the at least two child processes (104), an uncertainty value, U, based on a number of prior uses of the respective action, and(ii) selects (210) a subset of the actions based on the reward values predicted (204) by the child processes (104) and their respective uncertainty values, U, determined (208) by the parent process (108, 110); and perform (214) the selected (210) subset of the actions.

27. The node (1400; 1500) of claim 26, further configured to perform the steps of any one of claim 2 to 22.

28. The node (1400; 1500) of any one of claims 24 to 27 , wherein the node is a core node of a core network, CN, of the wireless telecommunications infrastructure, or wherein the node is a network node of a radio access network, RAN, of the wireless telecommunications infrastructure, optionally a base station or a central unit of the base station.

29. A communication system (1600; 1700) including a host computer (1630; 1710) comprising: processing circuitry (1718) configured to provide user data; and a communication interface (1716) configured to forward user data to a cellular or ad hoc radio network (1610) of a wireless telecommunications infrastructure for transmission to a user equipment, UE (1691; 1692; 1730), wherein the wireless telecommunications infrastructure comprises a radio interface (1402; 1737) and processing circuitry (1404; 1738), the processing circuitry (1404; 1738) of the wireless telecommunications infrastructure being configured to execute the steps of any one of claims 1 to 22.

30. The communication system (1600; 1700) of claim 31, further including a UE (1691; 1692; 1730), wherein the UE (1691; 1692; 1730) comprises a radio interface (1402; 1737) and processing circuitry (1404; 1738), the processing circuitry (1404; 1738) of the UE (1691; 1692; 1730) being configured for radio access to the wireless telecommunications infrastructure.

31. The communication system (1600; 1700) of claim 29 or 30, wherein the wireless telecommunications infrastructure comprises one or more nodes (1400; 1500) configured to execute the steps of any one of claims 1 to 22.

32. The communication system (1600; 1700) of any one of claims 29 to 31, wherein the radio network (1610) further comprises a base station (1400; 1500; 1612; 1720), or a radio device (1691; 1692; 1730) functioning as a gateway, which is configured to communicate with the UE (1691; 1692; 1730) and / or configured to execute the steps of any one of claims 1 to 22.

33. The communication system (1600; 1700) of any one of claims 29 to 32, wherein:the processing circuitry (1718) of the host computer (1630; 1710) is configured to execute a host application (1712), thereby providing the user data; and a processing circuitry (1404; 1738) of the UE (1691; 1692; 1730) is configured to execute a client application (1732) associated with the host application (1712).

Images