A hybrid system and method for application resource configuration in cloud

Abstract

Methods and systems are presented that can automatically generate resource configuration for applications or network functions in a communications network, such as 5G. Reinforcement learning can be used to select ideal resource configurations. Certain embodiments follow a hybrid approach where an application characteristics simulator interacts with a real cloud environment to extract and estimate key workload and performance-related parameters, such as relative workload distribution, service dependency, and relative service performance metrics per unit resources. These parameters can then be used to initialize an environment simulator. Certain systems and methods embodiments can perform both the learning and the inferencing in a hybrid manner – they can build the environment simulator in a way the reflects the real cloud system.

Description

A HYBRID SYSTEM AND METHOD FOR APPLICATION RESOURCE CONFIGURATIONIN CLOUDTECHNICAL FIELD

[0001] The present disclosure generally relates to the technical field of wireless communications and more particularly to resource configuration for telecommunication networks.BACKGROUND

[0002] In a Network Function Virtualization (NFV) ecosystem, a Network Service (NS) request generally contains a set of Virtual Network Functions (VNFs) with some dependencies and Service Level Agreement (SLA) requirements. An incoming NS can be either described by a chain of VNFs, commonly referred to as a Service Function Chain (SFC), or a more general graph topology, which is known as Virtual Network Function - Forwarding Graph (VNF- FG). In cloud computing, a similar concept can be found in microservices whereby an application is composed of a set of microservices that interact with each other to implement the functionality of the application.

[0003] A good example of this is the 5G core architecture as implemented by Ericsson’s dual-mode 5G core (5GC) portfolio. Figure 2 depicts the components of a 5GC Network Functions (NFs, see Figure 1) grouped together into a number of products. The software architecture is based on cloud-native design principles. In this architecture, each NF offers one or more services to other NFs through some APIs. Various NFs communicate with each other, hence, the performance of a given NF depends on the performance of the NF that offers services to the given NF. These cloud-native NFs can be virtualized or physical NFs (VNFs and PNFs). Data layer 110 comprises Cloud Core Data-Storage Manager 115 with UDSF (Unstructred Data Storage Function) and UDR (User Data Repository). Control plane 120 comprises Cloud Core Resource Controller 121, Cloud Core Subscription Manager 122, Cloud Core Policy Controller 123, Cloud Core Exposure Server 124, Packet Core Controller 125, and Signaling Controller 126. Cloud Core Resource Controller 121 has NSSF (Network Slice Selection Function), NRF (NF RepositoryFunction), NSACF (Network Slice Admission Control Function), and NWDAF (Network Data Analytics Function). Cloud Core Subscription Manager 122 has UDM (Unified Data Management), AUSF (Authentication Service Function), HSS (Home Subscriber Server), and 5G- EIR (5G Equipment Identity Register). Cloud Core Policy Controller 123 has PCF (Policy Control Function), NWDAF, and PCRF (Policy and Charging Rules Function). Cloud Core Exposure Server 124 has NEF (Network Exposure Function) and SCEF (Service Capability Exposure Function). Packet Core Controller 125 has AMF (Access and Mobility Management Function), SMF (Session Management Function), NWDAF, MME (Mobility Management Entity), SGW-C (Serving Gateway Control), and PGW-C (Packet Gateway Control). Signaling Controller 126 has SEPP (Security Edge Protection Proxy), BSF (Binding Support Function), DRA (Diameter Routing Agent) and SCP (Service Communication Proxy).

[0004] Dynamic resource configuration is one of the key problem areas in resource allocation in both these areas. Considering SFCs, there are clear dependencies for VNFs, since the resource configuration of a VNF does not depend only on the given VNF, but also on other VNFs in the VNF-FG (e.g., neighbors / dependent / supporter VNFs). This gets even more complicated when considering general and complex VNF-FG topologies, such as split and merge topologies, which makes developing accurate solutions for resource configuration a challenging task. By dynamic resource configuration in NFV we refer to dynamically finding the capabilities needed for VNFs in a VNF-FG and identifying the number of resources needed to accommodate a VNF in an SFC or VNF-FG. This is done by taking into consideration the inter-dependencies between VNFs in a VNF-FG and the expected performance specifications. The same is true for microservices where the challenge is to determine the resources required by individual microservices in order for the whole application (or, alternatively one or more microservices) to achieve a given KPI.

[0005] Existing resource configuration solutions rely on static-model-based resource allocation or estimate the resource configurations of stand-alone VNFs or microservices. Some studies rely on dynamic resource allocation. Some of the prior art is summarized below.

[0006] Prior art can be categorized into the following categories:• Profiling-based VNF resources configuration: References [1], [7], and [8] propose profiling-based resource allocation for stand-alone VNFs. In [1], the authors take all different workload and resource configurations into account, around 5400 differentconfigurations are tested for one VNF. Hence, complete profiling for one VNF took 45 hours. Similarly, in [8], the authors propose a stand-alone VNF profiling framework. They also proposed a sampling heuristic to help in selecting workload and resource configuration to test. In [7], the authors discuss VNF chains profiling, where chain performance estimation is derived from stand-alone VNF profiling. And then, online monitored data of the chain is used to adjust the estimated performance model.• ML-based VNF resources configuration: Authors in [2], [5], and [6] propose ML-based techniques for VNF resource configuration. In [2], the authors estimate VNFs needs in terms of CPU as a function of the traffic they will process using the Support Vector Regression (SVR) based approach. Similarly, in [5], the authors predict the future traffic demand of VNFs. Their goal is to dynamically scale the VNF resources proactively. Both [2] and [5] consider stand-alone VNFs. In [6], the authors predict VNF resources using ML approach. However, the SFC is taken into consideration in this work, where at each VNF, the resource information of all other VNS in the SFC is also collected.• Reinforcement Learning-based resources configuration in the cloud: Authors in [4],

[0010] ,

[0011] , and

[0012] propose a single-agent RL approach for resources allocation in the cloud, while in [9], authors propose a multi-agent reinforcement learning approach. Only authors in [4] consider the inter-dependency between a VNF and its neighbors. However, the solution can only predict the future scaling decision, and it provides high level architecture without explaining the details and specifications of the different methods proposed. Also, their proposal is an online solution. In [9], the authors propose a multi-agent model using Q-learning to optimize resource allocation in the cloud with the aim to improve fault tolerance, energy consumption, and load balancing. However, they consider independent tasks of an application without explicitly assuming a graph with dependencies and relationships between application components.

[0007] Another previous approach by the Applicant, PCT / IB2022 / 058976, proposed a system and a method that can automatically generate resource configuration for workloads. The solution follows an RL-based approach. It gets as input a VNF-FG, the target or the expected performance / KPI of VNFs in the VNF-FG, and the dependency relationship betweenthe VNFs. It then identifies the number of resources needed to accommodate VNFs in a VNF-FG. This is done by taking into consideration the inter-dependencies between the VNFs in a VNF-FG and the heterogeneity of the current infrastructures (e.g., accelerators). The system can achieve this by training it with different target performances and different applications. However, this previous work relied on a simulator to avoid the overhead associated with trial and error. In such cases, the simulator can only estimate the characteristic of the application for a given application performance and exchanged workload, which results in service dependency obtained from existing profiling data. Hence, the accuracy of the generated solution degrades if the profiling data does not reflect the current situation of the system when generating resource configuration (e.g., rush hours to normal hours, daily change in traffic and workload, etc.).

[0008] There currently exist certain challenges. Limitations of the existing solutions for automated resource configuration include:• Most of the works do not consider the dependencies between VNFs when generating the resources configuration of a VNF.• Most of the works relying on the VNF profiling approach take into account every possible configuration for VNFs to generate a resource configuration for the expected performance specification. This leads to a very long profiling time. In addition, profiling-based approaches do not always reflect the exact situation of the system, e.g., rush hours to normal hours, daily changes in traffic, etc.• Existing ML-based approaches for resource configuration rely heavily on existing training data, which is not always available. In addition, they barely consider the interdependency between VNFs.• Existing RL-based approaches consider generating resource configuration for a specific application / VNF-FG with a specific performance requirement given as input and they do not generalize their solutions to generate a resource configuration for any application with any target performance.• Existing RL solutions propose an online learning solution by iteratively collecting experience by interacting with the environment. In many settings, this sort of online interaction can be impractical either because data collection is expensive or dangerous. Even in domains where online interaction is feasible, it will still be challenging if thedomain is complex and effective generalization requires large datasets. In addition, online interaction, requires and consumes a lot of resources which will impose additional monetary cost.• The Applicant’s previous work, PCT / IB2022 / 058976, generates resource configurations by relying on a simulator which trains the model in an offline manner using profiling data. There is no feedback from the real system. If the profiling data doesn’t reflect the actual situation of the system, then the model will not be able to generate accurate resource configuration.SUMMARY

[0009] One embodiment under the present disclosure comprises a computer implemented method for performing automated resource configuration. The method steps include, receiving a NS request, the NS request comprising one or more VNFs, and one or more related SLA requirements, wherein the NS request is described by a VNF-FG; receiving a target KPI, for the VNF-FG; and estimating one or more performance measurements of each of the one or more VNFs per unit of one or more resources. The method further includes estimating one or more weights of dependencies between the one or more VNFs for performance evaluation; estimating one or more target key performance indicators, KPIs, of each of the one or more VNFs based at least in part on the target KPI for the VNF-FG; and generating one or more resource configurations for the VNF-FG.

[0010] Another embodiment comprises a system for performing automated resource configuration comprising processing circuitry and a memory. The memory contains instructions executable by the processing circuitry whereby the system / apparatus is operative to: receive a NS request, the NS request comprising one or more VNFs, and one or more related SLA requirements, wherein the NS request is described by a VNF-FG; and receive a target KPI, for the VNF-FG. The instructions are further operable to estimate one or more performance measurements of each of the one or more VNFs per unit of one or more resources; estimate one or more weights of dependencies between the one or more VNFs for performance evaluation; estimate one or more target KPIs of each of the one or more VNFs based at least in part on the target KPI for the VNF- FG; and generate one or more resource configurations for the VNF-FG.

[0011] Another embodiment comprises a computer implemented method for performing automated resource configuration. The method steps include deploying a VNF-FG in a cloud environment, wherein the VNF-FG comprises one or more VNFs, and one or more related SLA requirements; and monitoring the cloud environment. Steps further include extracting or estimating one or more performance parameters from the cloud environment; initializing an environment simulator with at least one of the one or more performance parameters; training a RL agent with the one or more performance parameters; and saving the trained RL agent.

[0012] Another embodiment comprises a system for performing automated resource configuration comprising processing circuitry and a memory. The memory contains instructions executable by the processing circuitry whereby the system / apparatus is operative to: deploy a VNF-FG in a cloud environment, wherein the VNF-FG comprises one or more VNFs, and one or more related SLA requirements; and monitor the cloud environment. The instructions are further executable to extract or estimating one or more performance parameters from the cloud environment; initialize an environment simulator with at least one of the one or more performance parameters; train a RL agent with the one or more performance parameters; and save the trained RL agent.

[0013] Another embodiment under the present disclosure comprises a computer implemented method for obtaining resource configurations. The method steps include receiving a target KPI for a VNF-FG, wherein the VNF-FG comprises one or more VNFs to be implemented in a cloud environment; receiving one or more saved performance parameters from the cloud environment; and initializing an environment simulator with the one or more saved performance parameters. The steps further include using a trained RL agent to identify one or more resource configurations to meet the target KPI; applying the one or more resource configurations in the cloud environment; checking whether the one or more resource configurations meet the target KPI for the VNF-FG; and obtaining a dataset of resource configurations identified by the trained RL agent that meet the target KPI.

[0014] Another embodiment under the present disclosure comprises a system for obtaining resource configurations comprising processing circuitry and a memory. The memory contains instructions executable by the processing circuitry whereby the system / apparatus is operative to: receive a target KPI for a VNF-FG, wherein the VNF-FG comprises one or more VNFs to be implemented in a cloud environment; receive one or more saved performanceparameters from the cloud environment; and initialize an environment simulator with the one or more saved performance parameters. The instructions are further executable to use a trained RL agent to identify one or more resource configurations to meet the target KPI; apply the one or more resource configurations in the cloud environment; check whether the one or more resource configurations meet the target KPI for the VNF-FG; and obtain a dataset of resource configurations identified by the trained RL agent that meet the target KPI.

[0015] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an indication of the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0016] For a more complete understanding of the present disclosure, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

[0017] Fig. 1 illustrates an example of 5G Core network architecture;

[0018] Fig. 2 illustrates an example of Ericsson’s dual-mode 5G Core architecture;

[0019] Fig. 3 illustrates a system embodiment of a hybrid resource configuration generation device under the present disclosure;

[0020] Fig. 4 illustrates a flow chart of a reinforcement learning embodiment under the present disclosure;

[0021] Fig. 5 illustrates a flow-chart of a method embodiment under the present disclosure;

[0022] Fig. 6 illustrates a flow-chart of a method embodiment under the present disclosure;

[0023] Fig. 7 illustrates a flow-chart of a method embodiment under the present disclosure;

[0024] Fig. 8 illustrates a flow-chart of a method embodiment under the present disclosure;

[0025] Fig. 9 illustrates a flow-chart of a method embodiment under the present disclosure;

[0026] Fig. 10 illustrates a flow-chart of a method embodiment under the present disclosure;

[0027] Fig. 11 illustrates an embodiment of a computing device under the present disclosure;

[0028] Fig. 12 shows a schematic of a communication system embodiment under the present disclosure;

[0029] Fig. 13 shows a schematic of a user equipment embodiment under the present disclosure;

[0030] Fig. 14 shows a schematic of a network node embodiment under the present disclosure;

[0031] Fig. 15 shows a schematic of a host embodiment under the present disclosure;

[0032] Fig. 16 shows a schematic of a virtualization environment embodiment under the present disclosure; and

[0033] Fig. 17 shows a schematic representation of an embodiment of communication amongst nodes, hosts, and user equipment under the present disclosure.DETAILED DESCRIPTION

[0034] Before describing various embodiments of the present disclosure in detail, it is to be understood that this disclosure is not limited to the parameters of the particularly exemplified systems, methods, apparatus, products, processes, and / or kits, which may, of course, vary. Thus, while certain embodiments of the present disclosure will be described in detail, with reference to specific configurations, parameters, components, elements, etc., the descriptions are illustrative and are not to be construed as limiting the scope of the claimed embodiments. In addition, the terminology used herein is for the purpose of describing the embodiments and is not necessarily intended to limit the scope of the claimed embodiments.

[0035] Certain aspects of the disclosure and their embodiments may provide solutions to the challenges identified above in the area of automated resource configuration. Forexample, the ability to automate the generation of the resource configuration for generic VNF-FGs or microservice applications in a hybrid manner (offline and online) is a key step to solving some problems in this technical field. Various embodiments are presented below for the case of VNFs, however, the teachings and described solutions are also applicable in a straightforward way to microservice-based applications. Certain embodiments include systems and methods that can automatically generate resource configuration for applications processing a given workload. Certain embodiments follow a hybrid approach where an application characteristics simulator interacts with a real cloud environment to extract and estimate key workload and performance- related parameters, such as relative workload distribution, service dependency, and relative service performance metrics per unit resources. These parameters can then be used to initialize an environment simulator. Certain systems and methods embodiments can perform both the learning and the inferencing in a hybrid manner - they can build the environment simulator in a way the reflects the real cloud system.

[0036] Certain proposed embodiments can map the performance specification in the SLA to an amount of resources in the infrastructure to ensure that the cloud infrastructure or the NFV infrastructure (NFVI) resources are utilized efficiently, and the target performance of the application represented by VNF-FG is met. Capabilities include:• estimating the performance of each VNF / microservice per unit resource;• estimating the weights of dependencies between VNFs / microservices for performance evaluation;• estimating the target KPI of each VNF / microservice given a target KPI for the entire VNF-FG;• generating resource configuration for a VNF-FG with any target performance / KPI;• generating resource configuration for a VNF-FG with any dependency relationship between VNFs; and• generating resource configuration for a VNF-FG with any workload characteristics (e.g., performance per resource unit).

[0037] Certain embodiments may provide one or more of the following technical advantages:• an automated solution to generate resource configuration for VNFs in a VNF-FG;• can handle any application (or any VNF-FG) with any expected performance / KPI, even when the workload characteristic changes - it provides a generic solution that can: a. generate a resource configuration for various expected / target performances / KPIs; b. generate a resource configuration for large number of VNF-FGs with various dependencies; c. generate a resource configuration for large range of workload characteristics;• can be applied to assist in the optimal placement and performance of VNFs;• can be applied to already deployed VNFs when a resource reconfiguration (e.g., scaling) is needed.

[0038] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.Descriptive Use Case

[0039] The 5G core (see Figure 1) includes many NFs that communicate with each other through various interfaces for different procedures. Different procedures trigger different NFs. Understanding these interactions and interdependencies among the NFs is important to achieve efficient resource configuration. One example of a procedure involved in 5GC is the registration procedure of the UE. The registration procedure is initiated from the UE and communicated to the 5GC through the 5G base station (gNB). When the registration request is received by the AMF (Access and Mobility Management Function) at the 5GC, it initiates the default PDU (Protocol Data Unit) session setup. At this stage, different 5GC NFs are involved to handle the registration request, including SMF (Session Management Function), PCF (Policy Control Function), and AUSF (Authentication Service Function). The AMF authenticates the UE after obtaining the keys from the AUSF and then, obtains subscription data from the UDM (Unified Data Management). It then creates a policy-association with the PCF. The PCF registers with the AMF so that it can be notified on events like location change and communication failure. The AMF then updates the SMF context and sends an initial context setup request to activate the default PDU session. The SMF assigns an IP address and the tunnel id to be used for sending uplink data and selects the UPF (User Plane Function) to be used for the session. The AMF notifies the SMFwhen the session is ready for uplink and downlink data transfer. Once the session is ready, the gNB enables security between the UE and itself and activates the default PDU session. At this point, downlink and uplink data streams are ready to flow between the UE and the Internet. Similar procedures are followed for other tasks, such as service requests, mobility management as well as session management.

[0040] When deploying a 5GC, there are often requirements specified in terms of minimum rate of requests to be handled per unit time such that the average latency of individual requests is below a specific threshold. For a given set of requirements, we need to understand how much resource to allocate to individual NFs. If the resources of an individual NF introduce a bottleneck, this can degrade the entire 5GC’s performance. Hence, a key challenge is to know how much resources to be allocated to individual NFs while considering the interdependencies between the NFs. The purpose of resource configuration is hence to ensure that the NFs within 5GC are allocated the correct amount of resource in order to achieve the expected performance with respect to the various operation requirements.System Overview

[0041] Figure 3 shows one view of a system embodiment under the present disclosure. System 200 comprises a hybrid resource configuration generation component 230, coupled to, or communicatively coupled to, cloud environment 210 and environment simulator 220. System 200 can take as input the service request or the VNF-FG, including the instances of VNFs in the VNF-FG. Figure 1, described above, shows an example of input which depicts the NFs in 5GC and their forwarding graph. System 200 can also take as input the performance expectation (or target KPI) of the VNF-FG. This can be e.g., the number of requests a VNF-FG must support, the number of subscribers, the expected delay, or the expected throughput. For the 5GC example, this can be the aggregated throughput considering supported procedures, which will result in a given average request latency for each type of procedure. (Determination of latencies given an aggregated throughput is outside the scope of this invention.)

[0042] System 200 can take the specified input, model the decision-making problem as a set of cooperating Markov Decision Processes (MDPs) or MDP variants (e.g., Markov Games, Partially Observable Markov Decision Process (POMDP), etc.) and adopt RL to solve the MDPs by training an RL agent on the MDP models to generate a model that maps thetarget performance to resource usage amount. The proposed system does that in a hybrid manner, where it interacts with an actual cloud environment 210 (e.g., Kubernetes cloud where the 5GC is deployed) and extracts workload and performance related parameters (e.g., relative workload distribution, service dependency relationships, relative service performance, etc.) by which a simulated environment is instantiated in environment simulator 220. The output is a solution for the resource configuration of each VNF in the VNF-FG. Resource configuration refers to the amount of resources needed by each VNF to achieve the target or the expected performance.MDP Modeling

[0043] In certain embodiments, the problem can be modeled as an MDP, which is mathematical framework for modeling decision-making problems. In other embodiments, it can be modeled as Markov Games where each agent has its own set of actions.

[0044] An MDP is defined by a tuple (S, A, p, r) where S is a finite set of states, A is a finite set of actions, p is a transition probability from state s to state s’ after action a is executed, and r is the immediate reward obtained after action a is performed. We denote it as a “policy” which is a mapping from a state to action. The goal of an MDP is to find an optimal policy by training agents to observe the state of the environment and take actions to ultimately maximize the reward function. We define our state space, action space, and reward function as follows.

[0045] State Space: considering we have n VNFs in a VNF-FG, in one embodiment, the state space can be composed of the current resources configuration (res) of each VNF, the current performance of each VNF (p), and the dependency relationship with other VNFs in the VNF-FG, represented by (d). The state then can be represented by:For our 5GC example in Figure 1, consider NF1 to be AMF 44. Then, resl could be the number of CPU cores allocated to each AMF 44, performance pl could be the average throughput of AMF 44 in terms of completed requests per second. Finally, for all procedures, if 20% of requests go to PCF 16 (NF2), 20% to AUSF 33 (NF3) and 10% to UDM 7 (NF4), 40% to SMF 35 (NF5), and it takes care of the remaining 10%, then dn=l 0%, di2=20%, di3=20% and so on.

[0046] Action Space: selecting an action means selecting or changing a resource configuration for VNFs and moving the system to a new state space. In one embodiment, an actioncan indicate increasing / decreasing a resource of a VNF by x% with predefined minimum and maximum values, or not changing the resource value. Each action causes the state to change to a new state. For the above embodiment, we can have a joint action, which is composed of the actions to be applied to each VNF in the VNF-FG.A = [a , a2, ... an], (2)In one embodiment, an action, a, on NF1 I(AMF) can indicate increase or decrease the resources AMF 44 by one of the following pre-defined scaling factors: [0.5, 1, 2], Here, 0.5 means decrease the resources by half (i.e., decrease from 3 CPU cores to 1.5), 1 means keep the current resource configuration, and 2 means double the resources (i.e., increase from 3 to 6 CPU cores).

[0047] Reward Function: the reward function is used to guide the RL agent(s) to find the optimal resource configuration. The reward is calculated in terms of the selected resource configuration and the given target performance, which can be seen as the objective function. The target performance part can be optimizing latency, throughput, or packet loss, for instance. The action leading to a better objective function is associated with a larger reward. In RL, the longterm cumulative reward is the optimization target of RL. And the immediate rewards are the VNFs performance feedback on the resulted new resource configuration.

[0048] In one embodiment that is detailed below the performance of individual NFs can be measured by a score where it is assumed that increase in performance is a good thing (for our example, this is the case, i.e., increasing throughput is good. However, if target KPI is specified in terms of e.g., latency, then increase in latency is bad).Scorevnf. = resource_amount * performance_per_resource_un.it, where performance_per_resource_unit indicates how much the performance improves for a given unit of resource. For instance, if increasing the CPU cores allocated to AMF 44 by one increases its throughput by 100, then the performance_per_resource_unit value will become 100. This value is estimated from the application deployed in cloud environment 210.

[0049] Then the global performance of a VNF considering its dependent VNFs is measured by:where dependents! indicates the VNFs in the VNF-FG that the current vnfi depends on, and a and P reflect the relative service dependency. Meaning, a indicates how much the global performanceof vnfi depends on itself and 0j indicates how much the global performance of vnfi depends on vnfj. These parameters are estimated from the cloud environment 210 by measuring the service dependency relationship. For the example outlined above for the 5GC 10 of Figure 1, where the target KPI is throughput, alpha and beta correspond to the dependency values defined above such that alpha=dn and betaj = dij.

[0050] The global performance value of each VNF is normalized as follows:where targetKPIvnfi is the performance target of each VNF. It is also estimated from application deployed in cloud environment 210 by measuring the relative workload distribution.

[0051] Next, we define a value called rewardable performance for each VNF. The rewardable performance is designed such that it is maximized when the normalized performance in equation (4) above is 1. A value of 1 presents the best performance. The goal is to find the optimal resources that achieve the required performance while minimizing resource waste. A value larger than 1 for the normalized performance indicates resource over provisioning while a value less than 1 indicates that the system is not performing well. The rewardable performance can be defined as follows: rewardableperfvnf= 1 — abs(performancevn^. — 1) (5)Following the equation (5) above, if the performance^. = 0 or performance^. = 2 then the rewardableperfvnf= 0. While if thethen the rewardableperfvnf= 1.

[0052] Finally, the reward is designed for each VNF as a vector where each value reflects the performance of taking one of the actions specified in the action space. A positive reward (e.g., +1, relative improvement of performance, etc.) is given when the rewardable performance is improving, a negative value (e.g., -1, relative degradation of performance, etc.) when it is degrading, and a 0 value when it does not change from a previous to current state. For instance, considering three possible actions for each VNF, then the reward for a given VNF can be formulated by: rewardvnf. = [r1(r2, r3] . (6)Reinforcement Learning-based Approach

[0053] After modeling the problem as an MDP, an RL-based approach is incorporated to determine the optimal policy for the MDP. An RL-based approach can be used to learn the best resource configuration through trial and error, by building a self-learning “generator” that can interact with the system and tries to learn an optimized policy.

[0054] The agent interacts with the environment and explores various resource configurations for the workloads through exploration and exploitation and improves over time. It applies different actions and monitors the environment to get the performance or reward for each action taken.

[0055] Considering that modern networks are large-scale and complicated, hence, the computational complexity of the techniques becomes unmanageable. As a result, Deep RL (DRL) developed to be an alternative. In the present disclosure, certain embodiments can utilize Deep Q-Network (DQN) to solve the problem.

[0056] Figure 4 shows one embodiment of how one embodiment of a developed RL-based approach using DQN works. The environment simulator 320 can be developed using different available simulators, such as OpenAI Gym

[0013] , The environment simulator 320 is instantiated with workload and performance related parameters from the actual cloud system, where we deployed a microservice-based application in cloud environment (e.g., Kubernetes) and monitored it to extract and estimate the needed parameters that are used to initialize the environment simulator. There is a Neural Network (NN) 340 for deep Q learning which generalizes from previously experienced states to new states through function approximation. It basically approximates Q values 350 for unmet states and actions using stochastic gradient descent function approximation, the Q values 350 measure the expected return or discounted sum of rewards obtained from state s by taking action a and following policy it thereafter. The optimal Q value follows the Bellman optimality equation:Q(st, at) <- Q(st, at) + a[(r(st, at) + y. max{Q(st+1, at+i)}) - <2 Ot, «t)] )

[0057] The state is given as input and the Q values 350 of all possible actions is generated as output. The input state 330 of the DNN consists of the current resources of a VNF, their normalized performance (as explained above regarding Figure 3), and the dependency relationships. The experience replay is a memory 310, which stores the agent experiences. The DNN is trained by selecting a random batch from this memory buffer to form a learning batch(These batches are used to feed the DQN and update the estimated Q value). The actions are selected at step 325 using the modified epsilon greedy policy. Traditional s-greedy policy is defined as follows: a. With probability s, we select a random action a; b. With probability 1 -s, we select an action that has a maximum value.

[0058] Certain embodiments under the present disclosure can use a modified s- greedy algorithm. In the existing s-greedy algorithm, at each episode all actions are selected either randomly (i.e., exploration) or using prior knowledge (i.e., exploitation). For certain embodiments of the present disclosure, randomness and exploitation can be selected per VNF. meaning that for a graph of VNFs, in a single episode, for some VNFs the agent takes random actions where for some others an action is selected by exploitation. For the exploration part, the random selection of actions, for each VNF an action is not selected completely randomly. Instead, the predicted Q values can be used to calculate the probability distribution of each action. Actions can then be randomly chosen considering their probabilities.

[0059] One of the differences with traditional DQN and the approach of certain embodiments is that in traditional DQN, a Q value out of all possible Q values that maximizes the reward function is selected. However, in the current case, there are n VNFs, so the output of the DNN; the Q values, are grouped into n groups, and the Q value of each group is selected to maximize the performance of its corresponding VNF.Hybrid Learning Phase

[0060] Figure 5 describes several steps followed in a proposed embodiment for the learning phase. The learning phase is done in a hybrid manner with two major phases.

[0061] In the first phase (steps 510, 515, 520), the embodiment deploys the application (represented as a VNF-FG) in a cloud environment 545 (e.g., Kubernetes) at step 510. It monitors, at step 515, the resources consumed by the VNFs from the hosting (physical / virtual) nodes and the performance of each VNF (or microservice) over some time period. By monitoring the resources and the performance of the application, it extracts some workload and performance related parameters at step 520. In a second phase, these parameters are then used to initialize the environment simulator at step 525. For example, for the 5GC example, this means deploying it in a cloud environment for instance and measuring the performance of its NFs in terms of throughput.

[0062] A key challenge in simulating NF performance is to capture the relationship between the resource allocated to an NF and its performance. In certain embodiments this relationship is modeled to be a piece-wise-linear. Specifically, the relationship is assumed to be linear for resource configurations within the scaling factors (such as described with regard to MDPs above) of the current value and this is called the performance per resource (ppr). This value is estimated by getting the average change in throughput and the average change in resource usage for each VNF. i.e., performance_per_resource_unit(vnfi) = throughput vnf ) / resource(vnfj (8)

[0063] The relative service dependency and the relative workload distribution depends highly on the workload mix that is currently being served by the application. For the 5GC example, depending on what fraction of the requests are registration requests and what fraction service requests, the dependency values (defined in section above) will change over time. These values need to be estimated by monitoring the load exchanged between the NFs. The following describes the situation where the target KPI is throughput.

[0064] The relative service dependency represents the weights or the percentage of how much a VNF’s performance depends on itself and on other dependent VNFs (a and 0 defined in the MDP model). For instance, considering that requests to a VNF are either processed by the VNF itself or forwarded to its dependents, then a and 0 can be calculated as:where “throughput” represents the average throughput over some period time.

[0065] The relative workload distribution represents the target KPI of each VNF. For a given VNF, it can be calculated by measuring the average throughput of the application (or the entry VNF if one such VNF exists) and a given VNFi as:

[0066] In the second phase (steps 525, 530, 535), the environment simulator is initialized. The simulator needs some data points to start with. The estimations generated in the first phase (steps 510, 515, 520) are used to initialize the environment simulator in step 525. The simulator basically can give the estimate of the performance of the application or network servicegiven the resource allocations to all NFs. For the 5GC example, this would mean that given the resources allocated to each of the NFs (such as AMF 44 and SMF 45 of Figure 1), it will be able to estimate what the average throughput will be for the 5GC. After initializing the simulator, the training of deep reinforcement learning agents starts at step 530 (further embodiments of this step are given below). Once the RL agent is trained and learned, the agent and the model are saved to be used later for inferencing at step 535.Training RL Agents

[0067] Figure 6 describes the steps of one embodiment of a proposed solution for training an RL agent to automatically generate resource configuration for any random target performance / KPI, any random dependency between VNFs (relative service dependency), and any random performance per resource (workload characteristics). These random numbers are generated within a range. The range of each of these parameters can be defined following different methods. One simple method can be dividing and multiplying the parameter by a factor e.g., by 10. Certain embodiments use Deep Reinforcement Learning to train the agents.

[0068] At step 705, a number of episodes is defined to train the agent for, and the process starts. An episode is all the states that come in between an initial state and a terminal state.

[0069] At step 720, at the beginning of each episode, the model is checked whether it is accurate enough for any random target performance. This can be checked, for instance, by checking the number of iterations it took to reach the target performance. If the model is not accurate enough for any target performance, a new random target performance is generated at step 715, and the same steps 750-780 are repeated. Step 715 can comprise, at the beginning of each episode, the agent generating a random target performance. This is the expected performance of the applications or the representative VNF-FGs. The expected performance can be defined randomly within a pre-specified range. This range is specified by extracting and estimating parameters from a real cloud system as explained above.

[0070] If the model is accurate enough for a random target performance, then the process goes to step 725, where it is determined if the model is accurate enough for a random dependency. If not, the process goes to step 740, where a random dependency(ies) is generated, and the same steps 750-780 are repeated. The generation of the dependencies is done such that theoriginal dependencies defined in the VNF-FGs are prioritized by giving higher probability, while lower probability is given for dependencies that are not in the VNF-FGs.

[0071] If the model is accurate enough for the random dependency, then at step 730, it is determined if the model is accurate enough for a random performance per resource. If it is, then the process ends at step 735. If not, then at step 745 a random performance(s) per resource is generated, and the same steps 750-780 are repeated. The generation of a random performance per resource parameter is done within the specified range generated during the learning phase.

[0072] At step 750, a number of iterations inside each episode is defined. Each iteration represents a frame within the episode.

[0073] At step 755, an action is selected for each VNF in the VNF-FG. The action is selected from an action list, such as defined above in regard to Figure 6. An action may indicate to change the resources usage of a VNF (e.g., reduce the current resources by 50%). Some embodiments can use constraints on the actions, meaning that a resource usage of a VNF cannot be increased over a specified threshold. This indicates the maximum allowed resources for a VNF and can be given as input. The selection of the action can be done using different policies. One example is the epsilon greedy policy. Here, a modified epsilon greedy policy is used, as described above in paragraph

[0058] , but the regular epsilon greedy policy could be used also. This policy consists of exploration and exploitation. It is defined as follows: with probability s, a random action a is selected; with probability 1 -s, an action that has a maximum value is selected.

[0074] At step 760, after selecting an action and applying it to the environment, the system moves to a new state. As a reminder, the state indicates the current resources configuration of a VNF, its current performance, and the inter-dependencies between VNFs.

[0075] At step 765, a reward function for each VNF is calculated, e.g., according to the equation (6) defined above.

[0076] At step 770, at the end of each iteration, the agent checks whether the target performance is met or not. The target performance is considered to be met when the performance of all VNFs in the VNF-FG is at the generated target performance of each VNF at step 715. If the target performance is reached, the agent breaks the iterations and moves to the next episode at step 710. If the target performance has not yet been reached, then the agent goes to step 775. At step 775, the selection of action and moving to a new state is continued, returning to step 750, until the defined number of iterations is reached.

[0077] At step 780, when the number of iterations is finished the agent goes to the next episode, returning to step 710. If there are no more episodes, the process ends at 785.

[0078] Once the episodes are finished, an agent will have been obtained for each VNF that is capable to find the best action from any state to reach 100% of the target performance.

[0079] In another embodiment, steps 720, 725, and 730 can be swapped. For instance, the model can be first trained for a specific target performance and performance per resource considering different dependencies. And then, once the model is accurate enough for any dependency, a new random target performance or performance per resource is generated to continue training the model.Inferencing Phase

[0080] Figure 7 describes the steps of one embodiment method 900 for an inferencing phase. The inferencing phase can also be done in a hybrid and iterative manner. The iteration can be particularly beneficial for two reasons. First, recall that the relationship between resource and NF performance is approximated by a linear function with slope of ppr. But since this relationship is generally non-linear, it means that the estimated target KPI (e.g., target throughput for a 5GC deployment) may not be accurate. In this situation, the iterative step allows an embodiment to zero-in to the correct resource configuration (and hence target KPI) by the successive feedback received from the cloud system. Secondly, the workload characteristics will also generally change over time. For the 5GC example, the fraction of mobility / handover procedures handled by the 5GC may increase in the morning and decline in the evening, which translates to changes to the relative workload distribution parameters. This requires changes in resource configurations of the NFs, which in turn can cause an estimate of the ppr to be erroneous and hence require more iterations. Proposed embodiments can receive a target KPI (or expected performance) for the application (or the VNF-FG). Its responsibility is to generate resource configurations for the VNFs in the VNF-FG such that the application meets the target KPI. The steps of method 900 are described in detail below.

[0081] After starting at 905, the embodiment receives at step 910 a target KPI (or expected performance) for the application (or the VNF-FG). Its responsibility is to generate resources configuration for the VNFs in the VNF-FG such that the application meets the target KPI.

[0082] At step 915, it then gets the last saved workload and performance-related parameters obtained from the cloud environment. Alternatively, these parameters can also be obtained directly from the cloud. At step 920, the environment simulator 945 is initialized with these parameters. At step 925, the learned and trained RL agent are called to generate the best resource configuration to meet the target KPI. At step 930, once the resources are generated, they can be applied on the cloud environment 950. If it is a new application, then the application is deployed in cloud environment 950 with the generated resource configuration. If the application is already deployed, then the applications’ VNFs' resources are changed following the generated resources with the goal to meet the new target KPI.

[0083] At step 935, the embodiment then checks whether the application meets the target KPI. If yes, then the process ends at step 960. This step may involve waiting for a predetermined amount of time to ascertain that the behavior of the application has stabilized following the change in configuration. At step 940, if the target KPI is not met, then the solution extracts the latest workload and performance related parameters from the cloud environment 950, including new values for ppr.

[0084] In step 955, the embodiment adjusts the parameters extracted from cloud environment 950 such that they ensure convergence of generated configurations towards an optimal one. This can involve, for instance, verifying that certain characteristic, such as performance per resource, are convex and that the changes in configuration do not, for instance, oscillate around a target point. If oscillation or divergence of configuration is detected, then the solution will raise an alarm and exit. If not, the solution goes back to step 920. At this stage, the environment simulator is updated with the latest workload and performance-related parameters obtained from the cloud environment 950 and the same steps are repeated until the target KPI is met.Additional Embodiments

[0085] Another possible method embodiment under the present disclosure is shown in Figure 8. Method 1100 comprises a method of performing automated resource configuration. Step 1110 is receiving a NS request, the NS request comprising one or more VNFs and one or more related SLA requirements, wherein the NS request is described by a VNF-FG. Step 1120 is receiving a target KPI, for the VNF-FG. Step 1130 is estimating one or more performancemeasurements of each of the one or more VNFs per unit of one or more resources. Step 1140 is estimating one or more weights of dependencies between the one or more VNFs for performance evaluation. Step 1150 is estimating one or more target key performance indicators, KPIs, of each of the one or more VNFs based at least in part on the target KPI for the VNF-FG. Step 1160 is generating one or more resource configurations for the VNF-FG.

[0086] Another embodiment possible method embodiment under the present disclosure is shown in Figure 9. Method 1300 comprises a method for performing automated resource configuration, the method comprising. Step 1310 is deploying a VNF-FG in a cloud environment, wherein the VNF-FG comprises one or more VNFs, and one or more related SLA requirements. Step 1320 is monitoring the cloud environment. Step 1330 is extracting or estimating one or more performance parameters from the cloud environment. Step 1340 is initializing an environment simulator with at least one of the one or more performance parameters. Step 1350 is training an RL agent with the one or more performance parameters. Step 1360 is saving the trained RL agent.

[0087] Another embodiment possible method embodiment under the present disclosure is shown in Figure 10. Method 1500 comprises a method for performing automated resource configuration. Step 1510 is receiving a target KPI for a VNF-FG, wherein the VNF-FG comprises one or more VNFs to be implemented in a cloud environment. Step 1520 is receiving one or more saved performance parameters from the cloud environment. Step 1530 is initializing an environment simulator with the one or more saved performance parameters. Step 1540 is using a trained RL agent to identify one or more resource configurations to meet the target KPI. Step 1550 is applying the one or more resource configurations in the cloud environment. Step 1560 is checking whether the one or more resource configurations meet the target KPI for the VNF-FG. Step 1570 is obtaining a dataset of resource configurations identified by the trained RL agent that meet the target KPI.

[0088] Methods 500, 700, 900, 1100, 1300, and 1500 can comprise a variety of additional, optional, and / or alternative steps. These methods can be computer-implemented, and / or performed (at least partially) by a network node, UE, or portion(s) of a 5GC architecture, such as shown in Figure 1. Performance of these methods can be accomplished across multiple devices or nodes that share processing power or capabilities.

[0089] Figure 11 shows a possible embodiment of a hardware or computing device comprising, e.g., hybrid resource configuration generation device 230 of Figure 3. Device 1700 can optionally also comprise cloud environment 210 and environment simulator 220 of Figure 3. Alternatively, cloud environment 210 and environment simulator 220 can be remote. Computing device 1700 can comprise processing circuitry 1710, memory 1720, network interface 1730, non- transitory storage 1740, and power source 1750. Computing device 1700 can comprise one or more of UE 2200, network node 3300, host 4400, and / or virtualization environment 5500, embodiments of which are described further below.

[0090] Processing circuitry 1710 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine- readable computer programs in the memory 1720. The processing circuitry 1710 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 1710 may include multiple central processing units (CPUs). Processing circuitry 1710 can be operatively coupled via a bus to an input / output interface (not shown), a power source 1750, a memory 1710, a network interface 1730, non-transitory storage 1740, and / or any other component, or any combination thereof. Certain computing devices 1700 may utilize all or a subset of the components shown in Figure 11. The level of integration between the components may vary from one embodiment to another. Further, certain computing devices may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0091] Memory 1720 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable readonly memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, memory 1720 includes one or more application programs, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data. The memory 1720 may store, for use by computing device 1700, any of a variety of various operating systems or combinations of operating systems. Thememory 1720 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, and / or others. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 1720, which may be or comprise a device-readable storage medium.

[0092] Network interface 1730 is used in wired or wireless communication of signaling and / or data between a computing device 1700 and any of a network node, access network, UE, host, virtualization environment, or other components. Network interface 1730 can comprise port(s) / terminal(s) to send and receive data, for example to and from a network over a wired connection. The network interface 1730 can include components such as radio front-end circuitry, antennas, transceivers, receivers, filters, amplifiers, and / or others. In other embodiments, the network interface 1730 may comprise different components and / or different combinations of components.

[0093] Non-transitory storage 1740 may comprise any form of volatile or nonvolatile computer-readable memory including, without limitation, persistent storage, solid-state memory, optical media, RAM, ROM, a hard disk, removable storage media, and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processing circuitry 1710.

[0094] Power source 1750 may be structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 1750 may further include power circuitry for delivering power from the power source 1750 itself, and / or an external power source, to the various parts of the computing device 1700 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 1750. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 1750 to make the power suitable for the respective components of the computing device 1700 to which power is supplied.

[0095] Figure 12 shows an example of a communication system 2100 in accordance with some embodiments. In the example, the communication system 2100 includes a telecommunication network 2102 that includes an access network 2104, such as a RAN, and a corenetwork 2106, which includes one or more core network nodes 2108. The access network 2104 includes one or more access network nodes, such as network nodes 2110a and 2110b (one or more of which may be generally referred to as network nodes 2110), or any other similar 3rd Generation Partnership Project (3GPP) access node or non-3GPP access point. The network nodes 2110 facilitate direct or indirect connection of UE, such as by connecting UEs 2112a, 2112b, 2112c, and 2112d (one or more of which may be generally referred to as UEs 2112) to the core network 2106 over one or more wireless connections.

[0096] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 1100 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 2100 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0097] The UEs 2112 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 2110 and other communication devices. Similarly, the network nodes 2110 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 2112 and / or with other network nodes or equipment in the telecommunication network 2102 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 2102.

[0098] In the depicted example, the core network 2106 connects the network nodes 2110 to one or more hosts, such as host 2116. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 2106 includes one more core network nodes (e.g., core network node 2108) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 2108. Example core network nodes include functions of oneor more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0099] The host 2116 may be under the ownership or control of a service provider other than an operator or provider of the access network 2104 and / or the telecommunication network 2102, and may be operated by the service provider or on behalf of the service provider. The host 2116 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.[000100] As a whole, the communication system 2100 of Figure 12 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z- Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.[000101] In some examples, the telecommunication network 2102 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 2102 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 2102. For example, the telecommunications network 2102 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) servicesto other UEs, and / or Massive Machine Type Communication (mMTC) / Massive loT services to yet further UEs.[000102] In some examples, the UEs 2112 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 2104 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 2104. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).[000103] In the example, the hub 2114 communicates with the access network 2104 to facilitate indirect communication between one or more UEs (e.g., UE 2112c and / or 2112d) and network nodes (e.g., network node 2110b). In some examples, the hub 2114 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 2114 may be a broadband router enabling access to the core network 2106 for the UEs. As another example, the hub 2114 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 2110, or by executable code, script, process, or other instructions in the hub 2114. As another example, the hub 2114 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 2114 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 2114 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 2114 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 2114 acts as a proxy server or orchestrator for the UEs, in particular in if one or more of the UEs are low energy loT devices.[000104] The hub 2114 may have a constant / persistent or intermittent connection to the network node 2110b. The hub 2114 may also allow for a different communication scheme and / or schedule between the hub 2114 and UEs (e.g., UE 2112c and / or 2112d), and between the hub 2114 and the core network 2106. In other examples, the hub 2114 is connected to the corenetwork 2106 and / or one or more UEs via a wired connection. Moreover, the hub 2114 may be configured to connect to an M2M service provider over the access network 1104 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 2110 while still connected via the hub 2114 via a wired or wireless connection. In some embodiments, the hub 2114 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 2110b. In other embodiments, the hub 2114 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 2110b, but which is additionally capable of operating as a communication start and / or end point for certain data channels.[000105] Figure 13 shows a UE 2200 in accordance with some embodiments. As used herein, a UE refers to a device capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle-mounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-IoT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE.[000106] A UE may support device-to-device (D2D) communication, for example by implementing a 3 GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to- everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).[000107] The UE 2200 includes processing circuitry 2202 that is operatively coupled via a bus 2204 to an input / output interface 2206, a power source 2208, a memory 2210, a communication interface 2212, and / or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in Figure 13. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.[000108] The processing circuitry 2202 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine- readable computer programs in the memory 2210. The processing circuitry 2202 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 2202 may include multiple central processing units (CPUs).[000109] In the example, the input / output interface 2206 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 2200. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presencesensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.[000110] In some embodiments, the power source 2208 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricityoutlet), photovoltaic device, or power cell, may be used. The power source 2208 may further include power circuitry for delivering power from the power source 2208 itself, and / or an external power source, to the various parts of the UE 2200 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 2208. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 2208 to make the power suitable for the respective components of the UE 2200 to which power is supplied.[000111] The memory 2210 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 2210 includes one or more application programs 2214, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 2216. The memory 2210 may store, for use by the UE 2200, any of a variety of various operating systems or combinations of operating systems.[000112] The memory 2210 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD- DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUICC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 2210 may allow the UE 2200 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 2210, which may be or comprise a device-readable storage medium.[000113] The processing circuitry 2202 may be configured to communicate with an access network or other network using the communication interface 2212. The communication interface 2212 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 2222. The communication interface 2212 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network). Each transceiver may include a transmitter 2218 and / or a receiver 2220 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 2218 and receiver 2220 may be coupled to one or more antennas (e.g., antenna 2222) and may share circuit components, software or firmware, or alternatively be implemented separately.[000114] In the illustrated embodiment, communication functions of the communication interface 2212 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / internet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.[000115] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 2212, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).[000116] As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.[000117] A UE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and / or software in dependence of the intended application of the loT device in addition to other components as described in relation to the UE 2200 shown in Figure 13.[000118] As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements, and transmits the results of such monitoring and / or measurements to another UE and / or a network node. The UE may in this case be an M2M device, which may in a 3 GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3 GPP NB-IoT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.[000119] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.[000120] Figure 14 shows a network node 3300 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with a UE and / or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)).[000121] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).[000122] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSRBSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).[000123] The network node 3300 includes a processing circuitry 3302, a memory 3304, a communication interface 3306, and a power source 3308. The network node 3300 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 3300 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 1300 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 3304 for different RATs) and some components may be reused (e.g., a same antenna 3310 may be shared by different RATs). The network node 3300 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1300, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1300.[000124] The processing circuitry 3302 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 3300 components, such as the memory 3304, to provide network node 3300 functionality.[000125] In some embodiments, the processing circuitry 3302 includes a system on a chip (SOC). In some embodiments, the processing circuitry 3302 includes one or more of radio frequency (RF) transceiver circuitry 3312 and baseband processing circuitry 3314. In some embodiments, the radio frequency (RF) transceiver circuitry 3312 and the baseband processing circuitry 3314 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 3312 and baseband processing circuitry 3314 may be on the same chip or set of chips, boards, or units.[000126] The memory 3304 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), readonly memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processing circuitry 3302. The memory 3304 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions capable of being executed by the processing circuitry 3302 and utilized by the network node 3300. The memory 3304 may be used to store any calculations made by the processing circuitry 3302 and / or any data received via the communication interface 3306. In some embodiments, the processing circuitry 3302 and memory 3304 is integrated.[000127] The communication interface 3306 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, the communication interface 3306 comprises port(s) / terminal(s) 3316 to send and receive data, for example to and from a network over a wired connection. The communication interface 3306 also includes radio front-end circuitry 3318 that may be coupled to, or in certain embodiments a part of, the antenna 3310. Radio front-end circuitry 3318 comprises filters 3320 and amplifiers 3322. The radio front-end circuitry 3318 may be connected to an antenna 3310 and processing circuitry 3302. The radio front-end circuitry may be configured to condition signals communicated between antenna 3310 and processing circuitry 3302. The radio front-end circuitry 3318 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 3318 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 3320 and / or amplifiers 3322. The radio signal may then be transmitted via the antenna 3310. Similarly, when receiving data, the antenna 3310 may collect radio signals which are then converted into digital data by the radio front-end circuitry 3318. The digital data may be passed to the processing circuitry 3302. In other embodiments, the communication interface may comprise different components and / or different combinations of components.[000128] In certain alternative embodiments, the network node 3300 does not include separate radio front-end circuitry 3318, instead, the processing circuitry 3302 includes radio frontend circuitry and is connected to the antenna 3310. Similarly, in some embodiments, all or some of the RF transceiver circuitry 3312 is part of the communication interface 3306. In still other embodiments, the communication interface 3306 includes one or more ports or terminals 3316, the radio front-end circuitry 3318, and the RF transceiver circuitry 3312, as part of a radio unit (not shown), and the communication interface 3306 communicates with the baseband processing circuitry 3314, which is part of a digital unit (not shown).[000129] The antenna 3310 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. The antenna 3310 may be coupled to the radio front-end circuitry 3318 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, the antenna 3310 is separate from the network node 3300 and connectable to the network node 3300 through an interface or port.[000130] The antenna 3310, communication interface 3306, and / or the processing circuitry 3302 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a UE, another network node and / or any other network equipment. Similarly, the antenna 3310, the communication interface 3306, and / or the processing circuitry 3302 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.[000131] The power source 3308 provides power to the various components of network node 3300 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 3308 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 3300 with power for performing the functionality described herein. For example, the network node 3300 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 3308. As a further example, the power source 3308 may comprise a source of power in the form of a battery or battery pack which is connected to, orintegrated in, power circuitry. The battery may provide backup power should the external power source fail.[000132] Embodiments of the network node 3300 may include additional components beyond those shown in Figure 14 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, the network node 3300 may include user interface equipment to allow input of information into the network node 3300 and to allow output of information from the network node 3300. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 3300.[000133] Figure 15 is a block diagram of a host 4400, which may be an embodiment of the host 2116 of Figure 12, in accordance with various aspects described herein. As used herein, the host 4400 may be or comprise various combinations hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 4400 may provide one or more services to one or more UEs.[000134] The host 4400 includes processing circuitry 4402 that is operatively coupled via a bus 4404 to an input / output interface 4406, a network interface 4408, a power source 4410, and a memory 4412. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 13 and 14, such that the descriptions thereof are generally applicable to the corresponding components of host 4400.[000135] The memory 4412 may include one or more computer programs including one or more host application programs 4414 and data 4416, which may include user data, e.g., data generated by a UE for the host 4400 or data generated by the host 4400 for a UE. Embodiments of the host 4400 may utilize only a subset or all of the components shown. The host application programs 4414 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (WC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 4414 may also provide for user authenticationand licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 4400 may select and / or indicate a different host for over-the-top services for a UE. The host application programs 4414 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.[000136] Figure 16 is a block diagram illustrating a virtualization environment 5500 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 5500 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized.[000137] Applications 5502 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment 5500 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.[000138] Hardware 5504 includes processing circuitry, memory that stores software and / or instructions executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 5506 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 5508a and 5508b (one or more of which may be generally referred to as VMs 5508), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 5506 may present a virtual operating platform that appears like networking hardware to the VMs 5508.[000139] The VMs 5508 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 5506. Different embodiments of the instance of a virtual appliance 5502 may be implemented on one or more of VMs 5508, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.[000140] In the context of NFV, a VM 5508 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each of the VMs 5508, and that part of hardware 5504 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 5508 on top of the hardware 5504 and corresponds to the application 5502.[000141] Hardware 5504 may be implemented in a standalone network node with generic or specific components. Hardware 5504 may implement some functions via virtualization. Alternatively, hardware 5504 may be part of a larger cluster of hardware (e.g., such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 5510, which, among others, oversees lifecycle management of applications 5502. In some embodiments, hardware 5504 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 5512 which may alternatively be used for communication between hardware nodes and radio units.[000142] Figure 17 shows a communication diagram of a host 6602 communicating via a network node 6604 with a UE 6606 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 2112a of Figure 12 and / or UE 2200 of Figure 13), network node (such as networknode 2110a of Figure 2 and / or network node 3300 of Figure 14), and host (such as host 2116 of Figure 12 and / or host 4400 of Figure 15) discussed in the preceding paragraphs will now be described with reference to Figure 17.[000143] Like host 4400, embodiments of host 6602 include hardware, such as a communication interface, processing circuitry, and memory. The host 6602 also includes software, which is stored in or accessible by the host 6602 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 6606 connecting via an over-the-top (OTT) connection 6650 extending between the UE 6606 and host 6602. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 6650.[000144] The network node 6604 includes hardware enabling it to communicate with the host 6602 and UE 6606. The connection 6660 may be direct or pass through a core network (like core network 2106 of Figure 12) and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.[000145] The UE 6606 includes hardware and software, which is stored in or accessible by UE 6606 and executable by the UE’s processing circuitry. The software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 6606 with the support of the host 6602. In the host 6602, an executing host application may communicate with the executing client application via the OTT connection 6650 terminating at the UE 6606 and host 6602. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 6650 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 6650.[000146] The OTT connection 6650 may extend via a connection 6660 between the host 6602 and the network node 6604 and via a wireless connection 6670 between the network node 6604 and the UE 6606 to provide the connection between the host 6602 and the UE 6606. The connection 6660 and wireless connection 6670, over which the OTT connection 6650 may be provided, have been drawn abstractly to illustrate the communication between the host 6602 andthe UE 1606 via the network node 6604, without explicit reference to any intermediary devices and the precise routing of messages via these devices.[000147] As an example of transmitting data via the OTT connection 6650, in step 6608, the host 6602 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 6606. In other embodiments, the user data is associated with a UE 6606 that shares data with the host 6602 without explicit human interaction. In step 6610, the host 6602 initiates a transmission carrying the user data towards the UE 6606. The host 6602 may initiate the transmission responsive to a request transmitted by the UE 6606. The request may be caused by human interaction with the UE 6606 or by operation of the client application executing on the UE 6606. The transmission may pass via the network node 6604, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 6612, the network node 6604 transmits to the UE 6606 the user data that was carried in the transmission that the host 6602 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 6614, the UE 6606 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 6606 associated with the host application executed by the host 6602.[000148] In some examples, the UE 6606 executes a client application which provides user data to the host 6602. The user data may be provided in reaction or response to the data received from the host 6602. Accordingly, in step 6616, the UE 6606 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input / output interface of the UE 6606. Regardless of the specific manner in which the user data was provided, the UE 6606 initiates, in step 6618, transmission of the user data towards the host 6602 via the network node 6604. In step 6620, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 6604 receives user data from the UE 6606 and initiates transmission of the received user data towards the host 6602. In step 6622, the host 6602 receives the user data carried in the transmission initiated by the UE 6606.[000149] One or more of the various embodiments improve the performance of OTT services provided to the UE 6606 using the OTT connection 6650, in which the wireless connection 6670 forms the last segment. More precisely, the teachings of these embodiments may improvethe data rate, latency, and / or power consumption and thereby provide benefits such as reduced user waiting time, relaxed restriction on file size, improved content resolution, better responsiveness, and / or extended battery lifetime.[000150] In an example scenario, factory status information may be collected and analyzed by the host 6602. As another example, the host 6602 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 6602 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 6602 may store surveillance video uploaded by a UE. As another example, the host 6602 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 6602 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and / or transmitting data.[000151] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 6650 between the host 6602 and UE 6606, in response to variations in the measurement results. The measurement procedure and / or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 6602 and / or UE 6606. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 6650 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 6650 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 6604. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 6602. The measurements may be implemented in that software causes messages to be transmitted,in particular empty or ‘dummy’ messages, using the OTT connection 6650 while monitoring propagation times, errors, etc.[000152] Although the computing devices described herein (e.g., UEs, network nodes, hosts) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and / or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.[000153] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer- readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to theprocessing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.Computer Systems of the Present Disclosure[000154] It will be appreciated that computer systems are increasingly taking a wide variety of forms. In this description and in the claims, the terms “controller,” “computer system,” or “computing system” are defined broadly as including any device or system — or combination thereof — that includes at least one physical and tangible processor and a physical and tangible memory capable of having thereon computer-executable instructions that may be executed by a processor. By way of example, not limitation, the term “computer system” or “computing system,” as used herein is intended to include personal computers, desktop computers, laptop computers, tablets, hand-held devices (e.g., mobile telephones, PDAs, pagers), microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, multi-processor systems, network PCs, distributed computing systems, datacenters, message processors, routers, switches, and even devices that conventionally have not been considered a computing system, such as wearables (e.g., glasses).[000155] The memory may take any form and may depend on the nature and form of the computing system. The memory can be physical system memory, which includes volatile memory, non-volatile memory, or some combination of the two. The term “memory” may also be used herein to refer to non-volatile mass storage such as physical storage media.[000156] The computing system also has thereon multiple structures often referred to as an “executable component.” For instance, the memory of a computing system can include an executable component. The term “executable component” is the name for a structure that is well understood to one of ordinary skill in the art in the field of computing as being a structure that can be software, hardware, or a combination thereof.[000157] For instance, when implemented in software, one of ordinary skill in the art would understand that the structure of an executable component may include software objects, routines, methods, and so forth, that may be executed by one or more processors on the computing system, whether such an executable component exists in the heap of a computing system, or whether the executable component exists on computer-readable storage media. The structure ofthe executable component exists on a computer-readable medium in such a form that it is operable, when executed by one or more processors of the computing system, to cause the computing system to perform one or more functions, such as the functions and methods described herein. Such a structure may be computer-readable directly by a processor — as is the case if the executable component were binary. Alternatively, the structure may be structured to be interpretable and / or compiled — whether in a single stage or in multiple stages — so as to generate such binary that is directly interpretable by a processor.[000158] The term “executable component” is also well understood by one of ordinary skill as including structures that are implemented exclusively or near-exclusively in hardware logic components, such as within a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), or any other specialized circuit. Accordingly, the term “executable component” is a term for a structure that is well understood by those of ordinary skill in the art of computing, whether implemented in software, hardware, or a combination thereof.[000159] The terms “component,” “service,” “engine,” “module,” “control,” “generator,” or the like may also be used in this description. As used in this description and in this case, these terms — whether expressed with or without a modifying clause — are also intended to be synonymous with the term “executable component” and thus also have a structure that is well understood by those of ordinary skill in the art of computing.[000160] In an embodiment, the communication system may include a complex of computing devices executing any of the method of the embodiments as described above and data storage devices which could be server parks and data centers.[000161] In terms of computer implementation, a computer is generally understood to comprise one or more processors or one or more controllers, and the terms computer, processor, and controller may be employed interchangeably. When provided by a computer, processor, or controller, the functions may be provided by a single dedicated computer or processor or controller, by a single shared computer or processor or controller, or by a plurality of individual computers or processors or controllers, some of which may be shared or distributed. Moreover, the term “processor” or “controller” also refers to other hardware capable of performing such functions and / or executing software, such as the example hardware recited above.[000162] In general, the various exemplary embodiments may be implemented in hardware or special purpose chips, circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor, or other computing device, although the disclosure is not limited thereto. While various aspects of the exemplary embodiments of this disclosure may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques, or methods described herein may be implemented in, as nonlimiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.[000163] While not all computing systems require a user interface, in some embodiments a computing system includes a user interface for use in communicating information from / to a user. The user interface may include output mechanisms as well as input mechanisms. The principles described herein are not limited to the precise output mechanisms or input mechanisms as such will depend on the nature of the device. However, output mechanisms might include, for instance, speakers, displays, tactile output, projections, holograms, and so forth. Examples of input mechanisms might include, for instance, microphones, touchscreens, projections, holograms, cameras, keyboards, stylus, mouse, or other pointer input, sensors of any type, and so forth.[000164] Accordingly, embodiments described herein may comprise or utilize a special purpose or general-purpose computing system. Embodiments described herein also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computing system. Computer-readable media that store computer-executable instructions are physical storage media. Computer-readable media that carry computer-executable instructions are transmission media. Thus, by way of example — not limitation — embodiments disclosed or envisioned herein can comprise at least two distinctly different kinds of computer-readable media: storage media and transmission media.[000165] Computer-readable storage media include RAM, ROM, EEPROM, solid state drives (“SSDs”), flash memory, phase-change memory (“PCM’), CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other physical andtangible storage medium that can be used to store desired program code in the form of computerexecutable instructions or data structures and that can be accessed and executed by a general purpose or special purpose computing system to implement the disclosed functionality or functionalities. For example, computer-executable instructions may be embodied on one or more computer-readable storage media to form a computer program product.[000166] Transmission media can include a network and / or data links that can be used to carry desired program code in the form of computer-executable instructions or data structures and that can be accessed and executed by a general purpose or special purpose computing system. Combinations of the above should also be included within the scope of computer-readable media.[000167] Further, upon reaching various computing system components, program code in the form of computer-executable instructions or data structures can be transferred automatically from transmission media to storage media (or vice versa). For example, computerexecutable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a “NIC”) and then eventually transferred to computing system RAM and / or to less volatile storage media at a computing system. Thus, it should be understood that storage media can be included in computing system components that also — or even primarily — utilize transmission media.[000168] Those skilled in the art will further appreciate that a computing system may also contain communication channels that allow the computing system to communicate with other computing systems over, for example, a network. Accordingly, the methods described herein may be practiced in network computing environments with many types of computing systems and computing system configurations. The disclosed methods may also be practiced in distributed system environments where local and / or remote computing systems, which are linked through a network (either by wired data links, wireless data links, or by a combination of wired and wireless data links), both perform tasks. In a distributed system environment, the processing, memory, and / or storage capability may be distributed as well.[000169] Those skilled in the art will also appreciate that the disclosed methods may be practiced in a cloud computing environment. Cloud computing environments may be distributed, although this is not required. When distributed, cloud computing environments may be distributed internationally within an organization and / or have components possessed across multiple organizations. In this description and the following claims, “cloud computing” is definedas a model for enabling on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services). The definition of “cloud computing” is not limited to any of the other numerous advantages that can be obtained from such a model when properly deployed.[000170] A cloud-computing model can be composed of various characteristics, such as on-demand self-service, broad network access, resource pooling, rapid elasticity, measured service, and so forth. A cloud-computing model may also come in the form of various service models such as, for example, Software as a Service (“SaaS”), Platform as a Service (“PaaS”), and Infrastructure as a Service (“laaS”). The cloud-computing model may also be deployed using different deployment models such as private cloud, community cloud, public cloud, hybrid cloud, and so forth.Abbreviations and Defined Terms[000171] To assist in understanding the scope and content of this written description and the appended claims, a select few terms are defined directly below. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure pertains.[000172] The terms “approximately,” “about,” and “substantially,” as used herein, represent an amount or condition close to the specific stated amount or condition that still performs a desired function or achieves a desired result. For example, the terms “approximately,” “about,” and “substantially” may refer to an amount or condition that deviates by less than 10%, or by less than 5%, or by less than 1%, or by less than 0.1%, or by less than 0.01% from a specifically stated amount or condition.[000173] Various aspects of the present disclosure, including devices, systems, and methods may be illustrated with reference to one or more embodiments or implementations, which are exemplary in nature. As used herein, the term “exemplary” means “serving as an example, instance, or illustration,” and should not necessarily be construed as preferred or advantageous over other embodiments disclosed herein. In addition, reference to an “implementation” of the present disclosure or embodiments includes a specific reference to one or more embodiments thereof, and vice versa, and is intended to provide illustrative examples without limiting the scopeof the present disclosure, which is indicated by the appended claims rather than by the present description.[000174] As used in the specification, a word appearing in the singular encompasses its plural counterpart, and a word appearing in the plural encompasses its singular counterpart, unless implicitly or explicitly understood or stated otherwise. Thus, it will be noted that, as used in this specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise. For example, reference to a singular referent (e.g., “a widget”) includes one, two, or more referents unless implicitly or explicitly understood or stated otherwise. Similarly, reference to a plurality of referents should be interpreted as comprising a single referent and / or a plurality of referents unless the content and / or context clearly dictate otherwise. For example, reference to referents in the plural form (e.g., “widgets”) does not necessarily require a plurality of such referents. Instead, it will be appreciated that independent of the inferred number of referents, one or more referents are contemplated herein unless stated otherwise.[000175] References in the specification to "one embodiment," "an embodiment," "an example embodiment," and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.[000176] It shall be understood that although the terms "first" and "second" etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed terms.[000177] It will be further understood that the terms "comprises", "comprising", "has", "having", "includes" and / or "including", when used herein, specify the presence of statedfeatures, elements, and / or components etc., but do not preclude the presence or addition of one or more other features, elements, components and / or combinations thereof.Conclusion[000178] The present disclosure includes any novel feature or combination of features disclosed herein either explicitly or any generalization thereof. Various modifications and adaptations to the foregoing exemplary embodiments of this disclosure may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings. However, any and all modifications will still fall within the scope of the non-limiting and exemplary embodiments of this disclosure.[000179] It is understood that for any given component or embodiment described herein, any of the possible candidates or alternatives listed for that component may generally be used individually or in combination with one another, unless implicitly or explicitly understood or stated otherwise. Additionally, it will be understood that any list of such candidates or alternatives is merely illustrative, not limiting, unless implicitly or explicitly understood or stated otherwise.[000180] In addition, unless otherwise indicated, numbers expressing quantities, constituents, distances, or other measurements used in the specification and claims are to be understood as being modified by the term “about,” as that term is defined herein. Accordingly, unless indicated to the contrary, the numerical parameters set forth in the specification and attached claims are approximations that may vary depending upon the desired properties sought to be obtained by the subject matter presented herein. At the very least, and not as an attempt to limit the application of the doctrine of equivalents to the scope of the claims, each numerical parameter should at least be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of the subject matter presented herein are approximations, the numerical values set forth in the specific examples are reported as precisely as possible. Any numerical values, however, inherently contain certain errors necessarily resulting from the standard deviation found in their respective testing measurements.[000181] Any headings and subheadings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. The terms and expressions which have been employed herein are used as terms of description and not oflimitation, and there is no intention in the use of such terms and expressions of excluding any equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the present disclosure. Thus, it should be understood that although the present disclosure has been specifically disclosed in part by certain embodiments, and optional features, modification and variation of the concepts herein disclosed may be resorted to by those skilled in the art, and such modifications and variations are considered to be within the scope of this present description.[000182] It will also be appreciated that systems, devices, products, kits, methods, and / or processes, according to certain embodiments of the present disclosure may include, incorporate, or otherwise comprise properties or features (e.g., components, members, elements, parts, and / or portions) described in other embodiments disclosed and / or described herein. Accordingly, the various features of certain embodiments can be compatible with, combined with, included in, and / or incorporated into other embodiments of the present disclosure. Thus, disclosure of certain features relative to a specific embodiment of the present disclosure should not be construed as limiting application or inclusion of said features to the specific embodiment. Rather, it will be appreciated that other embodiments can also include said features, members, elements, parts, and / or portions without necessarily departing from the scope of the present disclosure.[000183] Moreover, unless a feature is described as requiring another feature in combination therewith, any feature herein may be combined with any other feature of a same or different embodiment disclosed herein. Furthermore, various well-known aspects of illustrative systems, methods, apparatus, and the like are not described herein in particular detail in order to avoid obscuring aspects of the example embodiments. Such aspects are, however, also contemplated herein.[000184] It will be apparent to one of ordinary skill in the art that methods, devices, device elements, materials, procedures, and techniques other than those specifically described herein can be applied to the practice of the described embodiments as broadly disclosed herein without resort to undue experimentation. All art-known functional equivalents of methods, devices, device elements, materials, procedures, and techniques specifically described herein are intended to be encompassed by this present disclosure.[000185] When a group of materials, compositions, components, or compounds is disclosed herein, it is understood that all individual members of those groups and all subgroupsthereof are disclosed separately. When a Markush group or other grouping is used herein, all individual members of the group and all combinations and sub-combinations possible of the group are intended to be individually included in the disclosure.[000186] The above-described embodiments are examples only. Alterations, modifications, and variations may be effected to the particular embodiments by those of skill in the art without departing from the scope of the description, which is defined solely by the appended claims.

Claims

CLAIMS1. A computer implemented method for performing automated resource configuration, the method comprising: receiving a Network Service, NS, request, the NS request comprising one or more Virtual Network Functions, VNFs, and one or more related Service Level Agreement, SLA, requirements, wherein the NS request is described by a VNF Forwarding Graph, VNF-FG; receiving a target key performance indicator, KPI, for the VNF-FG; estimating one or more performance measurements of each of the one or more VNFs per unit of one or more resources; estimating one or more weights of dependencies between the one or more VNFs for performance evaluation; estimating one or more target key performance indicators, KPIs, of each of the one or more VNFs based at least in part on the target KPI for the VNF-FG; and generating one or more resource configurations for the VNF-FG.

2. The method of claim 1, wherein the one or more resource configurations are based on at least one of: the one or more target KPIs; the estimated one or more weights of dependencies between the one or more VNFs; and the one or more performance measurements.

3. The method of claim 1 or 2, further comprising creating one or more Markov Decision Processes, MDPs, to generate the one or more resource configurations.

4. The method of claim 3, wherein each of the one or more MDPs is defined by a tuple (5, A, p, r) where S is a finite set of states, A is a finite set of actions, p is a transition probability from state 5 to state .s ’ after action a is executed, and r is the immediate reward obtained after action a is performed.

5. The method of claim 1 or 2, further comprising creating one or more Markov Games to generate the first one or more resource configurations, the second one or more resource configurations, and the third one or more resource configurations.

6. The method of claim 3, 4, or 5, further comprising performing Reinforcement Learning, RL, to solve the one or more MDPs or one or more Markov Games.

7. The method of claim 4, wherein the finite set of states S, for one or more VNFs n, can be represented by:S = {< res^, P , d11, d12, , dln>, < res2, p2, d21, d22, ... , d2n... , < resn, pn,n, dn2, , ^nn > } wherein; resnis a current resources configuration of each of the one or more VNFs; pnis a current performance measurement of each of the one or more VNFs; and dnis a dependency between the one or more VNFs.

8. The method of claim 4 or 7, wherein selecting an action means selecting or changing a resource configuration for the one or more VNFs and moving the system to a new state S, and wherein the finite set of actions A, for the one or more VNFs n, can be represented by: A = [ai, a2, as ... . an] .

9. The method of any of claims 4 or 7-8, wherein the reward rvnfi for the one or more VNFs n comprises a vector indicating a performance of taking one of the finite set of actions i, such that rewardvnf. = [r^, ... r .

10. The method of claim 9, wherein the performance is given by a score, wherein:Scorevnf. = resource_amount * performance_per_resource_un.it, wherein perf ormance_p er _re source _unit indicates how much the performance improves for a given unit of resource.

11. The method of claim 9 or 10, wherein a global performance of the one or more VNFs considering the one or more weights of dependencies is given by:wherein dependents^ indicate the one of the one or more VNFs that the current vnf depends on, and a and f> reflect the relative service dependence such that a indicates how much the global performance of vnft depends on itself and (f indicates how much the global performance of vnf depends on vnf.

12. The method of claim 11, wherein the global performance is normalized as:wherein targetKPIvnf. is the performance target of each VNF.

13. The method of claim 6, wherein performing RL comprises use of at least one of: Deep RL, DRL; and Deep Q-Network, DQN.

14. The method of claim 13, wherein an optimal Q value follows the Bellman optimality equation:Q(st, at<- Q(st, at) + cr[(r(st, at) + y. max{Q(st+i, at+1)}) - Q(st, at)], such that a state s is given as input and the Q value of all possible actions a is generated as output.

15. The method of claim 14, wherein actions a are selected using a modified epsilon greedy policy.

16. A system for performing automated resource configuration comprising processing circuitry and a memory, the memory containing instructions executable by the processing circuitry whereby the system / apparatus is operative to:receive a Network Service, NS, request, the NS request comprising one or more Virtual Network Functions, VNFs, and one or more related Service Level Agreement, SLA, requirements, wherein the NS request is described by a VNF Forwarding Graph, VNF-FG; receive a target key performance indicator, KPI, for the VNF-FG; estimate one or more performance measurements of each of the one or more VNFs per unit of one or more resources; estimate one or more weights of dependencies between the one or more VNFs for performance evaluation; estimate one or more target key performance indicators, KPIs, of each of the one or more VNFs based at least in part on the target KPI for the VNF-FG; and generate one or more resource configurations for the VNF-FG.

17. A computer implemented method for performing automated resource configuration, the method comprising: deploying a Virtual Network Function Forwarding Graph, VNF-FG in a cloud environment, wherein the VNF-FG comprises one or more Virtual Network Functions, VNFs, and one or more related Service Level Agreement, SLA, requirements; monitoring the cloud environment; extracting or estimating one or more performance parameters from the cloud environment; initializing an environment simulator with at least one of the one or more performance parameters; training a Reinforcement Learning, RL, agent with the one or more performance parameters; and saving the trained RL agent.

18. The method of claim 17, wherein the training comprises: defining a number of episodes to train the RL agent, wherein an episode comprises all the states that come in between an initial state and a terminal state; generating, at the beginning of each episode, a random target performance; defining a number of iterations within each episode, each iteration representing a frame within an episode;selecting an action for each of the one or more VNFs; performing the action for each of the one or more VNFs; detecting a new state after performing the action, the new state indicating a current resource configuration of the one or more VNFs, a current performance of the one or more VNFs, and interdependencies between the one or more VNFs; calculating a reward function for each of the one or more VNFs; checking, after each iteration, whether the random target performance is met or not; proceeding to a next of the number of episodes if the random target performance is met; if the random target performance is not met, then selecting and performing an action and detecting a new state for the defined number of iterations; proceeding to the next of the number of episodes after the defined number of iterations; checking, at the beginning of each episode, whether the trained RL agent is accurate enough for the generated random target performance; if the trained RL agent is not accurate enough for the generated random target performance, then proceeding to the next episode; if the trained RL agent is accurate enough for the generated random target performance, then checking whether the RL agent is accurate enough for a random dependency; if the RL agent is not accurate enough for a random dependency, then generating a new random dependency; if the RL agent is accurate enough for the random dependency, then checking whether the RL agent is accurate enough for a random performance per resource; if the RL agent is accurate enough for the random performance per resource, then ending the training; and if the RL agent is not accurate enough for the random performance per resource, then generating a new random performance per resource.

19. A system for performing automated resource configuration comprising processing circuitry and a memory, the memory containing instructions executable by the processing circuitry whereby the system / apparatus is operative to:deploy a Virtual Network Function Forwarding Graph, VNF-FG in a cloud environment, wherein the VNF-FG comprises one or more Virtual Network Functions, VNFs, and one or more related Service Level Agreement, SLA, requirements; monitor the cloud environment; extract or estimating one or more performance parameters from the cloud environment; initialize an environment simulator with at least one of the one or more performance parameters; train a Reinforcement Learning, RL, agent with the one or more performance parameters; and save the trained RL agent.

20. A computer implemented method for obtaining resource configurations, comprising: receiving a target key performance indicator, KPI, for a Virtual Network FunctionForwarding Graph, VNF-FG, wherein the VNF-FG comprises one or more Virtual Network Functions, VNFs to be implemented in a cloud environment; receiving one or more saved performance parameters from the cloud environment; initializing an environment simulator with the one or more saved performance parameters; using a trained Reinforcement Learning, RL, agent to identify one or more resource configurations to meet the target KPI; applying the one or more resource configurations in the cloud environment; checking whether the one or more resource configurations meet the target KPI for the VNF- FG; and obtaining a dataset of resource configurations identified by the trained RL agent that meet the target KPI.

21. The method of claim 19, further comprising, if the target KPI is not met, then extracting one or more most recent performance parameters and adjusting the one or more most recent performance parameters to ensure convergence of the one or more resource configurations toward an optimal resource configuration.

22. A system for obtaining resource configurations comprising processing circuitry and a memory, the memory containing instructions executable by the processing circuitry whereby the system / apparatus is operative to: receive a target key performance indicator, KPI, for a Virtual Network Function Forwarding Graph, VNF-FG, wherein the VNF-FG comprises one or more Virtual Network Functions, VNFs to be implemented in a cloud environment; receive one or more saved performance parameters from the cloud environment; initialize an environment simulator with the one or more saved performance parameters; use a trained Reinforcement Learning, RL, agent to identify one or more resource configurations to meet the target KPI; apply the one or more resource configurations in the cloud environment; check whether the one or more resource configurations meet the target KPI for the VNF- FG; and obtain a dataset of resource configurations identified by the trained RL agent that meet the target KPI.

Images