Optimization of network parameters with world models

Abstract

A computer-implemented method performed by a computing device to jointly optimize a plurality of network parameters in a telecommunications network with a plurality of world models The method includes determining (104) a change to a plurality of network parameters based on querying respective individual world models in the plurality of world models to find a value of respective network parameters that result in a desired network state. The method further includes communicating (106), towards a network node, the change to the plurality of network parameters. Related methods and apparatus are also provided.

Description

OPTIMIZATION OF NETWORK PARAMETERS WITH WORLD MODELS TECHNICAL FIELD

[0001] The present disclosure relates generally to computer-implemented methods performed by a computing device to jointly optimize a plurality of network parameters in a telecommunications network with a plurality of world models, and related methods and devices.BACKGROUND

[0002] Optimizing a telecommunications network, such as a mobile network, can be challenging. Challenges can include, for example, that all network parameters cannot be safely tried in a live network to find a best network parameter. There can be many different network parameters that can be changed, and each network parameter can affect network performance in different ways. Moreover, there may not be accurate machine learning (ML) models that can capture complexities of real world deployments.SUMMARY

[0003] There currently exist certain challenges. Some ML-based optimization approaches are limited to optimizing one network parameter at a time. For example, some optimization approaches may include reinforcement learning (RL) based optimization where an agent is trained to recommend a change in a network parameter from an observation. RL based optimization may be done in simulation or by interacting with an individual world model (also referred to as a digital twin). RL based optimization, however, may have challenges including a need for re-training; a learning process with many interactions with the RL model which can be computationally expensive; and / or the individual world model cannot model all aspects of a network and is limited to the control of a single network parameter. Other ML-based optimization approaches may include a data driven ML model. However, data driven ML models also may have challenges including a need for a dataset that contains a diversity of conditions and network parameter configuration; learning such a single ML model may be intractable; and is limited to control of a single network parameter. Thus, ML-based optimization approaches may be limited to optimizing one network parameter at a time without orchestration of different network parameters; and the ML model is then limited to predicting the influence of a change in the single network parameter on a limited set of key performance indicators (KPIs).

[0004] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges.

[0005] Some embodiments provide a computer-implemented method performed by a computing device to jointly optimize a plurality of network parameters in a telecommunications network with a plurality of world models. The method comprises determining a change to a plurality of network parameters based on querying respective individual world models in the plurality of world models to find a value of respective network parameters that result in a desired network state; and communicating, towards a network node, the change to the plurality of network parameters.

[0006] Other embodiments provide a computer-implemented method performed by a computing device to learn a global world model to provide a joint network parameter change for a network state in a telecommunications network. The method comprises accessing a dataset from respective individual world models in a plurality of world models. The dataset comprises a first network state, a plurality of respective network parameter changes for the respective individual world models, and an estimate of a second network state when the respective network parameter changes are applied. The method further comprises learning the global world model from the dataset to obtain a learned global world model; and using the learned global world model to provide a joint network parameter change for a given network state.

[0007] Other embodiments provide a computing device configured to jointly optimize a plurality of network parameters in a telecommunications network with a plurality of world models. The computing device comprises processing circuity; and at least one memory connected to the processing circuitry and storing program code that is executed by the processing circuitry to perform operations. The operations comprise to determine a change to a plurality of network parameters based on querying respective individual world models in the plurality of world models to find a value of respective network parameters that result in a desired network state; and to communicate, towards a network node, the change to the plurality of network parameters.

[0008] Some embodiments provide a computing device configured to learn a global world model to provide a joint network parameter change for a network state in a telecommunications network. The computing device comprises processing circuity; and at least one memory connected to the processing circuitry and storing program code that is executed by the processing circuitry to perform operations. The operations comprise to access a dataset from respective individual world models in a plurality of world models. The dataset comprises a first network state, a plurality of respective network parameter changes for the respective individual world models, and an estimate of a second network state when the respective network parameter changes are applied. Theoperations further comprise to learn the global world model from the dataset to obtain a learned global world model; and to use the learned global world model to provide a joint network parameter change for a given network state.

[0009] Certain embodiments may provide one or more of the following technical advantage(s). Based on inclusion of a plurality of world models, a more accurate and / or efficient estimation of an impact of multiple network parameter changes may be obtained.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiments of the present disclosure. In the drawings:

[0011] Figures 1 and 2 are flowcharts illustrating operations of a computing device according to some embodiments;

[0012] Figure 3 is a block diagram illustrating a combination of multiple world models to estimate the effect of multiple network parameter changes according to some embodiments;

[0013] Figure 4 is a block diagram illustrating a distributed implementation according to some embodiments;

[0014] Figure 5 is a block diagram illustrating a distributed implementation that includes respective pairs of an individual world model and an optimizer according to some embodiments;

[0015] Figure 6 is a block diagram illustrating components and a search method according to some embodiments;

[0016] Figure 7 is a schematic diagram illustrating a tree search according to some embodiments;

[0017] Figure 8 is a sequence diagram illustrating operations according to some embodiments;

[0018] Figure 9 is a sequence diagram illustrating operations according to some embodiments;

[0019] Figure 10 is a block diagram of computing device according to some embodiments;

[0020] Figure 11 is a block diagram of a communication system according to some embodiments;

[0021] Figure 12 is a block diagram of another communication system in according to some embodiments; and

[0022] Figure 13 is a block diagram of a service management and orchestration framework according to some embodiments;

[0023] Figure 14 is a block diagram of a network node according to some embodiments; and

[0024] Figure 15 is a block diagram of a virtualization environment according to some embodiments.DETAILED DESCRIPTION

[0025] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art, in which examples of embodiments of the present disclosure are shown. Inventive concepts may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present / used in another embodiment.

[0026] One approach for trying to optimize a telecommunications network includes RL based optimization where an agent is trained to recommend a change in network parameters from an observation. The agent often may be trained in a simulated network and fine-tuned in the real network. RL-based optimization requires a dataset of interactions with the network; and often may be done in simulation or by interacting with an individual world model. Such an approach may have the following challenges: an agent may need to be re-trained for every change in the objective function; the learning process may require many interactions with the RL model, which can be computationally expensive; and / or a simulator may not be able to model all aspects of a network (e.g. a static simulator will not model handover). When a simulator can capture link level and mobility aspects, the simulator may be too complex for large scale training of ML agents and, thus, is limited to the control of a single network parameter in specific conditions.

[0027] Another approach may be data-driven ML model-based optimization that includes: an ML learning step that includes learning a predictive ML model that, given a current network state and a change of a network parameter, predicts a next network state; and an optimization algorithm which, given an ML model, may find a particular change of parameter. The ML model used for optimization may include an individual world model.

[0028] ML model-based optimization with a data driven ML model may not require intensive re-training when the objective function changes as the same ML model may be used for different downstream optimization tasks. The training process of the ML model may be performed using standard supervised learning, which may be more stable than RL, but can have challenges.

[0029] A first challenge may include that learning the ML model typically requires a good dataset that contains a diversity of conditions and parameter configuration. However, most network parameters may not be optimized, or if they are tuned, operators may tend to pick values from a very reduced set of options. Naively collecting a dataset from current network operations may be unlikely to provide enough diversity in the parameter values to learn a good world model.

[0030] Another challenge may include that learning a single ML model to capture the parameters of the network is intractable. There may be a large number of performance management (PM) counters and configuration management (CM) parameters. Thus, a very large dataset may be needed to explore every combination and a very complex ML model may be needed to represent every possible interaction between CM parameter and PM counter changes.

[0031] As a consequence of these challenges, such ML model-based optimization approaches are limited to optimizing one network parameter at a time: for example, maximum transmit power. For example, an individual world model that optimizes one network parameter at a time is then limited to predicting the influence of a change in transmit power on a limited set of KPIs, where a KPI is a value built using a formula based on PM counters for example.

[0032] Thus, it may be desirable to jointly optimize multiple network parameters of a telecommunications network to improve performance.

[0033] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges.

[0034] Some examples include a method to optimize multiple network parameters jointly by composing a global world model from several world models that model the impact of individual parameters.

[0035] Given multiple world models that each model the impact of a network parameter change, the method of some examples includes:An evaluation method to capture the joint impact of multiple parameter changes, which includes calling different world models in a way that can gradually and accurately incorporates the different changes;A gradient based optimization method which allows for each world model to be implemented as an independent rApp; and / orA search method for finding the optimal order of network parameter updates using the individual world models.

[0036] Some examples include a method to evaluate the impact of modifying multiple network parameters jointly using multiple world models, each modelling the impact of only a subset of the network parameters. The method calls the world models sequentially, with thenetwork state modified between each call to reflect the predicted impact of changing the network parameters associated with the called world model. The method of such examples may provide a technical advantage of providing a more accurate estimation of the impact of multiple network parameter changes than querying separately the world models for each network parameter.

[0037] Other examples include a method to jointly optimize multiple network parameters by using an optimization method on top of the world model chaining. The optimization method attempts to find the best joint change of parameters by querying the world models for different joint values of the configuration parameters. A technical advantage of these examples may include a joint optimization of multiple network parameters, which may lead to better performance compared to optimizing a single parameter at a time. The method is applicable for any type of network parameters, but may include many calls to the world models. An advantage of joint optimization is frequently observed in RAN, as many different configuration parameters may be used to try to address similar performance optimization.

[0038] In some examples, the joint optimization of all or a subset of the multiple network parameters is performed using gradient information exchanged across the different world models. A technical advantage of these examples may include a fast joint optimization of multiple continuous network parameters, by enabling the use of gradient-based optimizers, at the cost of requiring differentiable world models and exchanging gradient information between them. For example, the number of queries to the world models using the method of these examples may be greatly reduced compared the previous joint optimization method discussed above.

[0039] All or a subset of the world models can be associated to an optimizer. The optimizer can be queried to provide likely-efficient values for the associated network parameters. A joint optimization method can iteratively select the respective world models and then query the associated optimizers for an action to perform a more efficient search for the joint parameters configuration. A technical advantage of these examples may include making search-based methods more efficient, as the method may focus the search on likely-to-be-efficient parameter values.

[0040] Some examples include a method to evaluate the impact of modifying multiple network parameters jointly using multiple world models, each modelling the impact of only a subset of the network parameters. The method of these examples collects samples from the individual world models and learns a joint world model. The joint world model can be used to find the best set of joint network parameters optimizing the network.

[0041] Gradient-based methods and search-based methods may be combined in a same optimization method, in particular if some network parameters and world models permit gradientbased optimization while others do not.

[0042] Figure 1 is a flowchart illustrating operations of a computing device according to some embodiments. Referring to Figure 1, some embodiments are directed to a computer-implemented method performed by a computing device to jointly optimize a plurality of network parameters in a telecommunications network with a plurality of world models. The method includes determining 104 a change to a plurality of network parameters based on querying respective individual world models in the plurality of world models to find a value of respective network parameters that result in a desired network state. The method further includes communicating 106, towards a network node, the change to the plurality of network parameters.

[0043] Some examples provide a method to optimize multiple network parameters jointly by composing a global world model from individual world models that model the impact of individual network parameters. The term “global world model” refers to a composition of a plurality of individual world models.

[0044] In some embodiments, a change to the plurality of network parameters includes determining a change to at least one of (i) all of the plurality of network parameters and (ii) a subset of the plurality of network parameters, where the subset includes greater than one network parameter.

[0045] In some embodiments, a change to the plurality of network parameters includes a change to respective CM parameters; the network state includes a plurality of key performance indicators (KPIs); and a respective world model provides a respective estimate of the network state when the respective change to the respective CM parameters is applied. The KPIs can include or be based on, without limitation, PM counters.

[0046] For example, a world model for network optimization can take as input current PM counters, current CM parameters, and an intended change in CM parameters. The output of the world model, in this example, is a prediction of the PM counters after this change is performed.

[0047] As used herein, a world model refers to a ML model / function that, given a network state representation and a value for a control configuration parameter, or for a change in a configuration parameter, outputs a prediction of the network state after the control value is applied. A world model also may be referred to as a digital twin, and can be a simulator, a mathematical model, or an ML model trained on real or synthetic data. For example, a world model can be built based on a combination of mathematical models and real data, from synthetic data coming from a simulation campaign, or from real data coming from a measurement campaign (or historical network data when possible). The prediction of the network state also may be referred to as an estimated network state.

[0048] In some examples, multiple world models respectively model the impact of a network parameter change (also referred to herein as an action).

[0049] In some embodiments, respective network states overlap in at least a subset of the respective world models.

[0050] Referring to Figure 1, some embodiments further include evaluating 102 an impact of changing the plurality of network parameters based on sequentially calling the respective world models with the network state modified between each call to reflect the estimated impact on the network state of changing the plurality of network parameters.

[0051] For example, some examples include an evaluation method to capture the joint impact of multiple network parameter changes. The evaluation method in these examples includes calling different world models in a way that gradually and accurately incorporates the different changes.

[0052] In some embodiments, the telecommunications network includes an open-radio access network (O-RAN); the computing device includes at least one rApp; and the respective individual world models include respective additional rApps.

[0053] For example, the method of some embodiments can be implemented in a service management and orchestration (SMO) framework (FW) that includes signalling between the SMO and rApps, as discussed further herein. The individual world models can be implemented in the SMO FW as separate rApps.

[0054] In some embodiments, the determining operation 104 of Figure 1 includes using gradient information exchanged across the respective individual world models.

[0055] The gradient information is with respect to the network parameters in some examples. In other examples, the gradient information is with respect to weights of the respective world models.

[0056] As discussed further herein, some examples include a gradient based optimization method which allows for each world model to be implemented as an independent rApp.

[0057] Other examples, as discussed further herein, include using a distributed search to find an optimal order of the world models.

[0058] Some examples are discussed in the non-limiting context of network SMO problems and include a method for changing network parameters by optimizing a metric using multiple predictive world models.

[0059] Non-limiting examples of network parameters that can be configured include remote electrical tilt (RET) and downlink transmission power (TxPower). In some examples, these network parameters need to be controlled jointly as they can affect the same KPIs. For example,RET tuning can improve coverage and quality; and TxPower tuning can improve energy consumption but, at the same time, can deteriorate coverage and quality.

[0060] Further non-limiting examples of network parameters include tuning uplink power control parameters (ULPC) or cell individual offset.

[0061] Some examples include modelling the effect of each network parameter with a predictive model, e.g. a world model.

[0062] The network parameters can have an effect on the network state, which can be modeled by a combination of configuration parameters (e.g., current tilt, current power, antenna branches on, etc.), information about the network topology (e.g., frequency, height, inter-site distance, etc.), and network performance indicators (reference signal received power (RSRP), signal to interference plus noise ratio (SINR), throughput, etc.).

[0063] In one example, by combining a world model for RET and a world model for TxPower, a jointly optimized network state may be found.

[0064] An example optimization with one world model is as follows. The world model can include s' = f(s, ) where s is the current network state (including PM and CM values), and an action a is the value of the specific CM parameter that controls / tunes the world model, or a change in value (e.g. a delta), and s’ is the predicted network state. In one example, the world model is differentiable and can communicate gradients with respect to the control configuration parameter.

[0065] In this example, the optimization problem includes finding a configuration value that maximizes a network KPI or a combination of KPIs. More generally, a utility function U (s) can return a scalar value to be maximized.

[0066] Given a single world model, and a current network state s, in this example, the following utility function is solved:max U(f(s, a))a

[0067] In one example, this is solved using gradient ascent:a ← a + η∇aU(f(s, a))which, using the chain rule is VaU f s, a)) =

[0068] In another example, evaluation of network KPIs with multiple world models is performed. In this example, a set of world models is available to a joint optimizer node. In an example setup, the joint optimizer node is an rApp in the non-real time (non-RT) radio access network (RAN) intelligent controller (RIC) and the world models also are rApps. The world models, in this example, communicate their availability to the joint optimizer via the R1 interfaceas a registration process. The R1 interface comprises an rApps interface to the non-RT-RIC and / or SMO services.

[0069] In this example, the joint optimizer is configured with a utility formula to compute one or more network utility KPIs, which can be combined into one scalar objective to be maximized, or scalar constraints to be satisfied. In addition to the utility formula, a formula describing the order of chaining the world models is provided. The formula can be configured from the SMO framework (FW). In this example, the utility function is U(s’), which is a function of the predicted network state after applying joint parameter changes, and a function f(s, a) which models these changes.

[0070] The function f is a combination of multiple world models modeling a subset of the network parameters. As an example, let,2, A be three world models available to the joint optimizer. A utility function can be:U f^s, ai),2(s> a2)< (s<as)) ior canrely on composition / chaining of one or more of the available modelsE(A(A(A(S’ai)’a2)<as));orU(f3(s, 0.3), f2(f1(s, a±), a2)); or any similar combinations

[0071] Referring to Figure 1, in some embodiments, the method further includes determining 100 an order in which the plurality of world models is composed.

[0072] Heuristics can be used to determine the order in which world models can be composed. For example, the world models can be composed by decreasing accuracy: or by calling the most accurate model first and the least accurate one in the end, etc.

[0073] Conditions for being able to compose the world models include, for example, that a world model only needs to capture a subset of the KPIs of the full network state. For example, as long as there is some overlap between any subset of the output KPIs of one world model and any subset of the input KPIs of another world model, the world models can be composed. Missing inputs that are not output of the previous world model can be read from the initial, full, network state s.

[0074] In some examples, a world model predicts a change in a KPI. That is, the world model takes as input a change in a network parameter, and current KPI values, and predicts the change in KPI instead of the absolute value.

[0075] The composition of two world models can still be accurate, even if the second world model does not take the first parameter as input, as long as the dataset or world model that is used to generate the world model has enough diversity.

[0076] For example, world model fl models the impact of an increment in RET and world model f2 models the impact of a change in TxPower. s' = f1(s, a1), and s'2is the state input tof2.s2may have different state variables than s', but in this example there is some meaningful overlap since they belong to the same domain of network management.

[0077] f1outputs a change in RSRP given a current RSRP and a change in RET. f2outputs a change in RSRP given a current RSRP and a change in power. The tilt value is not part of s2. Yet the two world models can still be composed by feeding the updated RSRP to f2.

[0078] The world model f2, in this example, has been trained on a dataset of RSRP and TxPower. This world model is not explicitly modelling tilt changes, but as long as its dataset contains a variety of RSRP (e.g. coming from actual difference in tilt during data collection), the impact of tilt modeled by f1will impact RSRP which will lead f2to a different output than if it was used directly on s, the state without tilt changes.

[0079] Thus, in this example, by composingand f2, a joint change in tilt and power is modelled.

[0080] In an example where two network parameters are tightly coupled, the chaining of two independent world models may still be suboptimal. In this example, the two network parameters may be considered as one two dimensional parameter. Moreover, the state being passed to the different world model needs to contain all the necessary information to be able to model a change in state with the change of a single parameter. For example, for tilt and power, this condition may be satisfied, but there may be groups of network parameters that do not satisfy this condition and may need a translation mechanism for the input states to be valid.

[0081] In some embodiments, determining (operation 104 in Figure 1) the change to the plurality of network parameters includes using gradient information. Using the gradient information includes: combining the respective individual world models in an order; accessing an initial network states; taking an actual value for the respective network parameters; calling the respective world models with the initial network state and the actual value for the respective network parameter to obtain a respective estimated network state and gradient; and applying a backward pass to update the actual value of the respective network parameters.

[0082] An example that includes using gradient information in a method for optimization with multiple world models is as follows. In this example, the function U, and the way to combine the function ft is provided, and three world models are chained. The present disclosure is not so limited, however, and any number of multiple world models may be chained. In this non-limiting example, however, for three world models modeling each parameter, the utility function is:max U(f3(f2(f1(s,a1),a2),a3))ai,a2,a3

[0083] The impact of an action can be reflected in the state description of the network. All of the world models do not need to use the same information in s. However, in this example, the information the world models use is impacted by the change in the previous action.

[0084] Continuing with this example, each of the three world models is differentiable. Figure 3 is a block diagram illustrating a combination of world models 300a, 300b, 300c to estimate the effect of multiple network parameter changes al, a2, a3. The chaining order of world models 300a, 300b, 300c may be provided or may be automatically derived from the utility function 302. The utility function 302 is solved using a chain rule and applying a backpropagation method as shown in Figure 3. Figure 3 also includes one step of gradient ascent to maximize the cost function when using a composition of the three world models (WM) WM 300a for RET, WM 300b for TxPower, and WM 300c for ULPC.

[0085] The world models can be implemented in a distributed manner, as shown in Figure 4, where each world model is presented as an individual rApp. In the example in Figure 4, a gradient ascent process is performed by a separate node referred to in Figure 4 by the non-limiting term “joint optimizer”. This node is responsible for calling the world models and collecting the gradients.

[0086] Referring to Figures 3 and 4, in this example, the following operations are performed.

[0087] In a first operation, the joint optimizer 402 is configured with a formula indicating how to combine the world models 300a, 300b, 300c available to compute the utility function 404.

[0088] In a second operation, the joint optimizer 402 receives network substate s from a radio access network (RAN) 400 (e.g. through an 01 interface).

[0089] Joint optimizer 402, in a third operation, samples an initial value for the actions al, a2, a3. The initial values can be the current values of the network parameters.

[0090] In a fourth operation, joint optimizer 402 queries f1in WM 300a with the network sub state s and al, and outputs predicted sub state si (PSI) and gradients. The network sub state s is a subset of the total network state.

[0091] In a fifth operation, joint optimizer 402 queries f2in WM 300b with the predicted network substate si and a2, and outputs predicted substate s2 (PS2) and gradients.

[0092] In a sixth operation, joint optimizer 402 queries f3in WM 300c with the predicted network state s2 and a3, and outputs predicted substate s3 (PS3) and gradients.

[0093] Joint optimizer 402, in a seventh operation, evaluates the cost function 404 on s3 and computes its gradient.

[0094] In an eight operation, joint optimizer 402 has all the gradients and can apply the backward pass to update the values for each action. The backward pass is performed with respect to the input network parameters al, a2, a3, as shown in Figure 3.

[0095] The method returns to the third operation until a desired number of iterations is reached.

[0096] A gradient based optimization method can impose constraints on the world model: The world model should be differentiable as a function of the network parameter being modeled; and the network parameter is modeled as a possibly multi-dimensional continuous value to be able to take the gradients. However, for discrete parameters, approximations are possible such as considering continuous values and rounding them before implementations, or using a stop gradient approximation. The results may be dependent on the order of the query.

[0097] Figure 8 is a sequence diagram illustrating operations for performing a gradient method according to some embodiments. In the example in Figure 8, open-RAN signalling is used between SMO 800 and non-RT RIC 804, and between SMO 800 and open-central unit (O-CU) / open-distributed unit (O-DU) 808. As shown, SMO 800 includes configuration node 802 and joint optimizer 402; and non-RT RIC 804 includes world model 300a and world model 300b.

[0098] A registration process to a joint optimization service through the R1 interface includes operations 1 and 2. In operation 1, world model 300a sends input and output state specifications to joint optimizer 402 through Rl; and in operation 2, world model 300b sends input and output state specifications to joint optimizer 402 through Rl. In operation 3, configuration node 802 signals to joint optimizer 402 a composition order of world models 300a and 300b, e.g. world model 300a (world model 300b).

[0099] Optimization includes operations 4 and 5, and a loop of operations 6 - 10. In operation 4, O-CU / O-DU 808 signals to joint optimizer 402, through the 01, Al interface, PM and CM counters comprising the full network state. In operation 5, joint optimizer 402 initializes network parameters al and a2.

[0100] The loop of operations 6 - 10 is repeated until a desired number of iterations is reached. In operation 6, joint optimizer 402 signals network sub state s2 and a2 to world model 300b for evaluation and gradients; and in operation 7, world model 300b signals s2’ and gradients with respect to a2. In operation 8, joint optimizer 402 signals s2’ and al to world model 300a; and in operation 9, world model 300b signals to joint optimizer 402 si’, where si’ = WM(s2’, al) + gradients with respect to al. In operation 10, joint optimizer 402 updates network parameters al, a2.

[0101] In operation 11, joint optimizer 402 signals to O-CU / O-DU 808, through the 01, Al interface, deployment of the updated CM parameters al, a2.

[0102] In another embodiment, a respective world model is associated with a respective optimizer and models a respective impact of a change to a respective network parameter on a network state. In this embodiment, determining (operation 104 in Figure 1) the change includes querying a respective optimizer to obtain a respective change of a respective network parameter; and accessing, from a respective world model, an estimate of a future network state when the respective change of the respective network parameter is applied.

[0103] The accessing can include a search with an input comprising a current network state and an identity of the respective world models and respective optimizers and an output comprising an identity of the respective changes to the respective network parameter.

[0104] For example, joint optimization may be performed with multiple world models without exchanging gradients. In this example, each available world model comes with its own optimizer that may or may not be gradient based. Further, in this example the function U is used to guide the optimization, but the function f is not used. Instead, a search method finds a way, e.g. the best way, to chain the world models. Each world model and optimizer is responsible for modeling the influence of one network parameter.

[0105] In this example, the world model need not be differentiable, and the network parameter can be a categorical variable.

[0106] Figure 5 is a schematic diagram showing, for this example, data exchange between the joint optimizer 402 and three respective pairs of a world model 300a, 300b, 300c and its respective optimizer 500a, 500b, 500c. Joint optimizer 402 has the ability to query an optimal action from a single parameter optimizer 500a, 500b, 500c. Similarly, joint optimizer 402 can communicate with the associated world model 300a, 300b, 300c to obtain a prediction of a future network state if the action is applied.

[0107] Figure 6 shows an example search method. The search method includes as in input a current network state and a list of respective world model, optimizer pairs that can participate in the joint optimization. This list can be dynamic. The output of the search method include an ordered list of actions to be applied by the RAN 400.

[0108] As shown in the example in Figure 6, in operation 1, joint optimizer 402 sends a current network state to optimizer 500a. In operation 2, optimizer 500a sends an optimal action to joint optimizer 402. Joint optimizer 402, in operation 3, sends the network state and the optimal action to world model 300a. In operation 4, world model 300a sends a prediction of a future network state if the action is applied.

[0109] This example may have an advantage of not requiring gradient exchanges between the world model 300a, 300b, 300c and the joint optimizer 402. This example also does not assume an a priori ordering of the world models 300a, 300b, 300c to compose and is also applicable to optimize discrete-valued or categorical network parameters.

[0110] In some embodiments, the search includes a search of a tree that includes a plurality of nodes that respectively correspond to respective network states and a plurality of branches that respectively correspond to respective optimizers.[OHl] Some embodiment include iteratively selecting a respective world model; and querying the respective optimizer associated with the respective world model for the respective change to a respective network parameter.

[0112] Figure 7 is a schematic diagram illustrating an example search method that includes a tree search where the nodes correspond to network states and the branches correspond to a specific choice of optimizer.

[0113] Referring to Figure 7, a current network state is obtained from RAN 400. Optimizer 500a is selected and queries an optimal action for a change in RET, RET1. Optimizer 500b also is selected and an queries an optimal action for a change in TxPower, TxPower1. The current network state s and optimal action RET1, TxPower^ is sent to the respective world models 300a, 300b. World model 300a simulates the action RET1to predict a future network state si when the action RET1is applied; and world model 300b simulates the action TxPower^ to predict a future network state si when the action TxPower^is applied. A new level of the tree is started from the predicted state si in the branch that includes optimizer 500a and world model 300a.

[0114] The operations of selecting an optimizer(s) that queries an optimal action(s), and simulating the optimal action with the associated world model to obtain a predicted future state, is repeated until a desired depth is reached.

[0115] Specifically, in the example in Figure 7, two new levels of the tree are started from the predicted state si in the branch that includes optimizer 500a and world model 300a. In the first new level, optimizer 500a is selected and an queries a second optimal action for a change in RET, RET2. The predicted network state si and optimal action RET2 is sent to world model 300a. World model 300a simulates the action RET2 to predict a future network state S2 when the action RET2 is applied.

[0116] In the second new level, optimizer 500b is selected and queries an optimal action for a change in TxPower, TxPower^. The predicted network state S2 and optimal action TxPower^s sent to world model 300b. World model 300b simulates the action TxPower^ to predict a future network state S2 when the action TxPower^ is applied.

[0117] When a maximum depth is reached, the final state is evaluated, and the branch with the best value is chosen.

[0118] Methods for performing such search include, without limitation, Monte Carlo tree search or branch and bound. The hyperparameters of the search method can be configured in the joint optimizer 402. Example hyperparameters include the number of iterations, depth of the search, initialization of the cost function at leaf nodes, etc. At a leaf node, the function U can be used to evaluate the utility of the transformed state.

[0119] In this example search method, error may be propagated through the nodes of the tree because calls to world models are chained. In an additional or alternative example, the search method may be improved if each world model provides uncertainty bounds about the output state.

[0120] Figure 9 is a sequence diagram illustrating operations for performing a search method according to some embodiments. In the example in Figure 9, open-RAN signalling is used between SMO 800 and non-RT RIC 804, and between SMO 800 and O-CU / O-DU 808. As shown, SMO 800 includes joint optimizer 402; and non-RT RIC 804 includes a first pair of world model 300a and optimizer 500a and a second pair of world model 300b and optimizer 500b.

[0121] A registration process to a joint optimization service through the R1 interface includes operations 1 and 2. In operation 1, world model 300a sends input and output state specifications to joint optimizer 402 through Rl; and in operation 2, world model 300b sends input and output state specifications to joint optimizer 402 through Rl.

[0122] Optimization with a tree search includes operations 3 - 18, which includes a loop of operations 5 - 17. The loop includes alternative loops of operations 7 - 10 and 11 - 14. In operation 3, O-CU / O-DU 808 signals to joint optimizer 402, through the 01, Al interface, PM and CM counters comprising the full network state. In operation 4, joint optimizer 402 initializes a tree with a root node matching the current network state.

[0123] The loop of operations 5 - 17 is repeated until a desired number of iterations is reached. In operation 5, joint optimizer 402 chooses the root node in the tree and gets the full network states from the root node. In operation 6, joint optimizer 402 selections an action. If action al is selected, operations 7 - 10 are performed. Alternatively, if action a2 is selected, operations 11 - 14 are performed.

[0124] If action al is selected, in operation 7, joint optimizer 402 requests from optimizer 500a an action proposal for parameter al; and in operation 8, optimizer 500a sends the action proposal for al to joint optimizer 402. In operation 9, joint optimizer 402 signals network substate si and al to world model 300a for evaluation; and in operation 10, world model 300a signals si’ to joint optimizer 402.

[0125] If action a2 is selected, in operation 11, joint optimizer 402 requests from optimizer 500b an action proposal for parameter a2; and in operation 12, optimizer 500b sends the action proposal for a2 to joint optimizer 402. In operation 13, joint optimizer 402 signals network substate s2 and a2 to world model 300b for evaluation; and in operation 14, world model 300b signals s2’ to joint optimizer 402.

[0126] In operation 15, joint optimizer 402 adds a node to the tree with the predicted full network state. Joint optimizer 402, in operation 16, evaluates the predicted full network states with a cost function; and in operation 17, joint optimizer 402 backpropagates cost to the root node.

[0127] In operation 18, joint optimizer 402 selects the best path in the tree, giving (al, a2) or iterations of al and a2 values.

[0128] In operation 19, joint optimizer 402 signals to O-CU / O-DU 808, through the 01, Al interface, deployment of the new parameters al, a2.

[0129] Figure 2 is a flowchart illustrating operations of a computing device according to some embodiments. Referring to Figure 2, some embodiments are directed to a computer-implemented method performed by a computing device to learn a global world model to provide a joint network parameter change for a network state in a telecommunications network. The method includes accessing 200 a dataset from respective individual world models in a plurality of world models. The dataset includes a first network state, a plurality of respective network parameter changes for the respective individual world models, and an estimate of a second network state when the respective network parameter changes are applied. The method further includes learning 202 the global world model from the dataset to obtain a learned global world model; and using 204 the learned global world model to provide a joint network parameter change for a given network state.

[0130] Learning 202 can include supervised learning from the dataset. Learning 202 can be triggered responsive to availability of a new individual world model in the plurality of world models.

[0131] Using 204 the learned global world model can include finding an optimal joint network parameter change from the plurality of joint network changes. In some embodiments, using 204 the learned global world model includes taking a joint gradient over the learned global world model with respect to respective network parameter changes being optimized.

[0132] In some embodiments, using 204 the learned global world model includes applying an incremental update for respective network parameter changes based on the joint gradient to obtain an updated joint parameter change.

[0133] The first network state can include a plurality of current KPIs; the initial joint network parameter change can include a respective initial CM parameter for the respective individual worldmodels; and the plurality of estimates of the network state can include respective estimates of a change in the plurality of KPIs based on the plurality of current KPIs and a respective change to the respective initial CM parameters.

[0134] In some embodiments, the learned global world model comprises a neural network.

[0135] In an example, a joint world model, also referred to as a global world model, is learned from multiple individual world models.

[0136] In this example, individual world models are combined, e.g. when an exact way of combining the functions is unknown. The composition order is used to map the first network state and a plurality of network parameter changes to a second network state. Supervised learning can then be performed to learn this mapping.

[0137] The composition order can be determined each time a sample is collected in order to probe the world models in a sequence and get a next state s’. However, every sample may be in a different order.

[0138] Learning a global world model may be useful because chaining the world models may lead to suboptimal results compared to an option of jointly optimizing over all actions simultaneously.

[0139] Learning a global world model may have technical advantages including, for example, the individual world models do not need to be differentiable or to communicate gradients; the function in which to compose the individual world models f does not need to be specified; and / or the resulting joint world model is order invariant and can evaluate a joint change in parameter directly.

[0140] This example includes two separate stages. In the first stage, the joint optimizer learns a joint model. The learning can be triggered every time a new individual world model rApp is registered to the joint optimizer. In the second stage, the joint optimizer uses the learned model to provide an optimal joint parameter change for a given network state.

[0141] In the learning stage, joint optimizer 402 starts by collecting a joint dataset of the form (s, (a1,..., aM), s') by probing the individual world models.

[0142] An example strategy to collect the joint dataset from three individual world models f1, f2, f3, for example, can include:Pick an arbitrary composition order of the world models, for examplef3(f2(f1(s,a1),a2),a3).Choose a set of initial states s, and a set of joint actions (a1, a2, a3). They can be sampled uniformly in a specified range.Probe the world models in order: s1' = f1(s, a1), then s2' = f2(s1', a2), then s' = f3(s2', a3)Create a joint sample (s, (a1,..., aM), s') and add it to the dataset.Repeat the procedure by changing the world model composition order and sampling more initial state and joint actions.

[0143] Once a dataset is collected, a joint world model can be learned. For example, the joint world model can be learned through supervised learning. The joint world model can be a neural network and can be fully differentiable.

[0144] Continuing with the above non-limiting example of three world models with joint action (aq, a2, a3), the following operations are performed.

[0145] Joint optimizer 402 has learned the joint model f̂(s, a1, a2, a3) and can now solve the optimization problem jointly over all actions:max U( f(s, alta2, a3))

[0146] A multi-variate optimizer can be gradient-based, e.g. because a neural network can be differentiable, in which case the multi-variate optimizer initiates a: = (cfo a2, a3), for example. That is, in a first operation in this example, the joint optimizer 402 accesses the utility function 404. In a second operation, the joint optimizer 402 receives network substate s from a radio access network (RAN) 400 (e.g. through an 01 interface). Joint optimizer 402, in a third operation, samples an initial value for the actions al, a2, a3.

[0147] Next, in a fourth operation, the muti-variate optimizer queries learned multi-variate function f s, a) with input state s and joint action a to get output state s' and gradient vector V (s, a) = (^g^ > )’as we"asPartial derivative |y with respect to theutility function at s'.

[0148] In a fifth operation, the muti-variate optimizer has all the gradients and can apply the incremental update for each action a1(new) = a1(old) + ∂U / ∂f̂ · ∂f̂(s,a(old)) / ∂a1, and similarly for theO j Oupdates a2(new), a3(new).

[0149] The method, in this example, repeats the fourth and fifth operations until convergence (e.g., a desired number of iterations or incremental change less than a threshold).

[0150] Figure 13 is a block diagram of an 0-RAN architecture that includes an SMO FW according to some embodiments. This example 0-RAN architecture includes:(1) SMO framework 1300 comprises a component that oversees orchestration, management, and automation of RAN elements, and supports the 01, Al, and R1 interfaces;(2) Non-RT RIC 1302 comprises a logical function that enables non-real-time control and optimization of RAN elements and resources, artificial intelligence (AI) / ML workflow including model training and updates, and policy -based guidance of applications / features in near-RT RIC;(3) Near-RT RIC 1304 comprises a logical function that enables near-real-time control and optimization of 0-RAN elements and resources via fine-grained data collection and actions over an E2 interface;(4) O-CU 1312 comprises a logical node hosting radio resource controls (RRC), service data adaptation protocol (SDAP) and packet data convergence protocol (PDCP) protocols; (5) 0-RAN Central Unit - Control Plane (O-CU-CP) 1308 comprises a logical node hosting the RRC and the control plane part of the PDCP protocol;(6) 0-RAN Central Unit - User Plane (O-CU-UP) 1310 comprises a logical node hosting the user plane part of the PDCP protocol and the SDAP protocol;(7) 0-DU 1312 comprises a logical node hosting radio link control (RLC) / medium access control (MAC) / High-physical (PHY) layers based on a lower layer functional split; (8) 0-RAN Radio Unit (0-RU) 1314 comprises a logical node hosting Low-PHY layer and radio frequency (RF) processing based on a lower layer functional split. This is similar, for example, to the Third Generation Partnership Project (3 GPP) transmission / reception points (“TRP”) or remote radio head (“RRH”) but more specific in including the Low-PHY layer (fast Fourier transform (FFT) / inverse FFT (iFFT), physical random access channel (PRACH) extraction);(9) xAPP (not depicted) comprises a network function which, for example, can be an independent software plug-in to the Near-RT RIC 1304 platform to provide functional extensibility to a RAN by third parties;(10) rAPP (not depicted) comprises a network function which, for example, can be an independent software plug-in to the Non-RT RIC 1302 platform to provide functional extensibility to the RAN by third parties. The R1 interface comprises an rApps interface to non-RT RIC 1302 and / or SMO services; and(11) O-Cloud 1316 comprises hardware and / or software components that can provide cloud computing capabilities to execute RAN network functions.

[0151] As discussed, examples of the present disclosure can be implemented in the context of 0-RAN in the non-real time RIC. The respective world models can be rApps including, without limitation, from multiple vendors. The joint optimizer can be another rApp including from another vendor. The respective world models can be registered to the joint optimizer using the R1 interface, e.g. in an initial step. A network state can be accessed through 01, Al, R1 interfaces by the joint optimizer, for example. The world model predictions and gradients can be communicated to the joint optimizer through Rl. The joint optimizer can query the individual world models on a network state and network parameter, communicated through Rl for example.

[0152] Other examples of the present disclosure can be implemented in a 3GPP network. For example, a network data analytics function (NWDAF) can host the world models as network analytics functions. In a 3 GPP network, in examples that include a search method, the single parameter optimizers also can be hosted in the NWDAF as recommendation systems. The joint optimizer can be hosted in the SMO and can interact with the NWDAF. For example, the joint optimizer can request information from the world models and single parameter optimizers according to Figure 9 for example.

[0153] In examples where there is an optimizer per world model, the search may be accelerated. Such an optimizer can be implemented as a separate rApp, and that the joint optimizer communicates with the respective optimizers per world model. If a respective optimizer is combined with the world model rApp, then the rApp can provide a way to query either a state prediction from the world model or an action recommendation from the optimizer, or both at the same time.

[0154] The computing device can be any apparatus and / or virtual entity in a network configured to perform operations discussed herein. Examples of a computing device include, without limitation, a computer, a server, a wireless device, a network node, a distributed node, a cloud-based node, and / or an 0-RAN node.

[0155] Operations of a computing device can be performed by the computing device 1000 of Figure 10. Operations of the computing device (implemented using the structure of Figure 10) have been discussed with reference to the flow charts of Figures 1 and 2 according to some embodiments of the present disclosure. Operations 100 and 102 of Figure 1 may be optional with respect to some embodiments of computing devices and related methods. Modules may be stored in memory 1004 and / or program code 1008 of Figure 10, for example. Program code 1008 can include program code for performing the operations of Figures 1 and 2. These modules mayprovide instructions so that when the instructions of a module are executed by respective computing device processing circuitry 1002 (also referred to herein as a processor), computing device 1000 performs respective operations of the flow charts of Figures 1 and 2.

[0156] As shown in Figure 10, the computing device 1000 includes processing circuitry 1002 that is operatively coupled to memory 1004, network interface 1006, and / or any other component, or any combination thereof. Certain computing devices may utilize all or a subset of the components shown in Figure 10. Moreover, in some embodiments, components of the computing device may be distributed including, e.g. program code 1008. In some embodiments, the program code includes a learned global world model that is a neural network stored in a graphics processing unit (GPU). The level of integration between the components may vary from one computing device to another computing device. Further, certain computing devices may contain multiple instances of a component, such as multiple processors, memories, etc.

[0157] The processing circuitry 1002 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 1004. The processing circuitry 1002 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processor 802 may include multiple central processing units (CPUs).

[0158] Network interface 1006 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a display, a monitor, a printer, another output device, or any combination thereof. An input device may allow a user to capture information into the computing device 1000. Examples of an input device include a touch-sensitive or presence-sensitive display, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, and the like.

[0159] The memory 1004 may be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 1004 includes one or more application programs in program code 1008, such as the ML model, an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data. The memory 1004 may store,for use by the computing device 1000, any of a variety of various operating systems or combinations of operating systems.

[0160] The memory 1004 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUICC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 1004 may allow the computing device 1000 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 1004 which may be or comprise a device-readable storage medium.

[0161] The processing circuitry 1002 may be configured to communicate with a network using the network interface 1006. The network interface 1006 may comprise one or more communication subsystems. The network interface 1006 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another computing device, local computing device, edge node, cloud node, etc.). Each transceiver may include a transmitter and / or a receiver appropriate to provide network communications (e.g., optical, electrical, and so forth).

[0162] In the illustrated embodiment, communication functions of the network interface 1006 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / internet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

[0163] Although the computing device described herein may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and / or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the computing device, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processor and the network interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

[0164] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer-readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.

[0165] In certain embodiments, a computing device (402, 1000, 1108, 1400) configured to jointly optimize a plurality of network parameters in a telecommunications network with a plurality of world models is provided. The computing device includes processing circuitry (1002); and at least one memory (1004) connected to the processing circuitry and storing program code(1008) that is executed by the processing circuitry to perform operations. The operations include to perform some or all of the functionality described herein.

[0166] Figure 11 shows an example of a communication system 1100 in accordance with some embodiments.

[0167] In the example, the communication system 1100 includes a telecommunications network 1102 that includes an access network 1104, such as a RAN, and a core network 1106, which includes one or more core network nodes 1108. The access network 1104 includes one or more access network nodes or base stations of various types, access network nodes 1110A and 1110B are depicted (which may be collectively referred to as network nodes 1110), or any other similar 3 GPP access nodes or non-3GPP access points (APs). Some embodiments of the access network 1104 may include more than one access network technology. The network nodes 1110 of access network 1104 facilitate direct or indirect connection of wireless devices, also referred to as UEs, such as by connecting UEs 1112 A, 1112B, 1112C, and 1112D (one or more of which may be generally referred to as UEs 1112) to the core network 1106 over one or more wireless connections.

[0168] Moreover, a network node is not necessarily limited to an implementation in which a radio portion and a baseband portion are supplied and integrated by a single vendor. Thus, it will be understood that network nodes include disaggregated implementations or portions thereof. For example, in some embodiments, the telecommunications network 1102 includes one or more ORAN network nodes. An ORAN network node is a network node in the telecommunications network 1102 that supports an ORAN specification (e.g., a specification published by the O-RAN Alliance, or any similar organization) and may operate alone or together with other network nodes to implement one or more functionalities of any network node in the telecommunications network 902, including one or more access network nodes 1110 and / or core network nodes 1108.

[0169] Examples of an ORAN network node include an O-RU, an O-DU), an O-CU, including an O-CU-CP or an O-CU-UP, a RAN intelligent controller (near-real time or non-real time) hosting software or software plug-ins, such as a near-real time control application (e.g., xApp) or a non-real time control application (e.g., rApp), or any combination thereof (the adjective “open” designating support of an ORAN specification). An ORAN network node may support a specification by, for example, supporting an interface defined by the ORAN specification, such as an Al, Rl, Fl, Wl, El, E2, X2, Xn interface, an open fronthaul user plane interface, or an open fronthaul management plane interface. Moreover, an ORAN network node may be a logical node in a physical node. Furthermore, an ORAN network node may be implemented in a virtualization environment (described further below) in which one or more network functions are virtualized.For example, the virtualization environment may include an O-Cloud computing platform orchestrated by a Service Management and Orchestration Framework via an 0-2 interface defined by the 0-RAN Alliance or comparable technologies.

[0170] The network nodes 1110 facilitate direct or indirect connection of one or more UEs 1112 to the core network 1106 over one or more wireless connections. Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 1100 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 1100 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0171] The UEs 1112 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 1110 and other communication devices. Similarly, the network nodes 1108, 1110 are arranged, capable, configured, and / or operable to communicate directly or indirectly (e.g., via other devices of telecommunications network 1102) with the UEs 1112 and / or with other network nodes or equipment in the telecommunications network 1102 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunications network 1102. More specifically, UEs 1112 may send messages, data, and / or other signals to network nodes 1108, 1110 or other elements of the telecommunications network 1102 by transmitting such signals to the relevant device directly without the signals passing through any intervening devices or by transmitting such signals to the relevant device indirectly through an intervening device (or multiple intervening devices) that then transmit the signal to the relevant device. Similarly, network nodes 1108, 1110 may send messages, data, and other signals to UEs 1112, other network nodes 1108, 1110, and other devices in telecommunications network 1102 directly or indirectly. As one specific example, a core network node 1108 may transmit a particular message to a UE 1112 by transmitting the message to an access network node 1110 that will then transmit the message to the intended UE 1112. Similarly, a core network node 1108 may receive a particular message from a UE 1112 by receiving the message from an access network node 910 that itself received the message from the UE 1112.

[0172] In the depicted example, the core network 1106 connects elements of the access network 1104 (e.g., one or more of the network nodes 910) to one or more host computing systems, such as host 1116. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 1106 includes one or more core network nodes (e.g., core network node 1108) of various types, one or more of which may be generally referred to as network nodes 1108. Network nodes 1108 are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, access network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 1108. Example core network nodes provide functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0173] The host 1116 may be under the ownership or control of a service provider other than an operator or provider of the access network 904 and / or the telecommunications network 1102. The host 1116 may be operated by the service provider or on behalf of the service provider. The host 1116 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0174] As a whole, the communication system 1100 of Figure 11 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system 1100 may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); LTE, and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (Wi-Fi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (Wi-Max), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, Li-Fi, and / or any low-power wide-area network (LPWAN)standards such as LoRa and Sigfox. Moreover, the communication system 900 may be configured to support multiple different standards, protocols, or other rule sets, with individual components supporting all of the relevant rule sets or with different components or sub-systems within the communication system 900 supporting different standards, protocols, or rule sets.

[0175] As one example, in certain embodiments, access network 1104 may contain some access network nodes 1110 that support 3GPP radio access technologies (RAT), such as LTE or NR, while other access network nodes 910 support (or the same access network nodes 1110 additionally support) non-3GPP RATs, such as Wi-Fi or a proprietary RAT. As another example, telecommunications network 1102 may support multiple generations of related communication standards (e.g., 4G and 5G 3GPP communication standards) and, as a result, may include an access network 1104 and / or a core network 1106 that supports multiple different standard generations or may include multiple access networks 1104 and / or multiple core networks 1106 with individual networks 1104, 1106 supporting different standard generations.

[0176] Telecommunications network 1102 may support network slicing to provide different logical networks to different devices that are connected to the telecommunications network 1102. For example, the telecommunications network 1102 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC) / Massive loT services to yet further UEs.

[0177] In some examples, one or more of the UEs 1112 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 1104 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 1104. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

[0178] In the example, the hub 1114 communicates with the access network 1104 to facilitate indirect communication between one or more UEs (e.g., UE 1112C and / or 1112D) and network nodes (e.g., network node 1110B). In some examples, the hub 1114 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 1114 may be a broadband router enabling access to the core network 1106 for the UEs. As another example, the hub 1114 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may bereceived from the UEs, network nodes 1110, or by executable code, script, process, or other instructions in the hub 1114.

[0179] As another example, the hub 1114 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 1114 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 1114 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 1114 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 1114 acts as a proxy server or orchestrator for the UEs, in particular if one or more of the UEs are low energy loT devices.

[0180] The hub 1114 may have a constant / persistent or intermittent connection to the network node 1110B. The hub 1114 may also allow for a different communication scheme and / or schedule between the hub 1114 and UEs (e.g., UE 1112C and / or 1112D), and between the hub 1114 and the core network 1106. In other examples, the hub 1114 is connected to the core network 1106 and / or one or more UEs via a wired connection. Moreover, the hub 1114 may be configured to connect to an M2M service provider over the access network 1104 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 1110 while still connected via the hub 1114 via a wired or wireless connection. In some embodiments, the hub 1114 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 1110B. In other embodiments, the hub 1114 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 1110B, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0181] Figure 12 is another example of a communication system 1200 according to some embodiments. As used herein, the communication system 1200 includes multiple access points (APs) 1210 (with four exemplary APs 1210A, 1210B, 1210C, and 1210D being depicted) and multiple wireless devices, referred to in the context of communication system 1200 as stations (STAs) 1212 (referred to individually as STA 1212A, STA 1212B, STA 1212C, STA 1212D, and STA 1212E). STA 1212A is served by AP 1210A in a first basic service set (BSS) 1220A. STA 1210B and STA 1210C are served by AP 1210B in a second BSS, BSS 1220B. STA 1212D is served by AP 1210C in a third BSS, BSS 1220C. STA 1212E is served by AP 1210D in a fourth BSS, BSS 1220D. Stations 1212 may be non-AP STAs and correspond to various kinds of wireless devices, for example, user terminals, such as mobile or stationary computing devices likesmartphones, laptop computers, desktop computers, tablet computers, gaming devices, headmounted displays (HMDs) for Augmented Reality (AR) or Virtual Reality (VR), or the like. Further, stations 1212 could, for example, correspond to other kinds of equipment like smart home devices, printers, multimedia devices, data storage devices, or the like.

[0182] Each of STAs 1212 may connect through a radio link to one of APs 1210. For example, depending on location or channel conditions experienced by a given STA 1212, the STA may select an appropriate AP and BSS for establishing the radio link. The radio link may be based on one or more orthogonal frequency-division multiplexing (OFDM) carriers from a frequency spectrum that is shared on the basis of a contention-based mechanism, e.g., an unlicensed or license exempt band like 2.4 GHz Industrial, Scientific, and Medical (ISM) band, the 5 GHz band, the 6 GHz band, or the 60 GHz band.

[0183] Each AP 1210 may provide data connectivity to STAs 1212 connected to a particular AP 1210. As illustrated, APs 1210 may be connected to a data network 1230. In this way, APs 1210 may also provide data connectivity between STAs 1212 and other entities, e.g., to one or more servers, service providers, data sources, data sinks, user terminals, or the like. Accordingly, the radio link established between a given STA 1212 and its serving AP 1210 may be used for providing various kinds of services to STA 1212, e.g., a voice service, a multimedia service, or other data service. Such services may be based on applications that are executed on STA 1212 and / or on a device linked to STA 1212. By way of example, Figure 12 illustrates an application service platform 1232 provided in data network 1230. The application(s) executed on STA 1212 and / or on one or more other devices linked to STA 1212 may use the radio link for data communication with one or more other STA 1212 and / or the application service platform 1232, thereby enabling utilization of the corresponding service(s) at STA 1212.

[0184] Further definitions and embodiments are discussed below.

[0185] In the above-description of certain embodiments of the present disclosure, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which concepts of the present disclosure belong. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0186] When an element is referred to as being “connected”, “coupled”, “responsive”, or variants thereof to another element, it can be directly connected, coupled, or responsive to the other element or intervening elements may be present. In contrast, when an element is referred to as being “directly connected”, “directly coupled”, “directly responsive”, or variants thereof to another element, there are no intervening elements present. Like numbers refer to like elements throughout. Furthermore, “coupled”, “connected”, “responsive”, or variants thereof as used herein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and / or clarity. The term “and / or” (abbreviated “ / ”) includes any and all combinations of one or more of the associated listed items.

[0187] It will be understood that although the terms first, second, third, etc. may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Thus a first element / operation in some embodiments could be termed a second element / operation in other embodiments without departing from the teachings of concepts of the present disclosure. The same reference numerals or the same reference designators denote the same or similar elements throughout the specification.

[0188] As used herein, the terms “comprise”, “comprising”, “comprises”, “include”, “including”, “includes”, “have”, “has”, “having”, or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components, or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions, or groups thereof. Furthermore, as used herein, the common abbreviation “e.g ”, which derives from the Latin phrase “exempli gratia,” may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item. The common abbreviation “i.e ”, which derives from the Latin phrase “id est,” may be used to specify a particular item from a more general recitation.

[0189] Example embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, apparatus (systems and / or devices) and / or computer program products. It is understood that a block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and / or otherprogrammable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and / or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions / acts specified in the block diagrams and / or flowchart block or blocks, and thereby create means (functionality) and / or structure for implementing the functions / acts specified in the block diagrams and / or flowchart block(s).

[0190] These computer program instructions may also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions / acts specified in the block diagrams and / or flowchart block or blocks. Accordingly, embodiments of the present disclosure may be embodied in hardware and / or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as “circuitry,” “a module” or variants thereof.

[0191] It should also be noted that in some alternate implementations, the functions / acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality / acts involved. Moreover, the functionality of a given block of the flowcharts and / or block diagrams may be separated into multiple blocks and / or the functionality of two or more blocks of the flowcharts and / or block diagrams may be at least partially integrated. Finally, other blocks may be added / inserted between the blocks that are illustrated, and / or blocks / operations may be omitted without departing from the scope of the present disclosure. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[0192] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present disclosure. All such variations and modifications are intended to be included herein within the scope of present disclosure. Accordingly, the above disclosed subject matter is to be considered illustrative, and not restrictive, and the examples of embodiments are intended to cover all such modifications, enhancements, and other embodiments, which fall within the spirit and scope of the present disclosure. Thus, to the maximum extent allowed by law, the scope of the present disclosure is to be determined by the broadest permissible interpretation of the present disclosure including the examples ofembodiments and their equivalents, and shall not be restricted or limited by the foregoing detailed description.

Claims

Claims:

1. A computer-implemented method performed by a computing device to jointly optimize a plurality of network parameters in a telecommunications network with a plurality of world models, the method comprising:determining (104) a change to a plurality of network parameters based on querying respective individual world models in the plurality of world models to find a value of respective network parameters that result in a desired network state; andcommunicating (106), towards a network node, the change to the plurality of network parameters.

2. The computer-implemented method of Claim 1, wherein determining (104) the change comprises determining a change to at least one of (i) all of the plurality of network parameters and (ii) a subset of the plurality of network parameters, the subset comprising greater than one network parameter.

3. The computer-implemented method of any one of Claims 1 to 2, wherein determining (104) the change comprises using gradient information exchanged across the respective individual world models.

4. The computer-implemented method of Claim 3, wherein using gradient information comprisescombining the respective individual world models in an order,accessing an initial network state,taking an actual value for the respective network parameters,calling the respective world models with the initial network state and the actual value for the respective network parameter to obtain a respective estimated network state and gradient, and applying a backward pass to update the actual value of the respective network parameters.

5. The computer-implemented method of any one of Claims 1 to 2, wherein a respective world model is associated with a respective optimizer and models a respective impact of a change to a respective network parameter on a network state, and wherein determining (104) the change comprisesquerying a respective optimizer to obtain a respective change of a respective network parameter, andaccessing, from a respective world model, an estimate of a future network state when the respective change of the respective network parameter is applied.

6. The computer-implemented method of Claim 5, wherein the accessing comprises a search with an input comprising a current network state and an identity of the respective world models and respective optimizers and an output comprising an identity of the respective changes to the respective network parameters.

7. The computer-implemented method of Claim 6, wherein the search comprises a search of a tree comprising a plurality of nodes that respectively correspond to respective network states and a plurality of branches that respectively correspond to respective optimizers.

8. The method of any one of Claims 5 to 7, wherein the querying comprises iteratively selecting a respective world model, andquerying the respective optimizer associated with the respective world model for the respective change to a respective network parameter.

9. The computer-implemented method of any one of Claims 1 to 8, further comprising: determining (100) an order in which the plurality of world models is composed.

10. The computer-implemented method of Claim 9, wherein the respective network states overlap in at least a subset of the respective world models.

11. The computer-implemented method of any one of Claims 1 to 10, wherein the change to the plurality of network parameters comprises a change to respective configuration management, CM, parameters,wherein the network state comprises a plurality of key performance indicators, KPIs, and wherein a respective world model provides a respective estimate of the network state when the respective change to the respective CM parameters is applied.

12. The computer-implemented method of any one of Claims 1 to 11, further comprising: evaluating (100) an impact of changing the plurality of network parameters based on sequentially calling the respective world models with the network state modified between each call to reflect the estimated impact on the network state of changing the plurality of network parameters.

13. The computer-implemented method of any one of Claims 1 to 12, wherein the telecommunications network comprises an open-radio access network, O-RAN, the computingdevice comprises at least one rApp, and the respective individual world models comprise respective additional rApps.

14. A computer-implemented method performed by a computing device to learn a global world model to provide a joint network parameter change for a network state in a telecommunications network, the method comprising:accessing (200) a dataset from respective individual world models in a plurality of world models, the dataset comprising a first network state, a plurality of respective network parameter changes for the respective individual world models, and an estimate of a second network state when the respective network parameter changes are applied;learning (202) the global world model from the dataset to obtain a learned global world model; andusing (204) the learned global world model to provide a joint network parameter change for a given network state.

15. The computer-implemented method of Claim 14, wherein the learning (202) comprises supervised learning from the dataset.

16. The computer-implemented method of any one of Claims 14 to 15, wherein the learning (202) is triggered responsive to availability of a new individual world model in the plurality of world models.

17. The computer-implemented method of any one of Claims 14 to 16, wherein using (204) the learned global world model comprises finding an optimal joint network parameter change from the plurality of joint network changes.

18. The computer-implemented method of any one of Claims 14 to 17, wherein using (204) the learned global world model comprises taking a joint gradient over the learned global world model with respect to respective network parameter changes being optimized.

19. The computer-implemented method of Claim 18 wherein using (204) further comprises applying an incremental update for respective network parameter changes based on the joint gradient to obtain an updated joint parameter change.

20. The computer-implemented method of any one of Claims 14 to 19, wherein the first network state comprises a plurality of current key performance indicators, KPIs,wherein the initial joint network parameter change comprises a respective initial configuration management, CM, parameter for the respective individual world models, andwherein the plurality of estimates of the network state comprise respective estimates of a change in the plurality of KPIs based on the plurality of current KPIs and a respective change to the respective initial CM parameters.

21. The computer-implemented method of any one of Claims 14 to 20, wherein the learned global world model comprises a neural network.

22. A computing device (402, 1000, 1108, 1400) configured to jointly optimize a plurality of network parameters in a telecommunications network with a plurality of world models, the computing device comprising:processing circuitry (1002);at least one memory (1004) connected to the processing circuitry and storing program code (1008) that is executed by the processing circuitry to perform operations comprising:determine a change to a plurality of network parameters based on querying respective individual world models in the plurality of world models to find a value of respective network parameters that result in a desired network state; andcommunicate, towards a network node, the change to the plurality of network parameters.

23. The computing device of Claim 22, wherein the at least one memory (1004) is connected to the processing circuitry (1002) and stores program code (1008) that is executed by the processing circuitry to perform operations according to any one of Claims 2 to 13.

24. A computing device (402, 1000, 1108, 1400) configured to learn a global world model to provide a joint network parameter change for a network state in a telecommunications network, the computing device comprising:processing circuitry (1002);at least one memory (1004) connected to the processing circuitry and storing program code (808) that is executed by the processing circuitry to perform operations comprising:access a dataset from respective individual world models in a plurality of world models, the dataset comprising a first network state, a plurality of respective network parameter changes for the respective individual world models, and an estimate of a second network state when the respective network parameter changes are applied;learn the global world model from the dataset to obtain a learned global world model; and use the learned global world model to provide a joint network parameter change for a given network state.

25. The computing device of Claim 24, wherein the at least one memory (1004) is connected to the processing circuitry and stores program code (1008) that is executed by the processing circuitry to perform operations according to any one of Claims 14 to 21.

Images