Cell parameter management in a communication network

Abstract

A method (200) is disclosed for training a policy ML model to manage a parameter in a cell of a communication network. The method comprises obtaining a training data set comprising, for a plurality of cells, CTR data and a corresponding value of the parameter (210), and converting the CTR data to histograms (220). The method also comprises using the training data set to simultaneously train (230) a CNN to generate a reduced dimensionality latent representation of the histograms, and a DNN to predict values of the parameter from an input comprising the latent representations generated by the CNN. The method also comprises using a Reinforcement Learning process to train a policy ML model to generate control actions for the parameter in a cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN.

Description

[0001] Cell Parameter Management in a Communication Network

[0002] Technical Field

[0003] The present disclosure relates to a computer implemented method for training a policy Machine Learning (ML) model to manage a parameter in a cell of a communication network. The present disclosure also relates to a computer implemented method for managing a parameter in a cell of a communication network. The methods are performed by a training node and a policy node respectively, and the present disclosure also relates to a training node, a policy node, and to a computer program product configured, when run on a computer, to carry out methods fortraining a policy ML model to manage a parameter in a cell of a communication network, and for managing a parameter in a cell of a communication network.

[0004] Background

[0005] A Reinforcement Learning (RL) scheme that relies on a Deep Neural Network (DNN) is also referred to as Deep Reinforcement Learning (DRL). WO 2021 / 190772 proposes a DRL scheme that is suitable for optimising various cell parameters in an automatic way, with an agent automatically learning the optimal management policy from network statistics, without needing human intervention. The scheme proposed in WO 2021 / 190772 has proven to work very efficiently in optimising cell parameters that impact the Key Performance Indicators (KPIs) of the cell in question, but also the KPIs of neighbouring cells. While initially tested for optimisation of Remote Electrical Tilt (RET), this approach has also been used successfully to optimize the PONominal Physical Uplink Shared Channel (PUSCH) parameter for uplink Power Control (PC), and could be applied to the optimisation of many other parameters. However, onboarding a new parameter to this scheme requires advanced engineering work to be carried out by expert technicians to define the most appropriate state and reward definitions.

[0006] In the case of RET, the engineering work for state design was inspired by features previously designed for a RET optimisation module. These features are computed from periodic measurements obtained from Call Trace Recording (CTR) files using complex and elaborated formulae. Counters generated from CTR using such complex formulae are customized for a particular purpose (in this case, RET optimisation). Unlike Performance Management (PM) counters, they are not available for operators through regular interfacing tools. For this reason, and to differentiate them from regular PM counters, they are referred to herein as “synthetic counters”.

[0007] The design of the features that comprise the state for the RL scheme that optimizes the parameter PONominal PUSCH also required extensive research, resulting into new algorithms described in WO 2023 / 209428.

[0008] While the RL approach proposed in WO 2021 / 190772 is suitable to optimize many parameters in an automatic way, the approach requires a considerable initial investment for the design of the state and the reward, which should be typically based on PM data and CTR data. CTR data is particularly important, as it contains very valuable information, but that information is in a raw format that requires extensive postprocessing using procedures that have been carefully designed during a preceding engineering phase. In principle, the process of reward definition should involve less complexity, as it is only required to encapsulate the objectives. However, the state, which is required to reflect current conditions in a cell, and to convey relevant information that may only be available from the cell in specific formats, is much more convoluted to design, requiring extensive human input from domain experts.

[0009] It is an aim of the present disclosure to provide methods, a training node, and a computer program product which at least partially address one or more of the challenges discussed above. It is a further aim of the present disclosure to provide methods, a training node and a computer program product that cooperate to achieve automated design of a cell state for an RL agent, the state encompassing the most relevant information for management of a given cell parameter.

[0010] The invention of the present disclosure is defined in the independent claims, to which reference should now be made. Further features are set out in the dependent claims.

[0011] According to a first aspect of the present disclosure, there is provided a computer implemented method for training a policy Machine Learning (ML) model to manage a parameter in a cell of a communication network. The method, performed by a training node, comprises (i) obtaining a training data set. The training data set comprises, for each of a plurality of cells in the communication network, and for time instances during a period of operation of the communication network, records of a representation of a state of the cell, the representing including Call Trace Recording (CTR) data, and a corresponding value of the parameter. The method further comprises (ii) converting the CTR data from the state representations to histograms. The method further comprises (iii) using the training data set to simultaneously train (a) a Convolutional Neural Network (CNN) to generate a latent representation of the histograms, the latent representation having a reduced dimensionality with respect to the histograms, and (b) a Deep Neural Network (DNN) to predict values of the parameter from an input comprising the latent representations generated by the CNN. The method further comprises (iv) using a Reinforcement Learning (RL) process to train a policy ML model to generate control actions for the parameter in a cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN.

[0012] According to another aspect of the present disclosure, there is provided a computer implemented method for managing a parameter in a cell of a communication network. The method, performed by a policy node, comprises (i) obtaining Call Trace Recording (CTR) data, for the cell and a current value of the parameter in the cell, and (ii) converting the CTR data to a histogram. The method further comprises (iii) using a trained CNN to generate a latent representation of the histogram, the latent representation having a reduced dimensionality with respect to the histogram. The method further comprises (iv) using a trained policy ML model to generate a control action for the parameter in the cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises the latent representation generated by the trained CNN. The method further comprises (v) causing the generated control action to be implemented in the cell.

[0013] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer readable medium, the computer readable medium having computer readable code embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform a method according to any one or more of the aspects or examples of the present disclosure. According to another aspect of the present disclosure, there is provided a training node for training a policy ML model to manage a parameter in a cell of a communication network. The training node comprises processing circuitry configured to cause the training node to (i) obtain a training data set. The training data set comprises, for each of a plurality of cells in the communication network, and for time instances during a period of operation of the communication network, records of a representation of a state of the cell, the representing including Call Trace Recording (CTR) data, and a corresponding value of the parameter. The processing circuitry is further configured to cause the training node to (ii) convert the CTR data from the state representations to histograms. The processing circuitry is further configured to cause the training node to (iii) use the training data set to simultaneously train (a) a Convolutional Neural Network (CNN) to generate a latent representation of the histograms, the latent representation having a reduced dimensionality with respect to the histograms, and (b) a Deep Neural Network (DNN) to predict values of the parameter from an input comprising the latent representations generated by the CNN. The processing circuitry is further configured to cause the training node to (iv) use a Reinforcement Learning (RL) process to train a policy ML model to generate control actions for the parameter in a cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN.

[0014] According to another aspect of the present disclosure, there is provided a policy node for managing a parameter in a cell of a communication network. The policy node comprises processing circuitry configured to cause the policy node to (i) obtain Call T race Recording (CTR) data, for the cell and a current value of the parameter in the cell, and (ii) convert the CTR data to a histogram. The processing circuitry is further configured to cause the policy node to (iii) use a trained CNN to generate a latent representation of the histogram, the latent representation having a reduced dimensionality with respect to the histogram. The processing circuitry is further configured to cause the policy node to (iv) use a trained policy ML model to generate a control action for the parameter in the cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises the latent representation generated by the trained CNN. The processing circuitry is further configured to cause the policy node to (v) cause the generated control action to be implemented in the cell. Aspects of the present disclosure thus provide methods and nodes that facilitate automatic generation of a state representation for use in RL management of cell parameters. The automatically generated state representation encapsulates the most meaningful and relevant information for optimising a given cell parameter, using CTR data, and in some examples additional information including cell metrics such as PM counters.

[0015] Brief Description of the Drawings

[0016] For a better understanding of the present disclosure, and to show more clearly how it may be carried into effect, reference will now be made, by way of example, to the following drawings in which:

[0017] Figure 1 is a schematic representation of RL management of the RET parameter in a cell;

[0018] Figure 2 is a flow chart illustrating process steps in a method for training a policy ML model to manage a parameter in a cell of a communication network;

[0019] Figures 3a to 3d show flow charts illustrating process steps in another example of a method for training a policy ML model to manage a parameter in a cell of a communication network;

[0020] Figure 4 is a flow chart illustrating process steps in a method for managing a parameter in a cell of a communication network;

[0021] Figures 5a and 5b show flow charts illustrating process steps in another example of a method for managing a parameter in a cell of a communication network;

[0022] Figure 6 is a block diagram illustrating functional modules in an example training node;

[0023] Figure 7 is a block diagram illustrating functional modules in an example policy node;

[0024] Figure 8 is a schematic illustration of the generation of 4D histograms;

[0025] Figure 9 illustrates simultaneous training of a CNN and DNN to predict parameter values; Figure 10 illustrates SHAP analysis of a trained DNN;

[0026] Figure 11 illustrates training of an ML policy model;

[0027] Figure 12 illustrates variable down sampling for TA values; and

[0028] Figure 13 illustrates implementation of the methods disclosed herein in an O-RAN architecture.

[0029] Detailed

[0030] Example methods according to the present disclosure propose an automated process for defining a state for use by policy ML models, which may be in the form of DNN-based RL agents, that optimize cell parameters in wireless cellular networks. The state encapsulates information from CTR data, and may additionally use cell metrics including PM and Configuration Management (CM) data. The process includes a training phase in which CTR data is processed, and multidimensional (e.g. 4-dimensional) histograms are generated per cell. A Convolutional Neural Network (CNN) and a Deep Neural Network (DNN) are then simultaneously trained to jointly predict the current value of the parameter that is to be optimised by the policy ML model. The CNN works as a dimensionality reduction module to decrease the size of the (4D) histograms per cell, while keeping the information that is most relevant to predicting the value of the cell parameter (and consequently will be most relevant for management of that parameter by the policy ML model). The DNN may also be used to detect the most relevant cell metrics such as PM counters for inclusion in the cell state using a feature importance analysis such as a SHapley Additive exPlaination (SHAP) analysis or L1 regularization. The most relevant PM counters may be selected as those with the highest contribution to the prediction of the current value of the parameter to optimize. Finally in the training phase, a policy ML model is trained using a state definition based on the results of the previous steps, that is using CTR data encoded using the trained CNN, and in some examples also cell metrics selected as being most relevant. In an inference phase, the trained CNN and selected cell metrics are used to generate inputs to the trained policy ML model for management of the cell parameter in the network. Figure 1 is a schematic representation of RL management of the Remote Electrical Tilt (RET) parameter in a cell. During an initial training phase of the RL agent, the agent interacts with a simulation of the network, so as to avoid unacceptable performance degradation as the agent explores the state action space, and learns an optimal policy for RET management. The agent receives a representation of the state S? of the relevant cell, which may indicate one or more problems such as congestion, overlapping, interference, overshooting, low Reference Signal Received Power (RSRP) at cell edge, etc. The agent selects an action A to be implemented on the RET in the cell (increasing, decreasing, or maintaining the tilt angle), and this action is implemented in the simulation, or in the live network during inference. The impact of the action Atis assessed via a reward function, which reflects how KPIs such as SINR, RSRP, congestion etc. have been affected by the implemented action. The reward Rtis then fed back to the agent and used to update the policy for selecting a next action At+1on the basis of an updated cell state representation. As discussed above, a wealth of data is available about the current state of a communication network cell, and identifying the most relevant information for management of a given cell parameter is a problem that has previously required extensive engineering by domain experts.

[0031] Figure 2 is a flow chart illustrating process steps in a computer implemented method 200 for training a policy Machine Learning (ML) model to manage a parameter in a cell of a communication network. The method 200 is performed by a training node, which may comprise a physical or virtual node, and may be implemented in a computer system, computing device or server apparatus and / or in a virtualized environment, for example in a cloud, edge cloud or fog deployment. Examples of a virtual node may include a piece of software or computer program, a code fragment operable to implement a computer program, a virtualised function, or any other logical entity. The training node may for example be implemented in a core network of the communication network, and may be implemented in the Operation Support System (OSS). The training node may be implemented in an Orchestration And Management (OAM) system or in a Service Management and Orchestration (SMO) system. In other examples, the training node may be implemented in a Radio Access node, which itself may comprise a physical node and / or a virtualized network function that is operable to exchange wireless signals. In some examples, a Radio Access node may comprise a base station node such as a NodeB, eNodeB, gNodeB, or any future implementation of this functionality. The training node may be implemented as a function in an Open Radio Access Network (ORAN) or Virtualised Radio Access Network (vRAN). The training node may encompass multiple logical entities, as discussed in greater detail below, and may for example comprise a Virtualised Network Function (VNF).

[0032] Referring to Figure 2, the method 200 comprises, in a first step 210, obtaining a training data set. The training data set comprises, for each of a plurality of cells in the communication network, and for time instances during a period of operation of the communication network, records of (1) a representation of a state of the cell, the representing including Call Trace Recording (CTR) data, and (2) a corresponding value of the parameter. It will be appreciated that a value of the parameter “corresponds” to a given cell state representation in that the value and cell state representation are valid for the same time instance. Each data sample in the training data set therefore comprises a pair of (1) cell state representation and (2) parameter value, and each pair is representative of conditions for a different instance in time during the period of operation covered by the training data set. The method 200 then comprises, in step 220, converting the CTR data from the state representations to histograms. In step 230, the method 200 comprises using the training data set to simultaneously train a CNN and a DNN. The CNN is trained to generate a latent representation of the histograms, the latent representation having a reduced dimensionality with respect to the histograms. The DNN is trained to predict values of the parameter from an input comprising the latent representations generated by the CNN. In step 240, the method 200 comprises using a RL process to train a policy ML model to generate control actions for the parameter in a cell from an input comprising a representation of the state of the cell. The cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN. The control actions may include any adjustment of the parameter value, such as an incremental increase, decrease, or maintenance of the parameter value at its current level. The control actions may further include to modify the value of the parameter directly to one of a range of valid values.

[0033] The method 200 thus achieves the automated generation of a state representation for use in RL control of a cell parameter, the generated state representation including information from CTR data. The method 200 avoids the time consuming and difficult manual engineering of CTR data to generate relevant synthetic counters for a given parameter for optimisation, achieving a relevant state representation through a data driven process. The simultaneous training of the encoder CNN and prediction DNN result in a CNN capable of encoding CTR data in such a manner that the most relevant information to the given parameter is maintained. This processing of CTR data through a CNN is made possible by the conversion of the CTR data into histograms, so resembling the matrix representations of images with which CNNs are more regularly used.

[0034] In some examples of the present disclosure, the step 240 of using an RL process to train a policy ML model to generate control actions for the parameter in a cell may be carried out via a series of sub steps. Such sub steps may include obtaining CTR data, generating histograms of the CTR data, encoding the generated histograms using the trained CNN, inputting the encoded CTR data to the policy ML model which then generates a control action, executing the control action in the cell, obtaining a value of a reward function reflecting an impact of the control action on performance of the cell, and updating trainable parameters of the policy ML model. These steps may for example be performed on a simulation or digital twin of the cell initially, to avoid performance degradation of the cell in the early stages of training the policy ML model.

[0035] The method 200, and methods described below, train and use ML models. For the purposes of the present disclosure, the term “ML model” encompasses within its scope the following concepts: machine learning algorithms, comprising processes or instructions through which data may be used in a training process to generate a model artefact for performing a given task, or for representing a real-world process or system; and the model artefact that is created by such a training process, and which comprises the computational architecture that performs the task.

[0036] Figures 3a to 3d show flow charts illustrating process steps in a further example of method 300 for training a policy Machine Learning, ML, model to manage a parameter in a cell of a communication network. The method 300 provides various examples of how the steps of the method 200 may be implemented and supplemented to achieve the above discussed and additional functionality. As for the method 200, the method 300 is performed by a training node, which may be a physical or virtual node, and which may encompass multiple logical entities, as discussed in greater detail above with reference to Figure 2.

[0037] Referring to Figure 3a, in a first step 310 of the method 300, the training node obtains a training data set comprising, for each of a plurality of cells in the communication network, and for time instances during a period of operation of the communication network, records of a representation of a state of the cell, the representation including CTR data, and a corresponding value of the parameter. The plurality of cells may include 2, 3, 4 or more cells. The cells may be neighbouring cells, with substantially overlapping or contiguous coverage areas. In some examples, the plurality of cells may be under the control of a single logical entity. As discussed above, a value of the parameter “corresponds” to a given cell state representation in that the value and cell state representation are valid for the same time instance. Each data sample in the training data set therefore comprises a pair of (1) cell state representation and (2) parameter value, and each pair is representative of conditions for a different instance in time during the period of operation covered by the training data set. The parameter may comprise any one or more parameters relating to the cell, and which are operable for control. Such parameters may include:

[0038] Remote electrical tilt (with passive antennas)

[0039] Digital tilt (with AAS - Advance Antenna System) Maximum cell downlink transmission power

[0040] - Cell-specific Reference Signal (CRS) Power boost

[0041] P0 nominal PUSCH (for uplink power control)

[0042] P0 nominal PUCCH (for uplink power control)

[0043] - Alpha (for uplink power control)

[0044] Corresponding control actions to be generated by the policy ML model may include incremental increase or decrease of any of the above parameter values, maintenance of the parameter value at its current level, or modifying the parameter value directly to one of a range of valid values. For example, a control action may be to increase remote electrical tilt by 1 degree, decrease it by 1 degree or maintain the current value. Another control action may be to increase maximum downlink cell transmission power by 1 decibel, decrease it by 1 decibel or maintain the current value.

[0045] As illustrated at 310a, the training data set may further comprise cell metrics, which may include Performance Management (PM) counters, and may additionally include Configuration Management (CM) parameters. In further examples, additional parameters may also be included in the training data set, such as cell topology.

[0046] In step 320, the training node converts the CTR data from the state representations to histograms. As illustrated at 320a, converting the CTR data from the state representations to histograms may comprise, for records of CTR data for individual cells over a measurement time period, generating a histogram comprising, for each of a plurality of metrics included in the CTR data, reported values of the metrics and a count of the number of devices reporting each value of the metrics during the measurement time period. In some examples, the metrics for which counts of reported values are generated may include Serving cell Reference Signal Received Power (RSRP), Timing Advance (TA), Physical Cell ID (PCI) of a neighbouring cell detected and reported by the UE, and Neighbouring cell RSRP. In some examples, the measurement time period may be that covered by a single record, an hour, a day, a busy period during a day, etc. The period of operation of the communication network for which the training data set is obtained may consequently encompass multiple distinct measurement time periods, for example multiple measurement time periods of a day, over a period of operation of the communication network that comprises several weeks. Within each measurement time period, the training data set may comprise multiple individual data samples corresponding to individual time instances during the measurement time period, each data sample comprising a pair of cell state representation (including CTR data) at that time instance and parameter value at that time instance.

[0047] Following the generation of histograms of CTR data at step 320, the training node then, in step 325, reduces a size of the histograms by, for individual metrics represented in the histogram, performing at least one of reducing a range of metric values represented in the histogram (325a), reducing a resolution of metric values represented in the histogram (325b), and / or delta encoding of metric values with respect to a reference metric value (325c). Different size reduction actions, or combinations of actions, may be carried out for different metrics. For example, the histogram dimension for TA data may be reduced in resolution by down sampling, and / or the histogram dimension for PCI of a reported neighbour cell may be reduced in resolution by resolving to a lower resolution metric (e.g. PCI to Global Cell ID (GCID) mapping). The reference metric value for delta encoding may be that of neighbour cell values with respect to serving cell values (e.g. delta encoding of neighbour cell RSRP with respect to service cell RSRP). In further examples, neighbour RSRP values may be delta encoded and then filtered to maintain only a subset of the possible range, with range reduction based on a relevance criterion. This is discussed in further detail below, with reference to an example implementation of the methods presented herein.

[0048] Referring now to Figure 3b, in step 330, the training node uses the training data set to simultaneously train a CNN and a DNN. The CNN is trained to generate a latent representation of the histograms, the latent representation having a reduced dimensionality with respect to the histograms, and the DNN is trained to predict values of the parameter from an input comprising the latent representations generated by the CNN. In some examples, as illustrated in step 330, the DNN may be trained to predict values of the parameter from an input comprising the latent representations generated by the CNN and the cell metrics including PM counters, and in some examples CM parameters and other cell information such as topology information. The simultaneous training of the CNN and DNN enables the CNN to learn how to encode the CTR data into latent representation that maintain the most important information for predicting the value of a given cell parameter. This information may be inferred to also be the most important information for managing that parameter, and so for input to an ML policy model that will manage the parameter. It will be appreciated that with the inclusion of cell metrics such as PM counters in the input to the DNN, the CNN learns to encode not just the most important CTR information for a given parameter, but also to take account of the information that is provided by the PM counters, and so maintain in the encoded CTR data the information that is both most important for parameter value prediction and also is not conveyed by the PM counters.

[0049] In some examples of the method 300, as illustrated at 330a, the policy ML model may be for managing a composite parameter comprising a plurality of sub parameters, each of which is operable to be controlled independently. In such examples, the simultaneous training at step 330 may comprise training the CNN as above and training the DNN to predict values of each of the plurality of sub parameters from an input comprising the latent representations generated by the CNN.

[0050] Steps that may be performed in order to carry out the simultaneous training at step 330 are illustrated in Figures 3c and 3d. Referring now to Figure 3c, in order to carry out the simultaneous training, the training node initially inputs a histogram to the CNN in step 331 , the CNN being operable to process the histogram according to current values of its trainable parameters, and to output a latent representation of the histogram. The training node then, in step 332, inputs to the DNN the generated latent representation output by the CNN, as well as PM counters for the measurement time period covered by the data in the histogram, and in some examples CM parameters, cell topology information, etc. The DNN is operable to process the input latent representation, PM counters, and optional additional inputs according to current values of its trainable parameters, and to output a predicted value of the parameter (or plurality of sub parameters). In step 333, the training node then calculates a loss function according to a difference between the predicted value of the parameter (or sub parameters) and a measured value of the parameter (or sub parameters) for the measurement time period covered by the histogram in the training data set. In step 334, the training node then updates values of the trainable parameters of the CNN and DNN according to the calculated loss function. The loss function may comprise any suitable function, including for example Root Mean Square (RMS). The updating of parameters may be accomplished using for example backpropagation and gradient descent. Referring now to Figure 3d, in step 335, the training node checks for fulfilment of a termination condition, which may be any suitable condition for assessing convergence towards final values for the trainable parameters. This may for example be a threshold value for the loss function, a time condition, etc. If the termination condition is not fulfilled, the training node returns to step 331 and continues the training process with a next data sample. If the termination condition is fulfilled, the training node proceeds to a next step.

[0051] In some examples the training node may perform at least one additional iteration of the simultaneous training steps 331 to 335, in which the CNN is trained to generate a latent representation having a different reduced dimensionality with respect to the histograms. In this manner, a plurality of CNNs may be trained, each trained to generate a latent representation having a different dimensionality, each dimensionality being lower than that of the original histograms. In such examples, the training node may check, in step 336, whether another CNN should be trained to produce a latent representation of a different dimension. If yes, then the training node returns to step 331 and starts training the CNN architecture for the new dimension latent representation. If no additional CNN is to be trained, either because only one CNN is to be trained, or because all CNNs have been trained, the training node proceeds to step 337.

[0052] In step 337, the training node selects either the trained CNN from the first iteration of the simultaneous training, or the trained CNN from one or more additional iterations of the simultaneous training, to use for training the policy ML model in step 340. If only one CNN has been trained, then this step may be omitted. As illustrated at 337a, selecting a trained CNN may comprise identifying the trained CNN that fulfils a selection criterion balancing data size of the latent representation and prediction precision of the trained DNN. Steps 336 and 337 thus enable multiple versions of the CNN architecture to be trained to generate latent representations of different dimensionality, allowing for selection of an optimum dimensionality for the encoded CTR data. It will be appreciated that a higher dimensionality latent representation may promote greater prediction precision in the trained DNN (and may consequently support improved performance of the policy ML model), but incurs a cost in time and compute power. The selection criterion could for example identify the minimum size of latent representation that achieves a certain improvement in prediction precision when compared to not using a CNN.

[0053] Referring again to Figure 3b, following the simultaneous training of the CNN and DNN, the training node then, in step 338, performs a feature importance analysis on the trained DNN to identify a subset of cell metrics whose contribution to the output of the DNN exceeds an importance threshold. In some examples in which CM parameters were included in the input for the DNN, the feature importance analysis may also be carried out on the CM parameters. The feature importance analysis identifies those cell metrics (PM counters and in some examples CM parameters) that are the most relevant to prediction of a parameter value, and consequently will be most important for managing the parameter via the policy ML model. The feature importance analysis may for example comprise a SHAP analysis or an L1 regularization.

[0054] In step 340, the training node uses an RL process to train a policy ML model to generate control actions for the parameter in a cell from an input comprising a representation of the state of the cell. The cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN, and may also comprise cell metrics for the cell, such as PM counters and / or CM parameters, and / or additional cell information such as cell topology information. The cell metrics included in the state representation may comprise the subset of cell metrics identified at step 338 as providing a contribution to the output of the DNN that exceeds an importance threshold. As illustrated at 340c, the CNN used to generate the latent representation may be that selected at step 337, in the event that multiple CNNs were trained in the simultaneous training step 330.

[0055] As illustrated at 340a, the cell state representation input to the policy ML model may additionally comprise a current value of the parameter.

[0056] As discussed above, the policy ML model may be for managing a composite parameter comprising a plurality of sub parameters, each of which is operable to be controlled independently. In such examples as illustrated at 340b, step 340 may comprise training the policy ML model to generate control actions for each of the sub parameters in the cell.

[0057] The methods 200 and 300 may be complemented by a method 400 performed by a policy node, in which method the trained policy ML model is used to manage a cell parameter in a cell of a communication network.

[0058] Figure 4 is a flow chart illustrating process steps in a computer implemented method 400 for managing a parameter in a cell of a communication network. The method 400 is performed by a policy node, which may comprise a physical or virtual node, and may be implemented in a computer system, computing device or server apparatus and / or in a virtualized environment, for example in a cloud, edge cloud or fog deployment. Examples of a virtual node may include a piece of software or computer program, a code fragment operable to implement a computer program, a virtualised function, or any other logical entity. The policy node may for example be implemented in a core network of the communication network. In other examples, the policy node may be implemented in a Radio Access node, which itself may comprise a physical node and / or a virtualized network function that is operable to exchange wireless signals. In some examples, a Radio Access node may comprise a base station node such as a NodeB, eNodeB, gNodeB, or any future implementation of this functionality. The policy node may be implemented as a function in an Open Radio Access Network (ORAN) or Virtualised Radio Access Network (vRAN). The policy node may encompass multiple logical entities, as discussed in greater detail below, and may for example comprise a Virtualised Network Function (VNF).

[0059] Referring to Figure 4, the method 400 comprises, in a first step 410, obtaining CTR data for the cell and a current value of the parameter in the cell. The method 400 then comprises converting the CTR data to a histogram in step 420, and, in step 430, using a trained CNN to generate a latent representation of the histogram, the latent representation having a reduced dimensionality with respect to the histogram. The method 400 further comprises, in step 440, using a trained policy ML model to generate a control action for the parameter in the cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises the latent representation generated by the trained CNN. Finally, in step 450, the method 400 comprises causing the generated control action to be implemented in the cell. According to some examples of the method 400, the CNN and policy ML model may have been trained using examples of the method 200 and / or 300.

[0060] Figures 5a and 5b show flow charts illustrating process steps in a further example of method 500 for managing a parameter in a cell of a communication network. The method 500 provides various examples of how the steps of the method 400 may be implemented and supplemented to achieve the above discussed and additional functionality. As for the method 400, the method 500 is performed by a policy node, which may be a physical or virtual node, and which may encompass multiple logical entities, as discussed in greater detail above with reference to Figure 4.

[0061] Referring initially to Figure 5a, in a first step 510, the policy node obtains CTR data for the cell, and a current value of the parameter in the cell. The parameter for management may comprise any one or more parameters relating to the cell, and which are operable for control. Such parameters may include:

[0062] Remote electrical tilt (with passive antennas)

[0063] Digital tilt (with AAS - Advance Antenna System)

[0064] Maximum cell downlink transmission power

[0065] - CRS Power boost

[0066] P0 nominal PUSCH (for uplink power control)

[0067] P0 nominal PUCCH (for uplink power control)

[0068] - Alpha (for uplink power control).

[0069] As illustrated at 510a, the policy node may also obtain cell metrics for the cell which may include Performance Management (PM) counters, and may additionally include Configuration Management (CM) parameters. In further examples, the policy node may also obtain additional parameters such as cell topology information.

[0070] In step 515, the policy node filters the obtained cell metrics to retain a subset. As illustrated at 515a, the subset of cell metrics (for example PM counters and in some examples CM parameters) may have been identified during a training process for the CNN. Specifically, the subset may have been identified using a feature importance analysis carried out on a prediction DNN that was simultaneously trained with the CNN during the method 300. The policy node then converts the CTR data to a histogram in step 520. As illustrated at 520a, converting the CTR data to histograms may comprise, for obtained CTR data for the individual cell over a measurement time period, generating a histogram comprising, for each of a plurality of metrics included in the CTR data, reported values of the metrics and a count of the number of devices reporting each value of the metrics during the measurement time period. In some examples, the metrics for which counts of reported values are generated may include Serving cell Reference Signal Received Power (RSRP), Timing Advance (TA), Physical Cell ID (PCI) of a neighbouring cell detected and reported by the UE, and Neighbouring cell RSRP. In some examples, the measurement time period may be that covered by a single record, an hour, a day, a busy period during a day, etc.

[0071] Referring now to Figure 5b, following the generation of a histogram of CTR data at step 525, the policy node then reduces a size of the histogram by, for individual metrics represented in the histogram, performing at least one of reducing a range of metric values represented in the histogram (525a), reducing a resolution of metric values represented in the histogram (525b), and / or delta encoding of metric values with respect to a reference metric value (525c). As discussed above with reference to the method 300, different size reduction actions, or combinations of actions, may be carried out for different metrics. For example, the histogram dimension for TA data may be reduced in resolution by down sampling, and / or the histogram dimension for PCI of a reported neighbour cell may be reduced in resolution by resolving to a lower resolution metric (e.g. PCI to GCID mapping). The reference metric value for delta encoding may be that of neighbour cell values with respect to serving cell values (e.g. delta encoding of neighbour cell RSRP with respect to service cell RSRP). In further examples, neighbour RSRP values may be delta encoded and then filtered to maintain only a subset of the possible range, with range reduction based on a relevance criterion. This is discussed in further detail below, with reference to an example implementation of the methods presented herein.

[0072] In step 530, the policy node uses a trained CNN to generate a latent representation of the histogram, the latent representation having a reduced dimensionality with respect to the histogram. As illustrated at 530a, the CNN model may have been trained using examples of the method 200 and / or 300. In step 540, the policy node uses a trained policy ML model to generate a control action for the parameter in the cell from an input comprising a representation of the state of the cell. The cell state representation comprises the latent representation generated by the trained CNN. The cell state representation may also comprise cell metrics for the cell, such as PM counters and / or CM parameters, and / or additional cell information such as cell topology information. The cell metrics included in the state representation may comprise the subset of cell metrics discussed above with reference to step 510a.

[0073] As illustrated at 540a, the cell state representation input to the policy ML model may additionally comprise a current value of the parameter.

[0074] As discussed above, the policy ML model may be for managing a composite parameter comprising a plurality of sub parameters, each of which is operable to be controlled independently. In such examples as illustrated at 540b, step 540 may comprise using the trained policy ML model to generate control actions for each of the sub parameters in the cell.

[0075] According to some examples of the method 500, and as illustrated at 540c, the policy ML model may have been trained using examples of the method 200 and / or 300.

[0076] In step 550, the policy node causes the generated control action to be implemented in the cell. This may for example comprise directly implementing the control action, or sending a message or instruction to another logical entity with final responsibility for implementing the control action. Implementation of the control action comprises executing or putting the control action into effect in the cell. Thus, if the control action is an incremental change in the parameter value, causing the control action to be implemented in the cell comprises causing the incremental change to be carried out on the parameter value, either by directly executing the change, or instructing an appropriate entity to execute the change.

[0077] As discussed above, the methods 200 and 300 may be performed by a training node, and the present disclosure provides a training node that is adapted to perform any or all of the steps of the above discussed methods. The training node may comprise a physical node such as a computing device, server etc., or may comprise a virtual node. A virtual node may comprise any logical entity, such as a Virtualized Network Function (VNF) which may itself be running in a cloud, edge cloud or fog deployment. The training node may be operable to be instantiated in a cloud based deployment as discussed above with reference to the methods 200 and 300.

[0078] Figure 6 is a block diagram illustrating an example training node 600 which may implement the method 200 and / or 300, as illustrated in Figures 2 and 3a-d, according to examples of the present disclosure, for example on receipt of suitable instructions from a computer program 650. Referring to Figure 6 the training node 600 comprises a processor or processing circuitry 602, and may comprise a memory 604 and interfaces 606. The processing circuitry 602 is operable to perform some or all of the steps of the method 200 and / or 300 as discussed above with reference to Figures 2 and 3a-d. The memory 604 may contain instructions executable by the processing circuitry 602 such that the training node 600 is operable to perform some or all of the steps of the method 200 and / or 300, as illustrated in Figures 2 and 3a-d. The instructions may also include instructions for executing one or more telecommunications and / or data communications protocols. The instructions may be stored in the form of the computer program 650. In some examples, the processor or processing circuitry 602 may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, etc. The processor or processing circuitry 602 may be implemented by any type of integrated circuit, such as an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) etc. The memory 604 may include one or several types of memory suitable for the processor, such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, solid state disk, hard disk drive, etc.

[0079] As discussed above, the methods 400 and 500 may be performed by a policy node, and the present disclosure provides a policy node that is adapted to perform any or all of the steps of the above discussed methods. The policy node may comprise a physical node such as a computing device, server etc., or may comprise a virtual node. A virtual node may comprise any logical entity, such as a Virtualized Network Function (VNF) which may itself be running in a cloud, edge cloud or fog deployment. The policy node may be operable to be instantiated in a cloud based deployment, as discussed above with reference to methods 400 and 500.

[0080] Figure 7 is a block diagram illustrating an example policy node 700 which may implement the method 400 and / or 500, as illustrated in Figures 4 and 5a-b, according to examples of the present disclosure, for example on receipt of suitable instructions from a computer program 750. Referring to Figure 7, the policy node 700 comprises a processor or processing circuitry 702, and may comprise a memory 704 and interfaces 706. The processing circuitry 702 is operable to perform some or all of the steps of the method 400 and / or 500 as discussed above with reference to Figures 4 and 5a-b. The memory 704 may contain instructions executable by the processing circuitry 702 such that the policy node 700 is operable to perform some or all of the steps of the method 400 and / or 500, as illustrated in Figures 4 and 5a-b. The instructions may also include instructions for executing one or more telecommunications and / or data communications protocols. The instructions may be stored in the form of the computer program 750. In some examples, the processor or processing circuitry 702 may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, etc. The processor or processing circuitry 702 may be implemented by any type of integrated circuit, such as an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) etc. The memory 704 may include one or several types of memory suitable for the processor, such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, solid state disk, hard disk drive, etc.

[0081] Figures 2 to 5b discussed above provide an overview of methods which may be performed according to different examples of the present disclosure. These methods may be performed by a training node and a policy node respectively, as illustrated in Figures 6 and 7. The methods enable the automated generation of a state for use in RL management of a cell parameter in a communication network. There now follows a detailed discussion of how different process steps illustrated in Figures 2 to 5b and discussed above may be implemented. The functionality and implementation detail described below is discussed with reference to the modules of Figures 6 and 7 performing examples of the methods 200, 300, 400 and / or 500, substantially as described above.

[0082] The following implementation detail discusses the different steps which may be carried out according to the method presented herein as set out below:

[0083] Process CTR and create 4D histograms per cell (steps 220, 320, 420, 520) Train a CNN and a DNN to jointly predict the current value of the parameter to optimize (steps 230, 330)

[0084] Discard features with feature importance analysis (step 338)

[0085] Train policy ML model with encoded CTR data, remaining counters, and CM (steps 240, 340)

[0086] Process CTR and create 4D histograms per cell (steps 220, 320, 420, 520)

[0087] Figure 8 provides a schematic illustration of the generation of 4D histograms according to examples of the method disclosed herein. As discussed above, raw CTR data files contain periodic measurements reported by user equipment (UEs) to radio access nodes. In this step, these measurements are transformed into 4D histograms, in which the number of occurrences of reports of different measurement possibilities are counted. This conversion of raw CTR data to histograms places the raw CTR data into a more compact format without losing any information, and ensures the data is suitable for processing by a CNN. Each dimension of the histogram is associated with a different metric:

[0088] • Serving cell RSRP: RSRP level in dBm of the serving cell reported by the UE.

[0089] • TA: Timing Advance.

[0090] • PCI: Physical Cell ID (PCI) of the neighbouring cell detected and reported by the UE.

[0091] • Neighbouring cell RSRP: RSRP level in dBm of a neighbouring cell detected and reported by the UE.

[0092] Following histogram generation, the histograms may be reduced in size, as discussed in further detail below with reference to feasibility.

[0093] Train a CNN and a DNN to jointly predict the current value of the parameter to optimize (steps 230, 330)

[0094] Figure 9 illustrates simultaneous training of a CNN and DNN to predict parameter values. A CNN is used to reduce the size of the 4D CTR histograms while keeping the most relevant information needed for the optimization of the cell parameter under analysis. It is desirable for the CNN output (encoded CTR data) to have the smallest possible size that nonetheless contains enough information to optimize the parameter under analysis. The optimal size of the output of the CNN may be determined through trial and error as discussed below.

[0095] As illustrated in Figure 9, the output of the CNN is sent to a DNN, together with all available PM counters, and possibly with CM parameters and other cell information such as topology information. Both the CNN and DNN are simultaneously trained using forward-backward propagation with the target of predicting the current value of the cell parameter to optimize. This may comprise minimizing the loss, defined as the root mean square (RMS) error between the predicted and the actual cell parameter value. Each training sample is associated with the statistics of a cell for one day: PM statistics are aggregated during that day or during a busy period for the day, with the busy period defined as the top hours of the day with highest traffic in the network. The number of hours is a configurable parameter. CTR histograms may contain the data collected during the busy period of the same day. It is possible to combine data from multiple days and / or multiple networks to gather enough training samples, keeping a consistent collection period across days / networks.

[0096] The optimal size of the encoded CTR data can be selected in an automated way by training the proposed architecture for multiple sizes. The optimal size could be determined, for example by the elbow in the curve that describes the parameter prediction precision as a function of the encoded CTR data size, or by the minimum size that guarantees a maximum predefined prediction precision degradation with respect to not using a CNN.

[0097] Discard features with feature importance analysis (step 338)

[0098] While the CNN provides dimensionality reduction of the CTR data, the trained DNN can be used to determine which PM counters are contributing the most strongly to the prediction of the cell parameter to optimize. Such PM counters may be inferred to have a strong connection with the parameter under analysis, and consequently to be the most important to take into account in the policy model performing RL management of the parameter. This determination can be carried out using a feature importance analysis, such as a SHAP analysis, in which the top PM counters with highest SHAP values are selected and the rest are discarded, as illustrated in Figure 10. As an alternative to SHAP analysis, the least relevant PM counters may be discarded using L1 regularization, in which the extra penalty added to the loss function only includes the neurons of the first layer of the DNN that have PM counters as inputs. Alternatively, any other technique that provides feature importance may be used.

[0099] Train policy ML model with encoded CTR data, remaining counters, and CM (steps 240, 340)

[0100] Following the selection of relevant PM counters, the policy ML model can be trained to optimize the selected cell parameter, for example following the scheme proposed in WO 2021 / 190772. Figure 11 illustrates training of the ML policy model (illustrated as Learning Module DNN in the Figure), and the components of the cell state representation input to the policy ML model. In addition to the encoded CTR data and the selected PM counters, further cell configuration parameters may be included in the cell state representation, such as:

[0101] • Current value of the cell parameter to optimize.

[0102] • Other CM parameters related to the parameter to optimize, e.g., Cell-specific Reference Signal (CRS) gain related to maximum cell transmission power.

[0103] • Depending on the type of parameter to optimize, it may be useful to include topology parameters, such as inter-site distance (e.g., calculated as the average distance to the 5 closest intra-frequency sites) or antenna height.

[0104] It will be appreciated that the generation of the encoded CTR data is carried out using the CNN trained in the above described process. The trained CNN operates as a preprocessing module and is part of the DRL architecture to optimise the cell parameter. As discussed above, it is possible to include the additional cell configuration parameters together with the aggregated PM data as inputs to the DNN prediction architecture during simultaneous training of the CNN and DNN, and to include those inputs in the feature importance analysis, so as to be able to discard those with lowest importance from the state representation.

[0105] It will also be appreciated that during inference and in the training data, the current value of the cell parameter to optimize is known, and the target of the DRL agent (policy ML model) is to find the best next value based on the current state. The value of the cell parameter to optimize is used as the label value for training of the prediction DNN in the training phase, but this is to enable the CNN and feature importance analysis to learn the most important information for that parameter, and does not mean that the current value of the cell parameter to optimize is unknown.

[0106] As discussed above with reference to the methods 300 and 500, it is possible to use the trained policy ML model to optimise multiple parameters simultaneously, with the following considerations:

[0107] The policy ML model will use composite actions that are the combinations of the individual actions for each parameter separately. For example, an agent that optimizes RET and Maximum Cell Transmission Power (MaxTxPower) using incremental values will work with 9 possible actions (the combinations of 3 possible actions per parameter):

[0108] • Action 0: RET= -1 deg & MaxTxPower = -1 dB;

[0109] • Action 1 : RET= -1 deg & MaxTxPower = 0 dB;

[0110] • Action 2: RET= -1 deg & MaxTxPower = 1 dB;

[0111] • Action 3: RET= 0 deg & MaxTxPower = -1 dB;

[0112] • Action 4: RET= 0 deg & MaxTxPower = 0 dB;

[0113] • Action 5: RET= 0 deg & MaxTxPower = 1 dB;

[0114] • Action 6: RET= 1 deg & MaxTxPower = -1 dB;

[0115] • Action 7: RET= 1 deg & MaxTxPower = 0 dB;

[0116] • Action 8: RET= 1 deg & MaxTxPower = 1 dB;

[0117] During simultaneous training of the CNN and prediction DNN, the values of the multiple parameters are estimated simultaneously. This means that there are as many outputs as optimizable parameters at the prediction DNN illustrated in Figure 9 (two outputs in the previous example in which RET and MaxTxPower and optimized simultaneously). The prediction loss can be an aggregation function of the individual prediction errors of each individual parameter.

[0118] CTR data (steps 325 and 525)

[0119] It will be appreciated that the complexity of the CNN for CTR data encoding has a strong dependency on the size of the input data, i.e., the 4D histograms per cell in this example implementation. Each histogram may encompass the following sizes for each dimension:

[0120] • Serving cell RSRP G [-140, -44] dBm, i.e., 97 values.

[0121] • TA G [0, 32767] / 16, i.e., 2048 values.

[0122] • PCI G [0, 503], i.e., 504 values. Neighbouring cell RSRP G [-140, -44] dBm, i.e., 97 values.

[0123] The total values contained by one of these 4D matrices is: 97x1282x504x97 = 6,079,418,352 = 6.08e9 (=6 billion values). This is too large to be reasonable for CNN encoding. However, not all ranges provide useful information, and it is possible to reduce the size of the 4D matrices without losing useful information, as set out below.

[0124] Serving cell RSRP values can be reduced to the range [-130, -65] dBm, i.e., 66 values. It will be appreciated that outliers can be stored in the closest bin. In this case, values that are overly high are not relevant to an optimizing agent, because radio coverage and quality will remain high regardless of the suggested actions. Similar considerations apply to the very low values, i.e., under -130 dBm, which are way far from the minimum values to obtain coverage.

[0125] TA can be down sampled to bins with decreasing resolution, resulting in 128 values only. 3GPP guarantees higher resolution for TA values than what it is really required in some cases. In real systems, high resolution of TA values is only useful in femto, pico and micro cells. It is therefore proposed to use a variable resolution to store TA values, as illustrated in Figure 12.

[0126] PCI can be resolved into Global Cell ID (GCID), and only the most significant, for example the top 12, interferers included. PCI resolution is a function currently carried out by many applications fed with CTR data, and comprises assigning each measurement report to the GCID of the estimated cell with closest relative position (including azimuth), using the PCI of the reported cell, the GCID of the serving cell and the topographic information of the serving cell and all the neighbouring cells with the reported PCI.

[0127] Neighbouring cell RSRP can be delta encoded by replacing it with the difference between the RSRP of the serving cell and the RSRP of the neighbouring cell, i.e., RSRPs - RSRPn. For this new metric it can be assumed that all useful values are in the range [- 5, +15] dB, which means that no more than 21 values for this dimension of the 4D matrix. Lower values than -5 dB mean that the user has reported having received an RSRP from a neighbouring cell more than 5 dB higher than from the serving cell, which is a situation typically prevented by mobility management. On the other hand, RSRP from neighbours 15 dB lower than the serving cell can be considered negligible from interference point of view.

[0128] The total values contained by one of the 4D matrices with the above discussed reduced ranges of values is: 66x128x12x21 = 2,128,896 (=2 million values). This is a reasonable input size for image processing using CNNs.

[0129] The methods proposed herein can be implemented as a single rApp in the Non Real- Time Radio Access Network Intelligent Controller (Non-RT RIC) located in the Service Management & Orchestrator (SMO) Framework of the Open Radio Access Network (O- RAN) architecture. This is shown in Figure 13. The rApp can be used as a standalone app and / or the results can be used by other rApps.

[0130] Aspects of the present disclosure thus provide methods and nodes that enable the automated generation of a cell state representation for use in RL management of a cell parameter. The methods involve the conversion of CTR data to histograms, which can then be processed by a CNN in order to reduce the dimensionality of the CTR data while maintaining information relevant to the cell parameter for optimisation. The CNN is trained simultaneously with a DNN that is operable to predict values of the cell parameter for optimisation. In this manner, the CNN can learn what information contained within the CTR data is most strongly connected with the value of the relevant parameter, and should therefore be maintained in the latent representation. The DNN may also accept as input call metrics such as PM counters, and a feature importance analysis performed on the trained prediction DNN may be used to identify the most relevant PM counters for the cell parameter to be optimised. The identified PM counters, and trained CNN, can then be used to train a policy PM model for RL management of the cell parameter. During subsequent online use of the trained policy ML model, the trained CNN remains as a pre-processing part of the architecture, generating the reduced dimension latent representation of the CTR data. The identified subset of PM counters, together with other information such as the current parameter value, CM parameters, topology information, etc., is combined with the latent CTR representation to form the state for inputting to the trained policy ML model.

[0131] The automated generation of a relevant state representation for optimisation of a given parameter significantly reduces the time required to engineer RL management of a new parameter, as well as eliminating the need for expert support in manual state design. This closes the automation loop for the creation of new use cases associated with new parameters to be optimized with RL. In addition, the data driven design enables generation of a state that covers the maximum amount of information with the minimum number of features, something unfeasible when state is engineered by humans, regardless of their level of expertise.

[0132] The methods of the present disclosure may be implemented in hardware, or as software modules running on one or more processors. The methods may also be carried out according to the instructions of a computer program, and the present disclosure also provides a computer readable medium having stored thereon a program for carrying out any of the methods described herein. A computer program embodying the disclosure may be stored on a computer readable medium, or it could, for example, be in the form of a signal such as a downloadable data signal provided from an Internet website, or it could be in any other form.

[0133] It should be noted that the above-mentioned examples illustrate rather than limit the disclosure, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims or numbered embodiments. The word “comprising” does not exclude the presence of elements or steps other than those listed in a claim or embodiment, “a” or “an” does not exclude a plurality, and a single processor or other unit may fulfil the functions of several units recited in the claims or numbered embodiments. Any reference signs in the claims or numbered embodiments shall not be construed so as to limit their scope.

Claims

CLAIMS1 . A computer implemented method (200) for training a policy Machine Learning, ML, model to manage a parameter in a cell of a communication network, the method, performed by a training node, comprising:(i) obtaining a training data set (210) comprising, for each of a plurality of cells in the communication network, and for time instances during a period of operation of the communication network, records of: a representation of a state of the cell, the representing including Call Trace Recording, CTR, data, and a corresponding value of the parameter;(ii) converting the CTR data from the state representations to histograms (220);(iii) using the training data set (230) to simultaneously train: a Convolutional Neural Network, CNN, to generate a latent representation of the histograms, the latent representation having a reduced dimensionality with respect to the histograms; and a Deep Neural Network, DNN, to predict values of the parameter from an input comprising the latent representations generated by the CNN;(iv) using a Reinforcement Learning, RL, process to train a policy ML model to generate control actions for the parameter in a cell from an input comprising a representation of the state of the cell (240), wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN.

2. A method as claimed in claim 1 , wherein the training data set further comprises cell metrics including Performance Management, PM, counters (310a); wherein step (iii) comprises using the training data set to simultaneously train: a CNN to generate a latent representation of the histograms, the latent representation having a reduced dimensionality with respect to the histograms; and a DNN to predict values of the parameter from an input comprising the latent representations generated by the CNN and the cell metrics (330); and wherein step (iv) comprises using an RL process to train a policy ML model to generate control actions for the parameter in a cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises alatent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN, and cell metrics for the cell (340).

3. A method as claimed in claim 2, further comprising: performing a feature importance analysis on the trained DNN to identify a subset of cell metrics whose contribution to the output of the DNN exceeds an importance threshold (338); and wherein step (iv) comprises using an RL process to train a policy ML model to generate control actions for the parameter in a cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN, and the identified subset of cell metrics for the cell (340).

4. A method as claimed in any one of the preceding claims, wherein step (ii) comprises converting the CTR data from the state representations to histograms by, for records of CTR data for individual cells over a measurement time period: generating a histogram comprising, for each of a plurality of metrics included in the CTR data, reported values of the metrics and a count of the number of devices reporting each value of the metrics during the measurement time period (320a).

5. A method as claimed in any one of claims 2 to 4, wherein step (iii) comprises repeating the steps of: inputting a histogram to the CNN, wherein the CNN is operable to process the histogram according to current values of its trainable parameters, and to output a latent representation of the histogram (331); inputting to the DNN the generated latent representation output by the CNN and PM counters for the measurement time period covered by the data in the histogram, wherein the DNN is operable to process the input latent representation and PM counters according to current values of its trainable parameters, and to output a predicted value of the parameter (332); calculating a loss function according to a difference between the predicted value of the parameter and a measured value of the parameter for the measurement time period covered by the histogram in the training data set (333); and updating values of the trainable parameters of the CNN and DNN according to the calculated loss function (334).

6. A method as claimed in any one of the preceding claims, further comprising: performing at least one additional iteration of step (iii), wherein, in the additional iteration, the CNN is trained to generate a latent representation having a different reduced dimensionality with respect to the histograms (336); and selecting either the trained CNN from the first iteration of step (iii), or the trained CNN from the additional iteration of step (iii) to use for training the policy ML model in step (iv) (337, 340c).

7. A method as claimed in claim 6, wherein selecting either the trained CNN from the first iteration of step (iii), or the trained CNN from the additional iteration of step (iii) to use for training the policy ML model in step (iv) comprises: identifying the trained CNN that fulfils a selection criterion balancing data size of the latent representation and prediction precision of the trained DNN (337a).

8. A method as claimed in any one of the preceding claims, further comprising: reducing a size of the histograms (325) by, for individual metrics represented in the histogram, performing at least one of: reducing a range of metric values represented in the histogram (325a); reducing a resolution of metric values represented in the histogram (325b); delta encoding of metric values with respect to a reference metric value(325c).

9. A method as claimed in any one of the preceding claims, wherein in step (iv), the input to the policy ML model comprises a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN, and a current value of the parameter (340a).

10. A method as claimed in any one of the preceding claims, wherein in step (iv), the input to the policy ML model comprises a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN, and Configuration Management, CM, parameters for the cell (340).

11. A method as claimed in claim 10, when dependent on claim 2, wherein the cell metrics further include CM parameters (310a).

12. A method as claimed in any one of the preceding claims, wherein the policy ML model is for managing a composite parameter comprising a plurality of sub parameters, each of which is operable to be controlled independently; wherein step (iii) comprises using the training data set to simultaneously train: a CNN to generate a latent representation of the histograms, the latent representation having a reduced dimensionality with respect to the histograms; and a DNN to predict values of each of the plurality of sub parameters from an input comprising the latent representations generated by the CNN (330a); and wherein step (iv) comprises using an RL process to train a policy ML model to generate control actions for each of the sub parameters in a cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN (340b).

13. A computer implemented method (400) for managing a parameter in a cell of a communication network, the method, performed by a policy node, comprising:(i) obtaining Call Trace Recording, CTR, data for the cell and a current value of the parameter in the cell (410);(ii) converting the CTR data to a histogram (420);(iii) using a trained Convolutional Neural Network, CNN, to generate a latent representation of the histogram, the latent representation having a reduced dimensionality with respect to the histogram (430);(iv) using a trained policy Machine Learning, ML, model to generate a control action for the parameter in the cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises the latent representation generated by the trained CNN (440); and(v) causing the generated control action to be implemented in the cell (450).

14. A method as claimed in claim 13, wherein the CNN and policy ML model have been trained using a method according to any one of claims 1 to 12 (530a, 540c).

15. A method as claimed in claim 13 or 14, further comprising:obtaining cell metrics including Performance Management, PM, counters for the cell (510a); and wherein step (iv) comprises using a trained policy ML model to generate a control action for the parameter in the cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises the latent representation generated by the trained CNN, and obtained cell metrics (540).

16. A method as claimed in claim 15, further comprising: filtering the obtained cell metrics to retain a subset (515); and wherein step (iv) comprises using a trained policy ML model to generate a control action for the parameter in the cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises the latent representation generated by the trained CNN and the subset of obtained cell metrics (540).

17. A method as claimed in claim 16, wherein the subset of cell metrics has been identified during a training process for the CNN (515a).

18. A method as claimed in any one of the claims 13 to 17, wherein step (ii) comprises converting the CTR data to histograms by, for obtained CTR data for the individual cell over a measurement time period: generating a histogram comprising, for each of a plurality of metrics included in the CTR data, reported values of the metrics and a count of the number of devices reporting each value of the metrics during the measurement time period (520a).

19. A method as claimed in any one of claims 13 to 18, further comprising: reducing a size of the histogram (525) by, for individual metrics represented in the histogram, performing at least one of: reducing a range of metric values represented in the histogram (525a); reducing a resolution of metric values represented in the histogram (525b); delta encoding of metric values with respect to a reference metric value (525c).

20. A method as claimed in any one of claims 13 to 19, wherein in step (iv), the input to the policy ML model comprises a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR datafor the cell, the latent representation generated by the trained CNN, and a current value of the parameter (540a).

21. A method as claimed in any one of claims 13 to 20, wherein in step (iv), the input to the policy ML model comprises a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN, and Configuration Management, CM, parameters for the cell (540).

22. A method as claimed in claim 21 , when dependent on claim 15, wherein the cell metrics further include CM parameters (510a).

23. A method as claimed in any one of claims 13 to 22, wherein the policy ML model is for managing a composite parameter comprising a plurality of sub parameters, each of which is operable to be controlled independently; and wherein step (iv) comprises using a trained policy ML model to generate control actions for each of the sub parameters in the cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises the latent representation generated by the trained CNN (540b).

24. A computer program product comprising a computer readable medium, the computer readable medium having computer readable code embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform a method as claimed in any one of claims 1 to 23.

25. A training node (600) fortraining a policy Machine Learning, ML, model to manage a parameter in a cell of a communication network, the training node comprising processing circuitry (602) configured to cause the training node to:(i) obtain a training data set comprising, for each of a plurality of cells in the communication network, and for time instances during a period of operation of the communication network, records of: a representation of a state of the cell, the representing including Call Trace Recording, CTR, data, and a corresponding value of the parameter;(ii) convert the CTR data from the state representations to histograms;(iii) use the training data set to simultaneously train: a Convolutional Neural Network, CNN, to generate a latent representation of the histograms, the latent representation having a reduced dimensionality with respect to the histograms; and a Deep Neural Network, DNN, to predict values of the parameter from an input comprising the latent representations generated by the CNN;(iv) use a Reinforcement Learning, RL, process to train a policy ML model to generate control actions for the parameter in a cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises a latent representation of a histogram of CTR data for the cell, the latent representation generated by the trained CNN.

26. A training node as claimed in claim 25, wherein the processing circuitry is further configured to cause the training node to carry out a method according to any one of claims 2 to 12.

27. A policy node (700) for managing a parameter in a cell of a communication network, the policy node comprising processing circuitry (702) configured to cause the policy node to:(i) obtain Call Trace Recording, CTR, data, for the cell and a current value of the parameter in the cell;(ii) convert the CTR data to a histogram;(iii) use a trained Convolutional Neural Network, CNN, to generate a latent representation of the histogram, the latent representation having a reduced dimensionality with respect to the histogram;(iv) use a trained policy Machine Learning, ML, model to generate a control action for the parameter in the cell from an input comprising a representation of the state of the cell, wherein the cell state representation comprises the latent representation generated by the trained CNN; and(v) cause the generated control action to be implemented in the cell.

28. A policy node as claimed in claim 27, wherein the processing circuitry is further configured to cause the policy node to carry out a method according to any one of claims 14 to 23.

Images