Methods, apparatus and computer programs

Abstract

A method comprising: receiving, from a plurality of user equipment, information relating to one or more statistical parameters, the one or more statistical parameters relating to measurements made by the respective user equipment; determining, based on the received information, measurement and reporting configuration information for one or more of the plurality of user equpment; and sending the measurement and reporting configuration information to the one or more of the plurality of user equipment.

Description

[0001] METHODS, APPARATUS AND COMPUTER PROGRAMS

[0002] TECHNICAL FIELD

[0003] Various example embodiments of this disclosure relate to methods, apparatus, system and computer programs and in particular, but not exclusively, methods, apparatus, system and computer programs relating to the controlling of measurements made by a user equipment. BACKGROUND

[0004] A communication network can be seen as a facility that enables communications between two or more communication devices, or provides communication devices access to a data network. A mobile or wireless communication network is one example of a communication network. A communication device may be provided with a service by an application server.

[0005] Such communication networks operate in according with standards such as those provided by 3GPP (Third Generation Partnership Project) or ETSI (European Telecommunications Standards Institute). Examples of standards are the so-called 5G (5th Generation) standards provided by 3GPP. SUMMARY

[0006] Some example embodiments of this disclosure will be described with respect to certain aspects. These aspects are not intended to indicate key or essential features of the embodiments of this disclosure, nor are they intended to be used to limit the scope of thereof. Other features, aspects, and elements will be readily apparent to a person skilled in the art in view of this disclosure.

[0007] According to a first aspect, there is provided a method comprising: determining one or more statistical parameters, the one or more statistical parameters relating to measurements made by the user equipment; sending information relating to the one or more statistical parameters to a network entity; and receiving from the network entity, measurement and reporting configuration information.

[0008] The measurement and reporting configuration information may comprise information indicating that the user equipment is to improve a quality of the measurements made by the user equipment.

[0009] The measurement and reporting configuration information may comprise a request for one or more further measurements to be performed.

[0010] The measurement and reporting configuration information may comprise a cause for the request for the performance of further measurements. The measurement and reporting configuration information may comprises one or more of information about a configuration to be used by the user equipment; information about a changed sampling rate to be used by the user equipment when performing further measurements; information about a filtering to be applied by the user equipment to the further measurements; and information about reporting further measurements,

[0011] The measurement and reporting configuration information may comprises information indicating that a quality of data provided by the user equipment is of a relatively low quality. This may be in comparison to a quality of data provided by other user equipment. In response to this information, the method may comprising changing a measurement configuration.

[0012] The one or more statistical parameters may provide a measure of a quality of the measurements made by the user equipment.

[0013] The one or more statistical parameters may comprise one or more of outlier information relating to outlying measurements of the measurements made by the user equipment; feature quality information related to one or more features, the feature quality information providing a measurable property of the measurements made by the user equipment; prediction confidence information relating to a prediction provided by the user equipment; prediction correlation coefficient information relating to a prediction provided by the user equipment; missing values information indicating an amount of missing data from the measurements made by the user equipment; and / or discrepancy-aware level providing information about a deviation of a distribution of the measurements made by the user equipment from a uniform distribution.

[0014] The method may comprise updating a local machine learning model and sending to the network entity updated local machine learning model parameters of the local machine learning model.

[0015] The updated local machine learning model parameters may be used by the network entity to update a global machine learning model, the global machine learning model being trained with updated local machine learning model parameters from a plurality of user equipment, and the method may comprise receiving updated global machine learning model parameters from the network entity and applying the updated global machine learning model parameters to the local machine learning model.

[0016] The measurement and reporting configuration information may comprises information indicating that the user equipment is to use synthetic data when updating the local machine learning model. The local machine learning model may be for providing machine learning enabled beam management and the measurements may relate to beams.

[0017] The local machine learning model may be for predicting downlink transmission beam prediction or for downlink transmission reception beam pair prediction, and the measurements may relate to reference signal received power.

[0018] The local machine learning model may be for providing machine learning enabled channel state indicator feedback and the measurements may relate to reference signals.

[0019] The local machine learning model may be for providing machine learning enabled positioning and the measurements may relate to positioning.

[0020] The method may be performed by an apparatus.

[0021] The apparatus may be a user equipment.

[0022] The apparatus may comprise at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform any of the methods discussed previously.

[0023] According to a second aspect, there is provided a user equpment comprising: means for determining one or more statistical parameters, the one or more statistical parameters relating to measurements made by the user equipment; means for sending information relating to the one or more statistical parameters to a network entity; and means for receiving from the network entity, measurement and reporting configuration information.

[0024] The measurement and reporting configuration information may comprise information indicating that the user equipment is to improve a quality of the measurements made by the user equipment.

[0025] The measurement and reporting configuration information may comprise a request for one or more further measurements to be performed.

[0026] The measurement and reporting configuration information may comprise a cause for the request for the performance of further measurements.

[0027] The measurement and reporting configuration information may comprises one or more of: information about a configuration to be used by the user equipment; information about a changed sampling rate to be used by the user equipment when performing further measurements; information about a filtering to be applied by the user equipment to the further measurements; and information about reporting further measurements,

[0028] The measurement and reporting configuration information may comprises information indicating that a quality of data provided by the user equipment is of a relatively low quality. This may be in comparison to a quality of data provided by other user equipment. In response to this information, the user equipment may comprise means for changing a measurement configuration.

[0029] The one or more statistical parameters may provide a measure of a quality of the measurements made by the user equipment.

[0030] The one or more statistical parameters may comprise one or more of: outlier information relating to outlying measurements of the measurements made by the user equipment; feature quality information related to one or more features, the feature quality information providing a measurable property of the measurements made by the user equipment; prediction confidence information relating to a prediction provided by the user equipment; prediction correlation coefficient information relating to a prediction provided by the user equipment; missing values information indicating an amount of missing data from the measurements made by the user equipment; and / or discrepancy-aware level providing information about a deviation of a distribution of the measurements made by the user equipment from a uniform distribution.

[0031] The user equipment may comprise means for updating a local machine learning model and sending to the network entity updated local machine learning model parameters of the local machine learning model.

[0032] The updated local machine learning model parameters may be used by the network entity to update a global machine learning model, the global machine learning model being trained with updated local machine learning model parameters from a plurality of user equipment, and the user equipment may comprise means for receiving updated global machine learning model parameters from the network entity and means for applying the updated global machine learning model parameters to the local machine learning model.

[0033] The measurement and reporting configuration information may comprises information indicating that the user equipment is to use synthetic data when updating the local machine learning model.

[0034] The local machine learning model may be for providing machine learning enabled beam management and the measurements may relate to beams.

[0035] The local machine learning model may be for predicting downlink transmission beam prediction or for downlink transmission reception beam pair prediction, and the measurements may relate to reference signal received power.

[0036] The local machine learning model may be for providing machine learning enabled channel state indicator feedback and the measurements may relate to reference signals. The local machine learning model may be for providing machine learning enabled positioning and the measurements may relate to positioning.

[0037] According to a third aspect, there is provided a method comprising: receiving, from a plurality of user equipment, information relating to one or more statistical parameters, the one or more statistical parameters relating to measurements made by the respective user equipment; determining, based on the received information, measurement and reporting configuration information for one or more of the plurality of user equpment; and sending the measurement and reporting configuration information to the one or more of the plurality of user equipment.

[0038] The one or more statistical parameters may provide a measure of a quality of the measurements made by the respective user equipment.

[0039] The one or more statistical parameters may provide a measure of a quality of the measurements made by the respective user equipment whilst maintaining privacy associated with the measurements.

[0040] The one or more statistical parameters may comprise one or more of: outlier information relating to outlying measurements of the measurements made by a user equipment; feature quality information related to one or more features, the feature quality information providing a measurable property of the measurements made by a user equipment; prediction confidence information relating to a prediction provided by a user equipment; prediction correlation coefficient information relating to a prediction provided by a user equipment; missing values information indicating an amount of missing data from the measurements made by a user equipment; and / or discrepancy-aware level providing information about a deviation of a distribution of the measurements made by a user equipment from a uniform distribution.

[0041] The method may comprise using the respective information relating to the one or more statistical parameters for a respective one of the plurality of user equipment to determine weight information for the respective user equipment, and determining the measurement and reporting configuration information for one or more of the plurality of user equipment based on the respective weight information.

[0042] The method may comprise determining the weight information based on an evaluation score for each of the plurality of user equipment, said evaluation score being determined based on a respective weighting applied to the one or more statistical parameters.

[0043] The method may comprise allocating each user equipment to a group of a plurality of groups based on the determined evaluation score, the measurement and reporting configuration information being associated with a respective one or more of the plurality of groups. The method may comprise receiving from the plurality of user equipment, updated local machine learning model parameters of a local machine learning model and using at least some of the updated local machine learning model parameters to update global machine learning model parameters of a global machine learning model.

[0044] The method may comprise determining for each of the plurality of groups a set of updated local machine learning model parameters.

[0045] The set of updated local machine learning model parameters for a group may be an average of the updated local machine learning model parameters provide by the user equipment in that group.

[0046] The method may comprise determining a group weight associated with each group and using the group weight when combining the set of updated local machine learning model parameters of each group to provide updated global machine learning parameters of the global machine learning model.

[0047] The group weight associated with each group is dependent on a highest evaluation score associated with a group.

[0048] The measurement and reporting configuration information may comprise one or more of: information requesting the user equpment to perform further measurements; information about a configuration to be used by the user equipment; information about a changed sampling rate to be used when performing further measurements; information about a filtering to be applied to the further measurements; information about reporting further measurements; and information indicating that the user equipment is to use synthetic data with a machine learning model.

[0049] The method may comprise providing information to a respective user equipment which has been requested to perform further measurements indicating a cause for the request for the performance of further measurements.

[0050] The method may comprise providing information to a respective user equipment about a quality of the measurements of that user equipment compared to one or more other user equipment.

[0051] The method may comprise receiving, from one or more user equipment of a group requested to perform further measurements, information relating to one or more statistical parameters relating to the further measurements made by the respective user equipment and updated local machine learning model parameters, and using the updated local machine learning model parameters to update the set of updated local machine learning model parameters. The machine learning model may be for providing machine learning enabled beam management and the measurements relate to beams.

[0052] The machine learning model may be for downlink transmission beam prediction or for downlink transmission reception beam pair prediction, and the measurements may relate to reference signal received power.

[0053] The machine learning model may be for providing machine learning enabled channel state indicator feedback and the measurements may relate to reference signals.

[0054] The machine learning model may be for providing machine learning enabled positioning and the measurements may relate to positioning.

[0055] The method may be performed by an apparatus.

[0056] The apparatus may be a network entity. The apparatus may be a base station.

[0057] The apparatus may comprise at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform any of the methods discussed previously.

[0058] According to a fourth aspect, there is provided a means for receiving, from a plurality of user equipment, information relating to one or more statistical parameters, the one or more different statistical parameters relating to measurements made by the respective user equipment; means for determining, based on the received information, measurement and reporting configuration information for one or more of the plurality of user equpment; and means for sending the measurement and reporting configuration information to the one or more of the plurality of user equipment.

[0059] The one or more statistical parameters may provide a measure of a quality of the measurements made by the respective user equipment.

[0060] The one or more statistical parameters may provide a measure of a quality of the measurements made by the respective user equipment whilst maintaining privacy associated with the measurements.

[0061] The one or more statistical parameters may comprise one or more of: outlier information relating to outlying measurements of the measurements made by a user equipment; feature quality information related to one or more features, the feature quality information providing a measurable property of the measurements made by a user equipment; prediction confidence information relating to a prediction provided by a user equipment; prediction correlation coefficient information relating to a prediction provided by a user equipment; missing values information indicating an amount of missing data from the measurements made by a user equipment; and / or discrepancy-aware level providing information about a deviation of a distribution of the measurements made by a user equipment from a uniform distribution.

[0062] The apparatus may comprise means for using the respective information relating to the one or more statistical parameters for a respective one of the plurality of user equipment to determine weight information for the respective user equipment, and determining the measurement and reporting configuration information for one or more of the plurality of user equipment based on the respective weight information.

[0063] The apparatus may comprise means for determining the weight information based on an evaluation score for each of the plurality of user equipment, said evaluation score being determined based on a respective weighting applied to the one or more statistical parameters.

[0064] The apparatus may comprise means for allocating each user equipment to a group of a plurality of groups based on the determined evaluation score, the measurement and reporting configuration information being associated with a respective one or more of the plurality of groups.

[0065] The apparatus may comprise means for receiving from the plurality of user equipment, updated local machine learning model parameters of a local machine learning model and using at least some of the updated local machine learning model parameters to update global machine learning model parameters of a global machine learning model.

[0066] The apparatus may comprise means for determining for each of the plurality of groups a set of updated local machine learning model parameters.

[0067] The set of updated local machine learning model parameters for a group may be an average of the updated local machine learning model parameters provide by the user equipment in that group.

[0068] The apparatus may comprise means for determining a group weight associated with each group and using the group weight when combining the set of updated local machine learning model parameters of each group to provide updated global machine learning parameters of the global machine learning model.

[0069] The group weight associated with each group is dependent on a highest evaluation score associated with a group.

[0070] The measurement and reporting configuration information may comprise one or more of: information requesting the user equipment to perform further measurements; information about a configuration to be used by the user equipment; information about a changed sampling rate to be used when performing further measurements; information about a filtering to be applied to the further measurements; information about reporting further measurements; and information indicating that the user equipment is to use synthetic data with a machine learning model.

[0071] The apparatus may comprise means for providing information to a respective user equipment which has been requested to perform further measurements indicating a cause for the request for the performance of further measurements.

[0072] The apparatus may comprise means for providing information to a respective user equipment about a quality of the measurements of that user equipment compared to one or more other user equipment.

[0073] The apparatus may comprise means for receiving, from one or more user equipment of a group requested to perform further measurements, information relating to one or more statistical parameters relating to the further measurements made by the respective user equipment and updated local machine learning model parameters, and using the updated local machine learning model parameters to update the set of updated local machine learning model parameters.

[0074] The machine learning model may be for providing machine learning enabled beam management and the measurements relate to beams.

[0075] The machine learning model may be for downlink transmission beam prediction or for downlink transmission reception beam pair prediction, and the measurements may relate to reference signal received power.

[0076] The machine learning model may be for providing machine learning enabled channel state indicator feedback and the measurements may relate to reference signals.

[0077] The machine learning model may be for providing machine learning enabled positioning and the measurements may relate to positioning.

[0078] According to another aspect, there is provided a computer readable medium comprising program instructions stored thereon for performing at least one of the above methods.

[0079] According to an aspect, there is provided a non-transitory computer readable medium comprising program instructions stored thereon for performing at least one of the above methods.

[0080] According to an aspect, there is provided a non-volatile tangible memory medium comprising program instructions stored thereon for performing at least one of the above methods.

[0081] In the above, many different aspects have been described. It should be appreciated that further aspects may be provided by the combination of any two or more of the aspects described above. Various other aspects are also described in the following detailed description and in the attached claims.

[0082] DESCRIPTION OF FIGURES

[0083] Some example embodiments will now be described, by way of non-limiting and illustrative example only, with reference to the accompanying Figures in which:

[0084] FIG. 1 shows a schematic representation an access node and UEs (user equipment);

[0085] FIG. 2 shows a representation of an apparatus according to some example embodiments;

[0086] FIG. 3 shows a representation of user equipment for the communication system of FIG. 1 according to some example embodiments;

[0087] FIG. 4 shows a first example procedure of some embodiments;

[0088] FIG. 5 shows schematically a hierarchical weighted aggregation of some embodiments;

[0089] FIG. 6 shows a second example procedure of some embodiments, where FL (federated learning) is used;

[0090] FIG. 7 shows a third example procedure of some embodiments, in a beam management scenario;

[0091] FIG. 8 shows a first method of some embodiments; and

[0092] FIG. 9 shows a second method of some embodiments.

[0093] DETAILED DESCRIPTION

[0094] In the following, various example embodiments are explained with reference to communication devices capable of communication with a communication system. Before explaining in detail the embodiments of the methods and apparatuses of the present disclosure, a simplified example is shown in FIG. 1 where an access node 200 serves four user equipment (UE) 202, 204, 206 and 208. It should be appreciated that this is by way of example and more or less than four UE may be provided. The access node may be a base station.

[0095] In a 5G communication system (5GS), the access node 200 may be provided as part of an access network such as a 5G radio access network (5G-RAN) or next generation radio access network (NG-RAN. The access node may comprise a gNodeB. A gNB may include one or more gNodeB (gNB) distributed units connected to one or more gNodeB (gNB) centralized units.

[0096] In some embodiments, a user equipment is configured to provide assistance information to the access node. The assistance information is a measure of quality of data at the UE side. The assistance information may be one or more statical parameters determined by the UE. The assistance information may relate to a given period of time.

[0097] Some examples of assistance information are as follows:

[0098] Features quality: One criterion for evaluating data quality is the number of independent features. A feature may be a measurable property of a data sample collected or measure by the UE. The UE may compute mutual information (MI) among each pair of features. This may be used to determine a feature quality.

[0099] Prediction confidence: Sampling error is one of the inferential noise types that may be measured via a confidence interval. The UE may compute a confidence interval corresponding to their prediction. The UE may an averaged computed confidence interval as a prediction confidence.

[0100] Discrepancy-aware level: A UE may compute deviation of its local category distribution (e.g., using L2 difference or KL-Divergence) with reference to a uniform distribution.

[0101] Missing values - a UE may send information indicating a percentage of missing data in their local data sets.

[0102] A UE can estimate the percentage of the outliers in its available data.

[0103] Prediction correlation coefficient - this is a statistical measure that quantifies the strength and direction of the relationship between two variables.

[0104] More detailed information about the assistance information is provided later which may be applied in some embodiments..

[0105] Reference is made to FIG. 4 which shows a method flow of some examples.

[0106] A referenced SI, the network entity, for example the access node or gNodeB, sends a message with configuration to a UE. The configuration information may comprise information to configure the measurements made by the UE and the associated reporting of those measurements.

[0107] As referenced S2, the UE will make measurements and / or collect data accordance with the configuration received from the network and send assistance information or any other suitable information to the network entity.

[0108] As referenced S3, the network entity uses the assistance information from the UE to determine what, if any, management instruction is to be sent to the UE. The network entity based on the received assistance information may evaluate the correctness of measurement and data which are provided by the UE. The management instruction may take any suitable form and may be for example a request for the UE to make more measurements, make a change to its configuration, and / or to use synthetic data.

[0109] As referenced S3, the network entity sends the management instruction to the UE. The UE on receiving the management instruction will operate in accordance with that instruction.

[0110] For a BM (beam management) use case, in a non-machine learning scenario, a network entity may uses an evaluation score to cluster the UEs. Those UEs with bad measurements will be identified based on the evaluation score metric. Those UEs may be grouped and requested to do more measurements for getting a better measurement. In the following examples, a method for determining an evaluation score which may be applied in non-machine learning examples.

[0111] Feature quality, missing data, outliers, and discrepancy-aware level are related to the data characteristics. In the context of a ML model, a feature is an input to the ML model. This information may provide the network entity with insights into the quality of measurements and sampling rates. In other words, it can also provide the network entity with information about the hardware and measurement algorithms of the UE device without requiring detailed knowledge of their implementation and hardware. For example, the measured RSRP (reference signal received power) which may be collected for beam management may depend on the device hardware and the implemented algorithms. If the outlier is high, it means that the received measured data deviate from the underlying distribution estimated using the data received from all the other UEs. Hence, the UE hardware or algorithms might be not be able to measure the RSRP correctly. If the feature quality is low, it may demonstrate that the UE hardware cannot remove the noise from the receiving signal very well. If the discrepancy-aware level is low, it shows that the sampling rate of UE is low and not enough to capture the signal pattern. Consequently, gNB can consider such valuable assistance information about RSRP while performing beam management even in a non-ML case.

[0112] Some embodiments may be used in machine learning (ML) applications.

[0113] In this regard, an overview of some example embodiments is now described with reference to FIG. 5.

[0114] By way of example, some embodiments may be used for federated learning (FL) applications.

[0115] Some applications in mobile telecommunication networks require a large amount of verified and labelled data from distributed sources. In the following examples, the distributed sources are UEs. The data from the distributed sources is used to train a single common ML model. To minimize or even remove the training data exchange among collaborating network entities, FL may be applied. FL is a form of collaborative machine learning where different versions of the global (FL) model are locally trained at the different distributed nodes and only parameters of the local models are exchanged in an iterative fashion. In the following example, the distributed nodes are UEs.

[0116] In each distributed node (UE) in a FL scenario has its own local training data which may not come from the same distribution as the data at other similar peer nodes. Each distributed node computes parameters for its local ML model and shares the computed parameters for the local ML model with an aggregator. In the described examples, the aggregator is a network entity such as an access node such as a base station or may be provided in a core network. The central host (that is the aggregator) does not compute a version or part of the model but combines parameters of all the distributed local models to generate a global model.

[0117] Thus after training a local ML model, each distributed node transfers its local model parameters (e.g., weight and bias values of a neural network), instead of a raw training dataset, to the aggregator. The aggregator utilizes the local model parameters from the distributed nodes to update a global model. The updated global model may be provided to the distributed nodes for further local model updating iterations until the global model converges. This converging may be defined by a one or more stopping criterion (e.g., a predefined number of iterations and / or a maximum tolerable value of the loss function to be minimized).

[0118] As a result, each local distributed node benefits from the datasets of the other local distributed node through the global model, shared by the aggregator. This is without having access to any data which might be privacy-sensitive of another distributed node.

[0119] The FL training process comprises an initialization phase where a machine learning model (e.g., linear regression or neural network) is chosen to be trained on distributed nodes and initialized at the aggregator.

[0120] There may be a client selection phase where all or a subset of local nodes are selected to start training the global model using local data. The selected distributed nodes acquire the current statistical model (global model) from the aggregator. The other distributed nodes will wait for selection in another federated round.

[0121] Each selected node sends its locally updated version of the model to the aggregator for aggregation. The selected nodes may provide the updated parameters of the model. These updated parameters may comprise the weights and / or biases of the mode. The aggregator aggregates the received models and sends back the model updates to the distributed nodes. This process may be repeated iteratively.

[0122] The training phase may be terminated once a pre-defined termination criterion is met (e.g., a maximum number of iterations is reached). The aggregator aggregates the updates and finalizes the global model.

[0123] FL allows the training dataset to be kept where it is generated (that is at the distributed node) and training of the model to be done at the distributed node. This means that the raw data does not need to be shared with the aggregator. This means that the privacy with respect to the raw data can be ensured.

[0124] Model aggregation may be performed by a centralized aggregator.

[0125] In other embodiments, the model aggregation may be performed by one or more specific peer entities that collect model data from their neighbours for aggregation where interlearner communication is an option.

[0126] In some embodiments, a UE may train, re-train, or finetune, models locally at UE side. This can help to reduce the amount of data transmitted over air interface by using local training data. However, this may not be suitable when a UE is in a fast-changing radio and non-radio environment from where the input data is collected, and every change in environment requires potentially a new ML model version for optimal inference results. Some embodiments may support the providing of these new ML model versions supporting changes in the environment of the UE.

[0127] Generally with ML techniques, the quality of the ML technique is dependent on the reliability and usefulness of the data. For example, in supervised learning, where models are trained on labelled data sets, the accuracy, quality, relevance, and / or completeness of the data may directly influence the performance of the model.

[0128] In the global aggregation carried out by the aggregator, the quality of the local extracted knowledge (from the distributed nodes) needs to be considered. This allows, for example, for the penalization of erroneous and / or noisy contributions. Hence, the impact of erroneous and / or unreliable data on collective knowledge may be mitigated. This may enhance the accuracy and utility of the aggregated information for subsequent decision-making processes.

[0129] Thus, in FL, the goal of an FL central aggregator (FL CA) is to produce a unified global model that is based on averaging parameters across all participating UEs in the FL training. Due to non-independently and identically distributed (non-IID) local training data in each UE, along with potential noisy, missing, and / or biased features / labels, performance and / or reliability of a resultant global model may vary between UEs if a FL trained model is not fully converged.

[0130] If training data has a relatively high variance, training a FL model may take a long time and may consume a lot of network resources to collect the parameters of the local models in hundreds or even thousands of iterations. This may mean that for a UE which has access to suitable data, the performance of the UE may suffer (or biased) if that UE participates in global FL training.

[0131] One approach is to assign more weight to the model parameters received from a UE with “better” training data in the aggregation performed by the aggregator. By providing more weight to the model parameters from such UEs may mean that these UEs are significant in aggregated FL model training.

[0132] In some embodiments, a UE will provide assistance information to the aggregator. Examples of assistance information will be described in more detail below. In some embodiments, the assistance information which is provided is such that the privacy of the data of the UE is maintained. The assistance information may be a statistical measure or statistical parameter of the quality of the data without disclosing information about the raw data itself. The privacy of the raw data is maintained. The assistance information describes the quality of the data used in training and / or inference of the ML model. The assistance information which is provided by the UE may be such that at least some of the private information is abstracted to provide privacy for the UE. The assistance information allows the gNB (or other aggregator)to identify the quality of the data used for training and / or inference at a given UE.

[0133] For some distributed nodes, their training data may be sufficiently different from the training used by other distributed nodes such that this may jeopardize the FL model convergence. These may be regarded as “comer cases”. For example, this may be a result of a distributed node being in a particular location. The use of such different data may introduce bias data for all other models and may jeopardize FL model aggregation by introducing bias.

[0134] Some embodiments may recognize such ‘comer cases’ and treat their local models accordingly in data aggregation.

[0135] In some embodiments, a UE may send assistance information to the aggregator and, in response, may receive one or more management instructions from the aggregator.

[0136] Examples of management instructions and assistance information will be discussed later in this document. In some embodiments, where assistance information is received from a plurality of UEs, weights are determined by the aggregator for each UE. Based on weights, management instructions are sent to one or more of the UEs.

[0137] Thus, in some embodiments, assistance information is provided from a UE to the aggregator.

[0138] In some embodiments, the management instructions provided by the aggregator to a UE may be a notification to the UE that new measurements are required and to be reported.

[0139] The management instructions may be information that the UE is to improve its measurements. For example, the management instructions may comprise a reason associated with a request from the UE relating to additional measurements. The management instruction may be an indication that the UE is to improve one or more aspects relating to its measurements.

[0140] The UE may be configured to take the required action or actions in response to the management instructions.

[0141] In some embodiments, the number of UEs required to report can be reduced. This may reduce the required signalling in a cell, for example. For example, the aggregator can prioritize one or more UEs which are to collect measurements.

[0142] In some embodiments, one example of assistance information comprises a discrepancy- aware level. A discrepancy-aware level may abstract (non-inversibly) both biases (which can be impacted by imbalance local data distribution) and local data set size, without disclosing any privacy-endangering information. The discrepancy aware level relates to the number of measurements made by the UE and the deviation of the distribution of the actual measurements from a theoretical uniform distribution of measurements.

[0143] For the discrepancy-aware level, each participating UE needs to compute deviation of its local category distribution (e.g., using L2 (squared error loss) difference or KL(Kullback- Leibler)-Divergence) with a uniform distribution. This may promote fairness across all categories and the generalization capability of the global model. The distribution deviation factor together with the sample size, for examples as formulated and proposed in Ye, Rui, et al. "FedDisco: Federated Learning with Discrepancy -Aware Collaboration." arXiv preprint arXiv:2305.19229 (2023).] has been shown to improve the aggregation approach using solely dataset sample size, theoretically and empirically. Each UE needs to compute the deviation (using L2 norm or KL-Divergence) of its data distribution with the uniform distribution (to promote fairness across all categories). Then this deviation measurement and number of local data may be used to abstract both of these values as discrepancy aware level. Hence, a UE may send this discrepancy-aware level to the aggregator instead of data sample size and biases. This approach abstracts non-inversibly the dataset size and the distribution deviation from the uniform distribution. Thus, this means that the UE does not need to reveal any confidential information to the aggregator. This approach may prioritize UEs with larger dataset sizes and / or smaller imbalance data distribution.

[0144] Other examples of assistance information may relate to data characteristics such as features quality, prediction confidence, prediction correlation coefficient, missing values, and outlier.

[0145] Features quality- this is one criterion for evaluating data quality and is the number of independent features that can be informative for a target (sometime referred to a response or ground truth). In the case of a beam management example, the features may comprise one or more Ll-RSRP, UE velocity, beam prediction accuracy, and UE throughput. The UEs need to compute mutual information (MI) among each pair of features and among each feature and the target. A higher number of features with lower mutual information, and a greater number of features exhibiting higher mutual information with respect to the target variable, may be preferred. The number of informative features in a dataset can impact the performance of the model.

[0146] The aggregator may assign higher weights to the model parameters of those participating UEs which have access to more informative features. A UE may need to compute average MI among the feature pairs (avg MI features) and average MI among the MI between each feature and target (avg MI target). The UE may determine an avg MI target / avg MI features, as a normalized quantification of features quality. This may be provided as a measure of the feature quality to the aggregator.

[0147] Prediction confidence is one criterion for evaluating sampling error. Sampling error is an inferential noise types that can be measured via a confidence interval. Sampling error refers to the discrepancy between a sample statistic and the true population parameter it represents, which arises due to random variation in the selection of samples. It can be quantified using a confidence interval. A UE can compute a confidence interval corresponding to their prediction using approaches such as confidence interval and uncertainty quantification. Confidence interval and uncertainty quantification can be estimated by employing Bayesian methods. Specifically, Bayesian approaches establish a probability distribution over the model parameters, allowing for assessment of uncertainty by integrating across all possible configurations of these parameters. This integration provides a probabilistic measure of uncertainty rather than a fixed value, . Then, a UE may send the averaged computed confidence interval to the aggregator. If a participating UE has a lower confidence over its prediction, its contribution may be penalized or given less weight by the aggregator. This may give an aggregator an overview of the reliability of the training process in each UE. The reliability may be taken into account by the aggregator.

[0148] Outliers can distort the patterns and relationships present in the data and this may lead to inaccurate models. Outliers can disproportionately affect the parameters of a model and the predictions provided by a model. This may result in a poor performance. A UE can estimate the percentage of the outliers in its available data. The outliers are the outlying measurements of the measurements made by the UE. Some approaches such as DBSCAN (Density -based spatial clustering of applications with noise) can be used to observe density of the data and detect those data that do not belong to any of the data clusters and indicate them as outliers. DBSCAN identifies outliers by clustering data points based on their density, distinguishing outliers as noise points that do not belong to any cluster, thus aiding in the detection of outlying measurements in the data collected by the UE. The UE will report a number of outliers (or a percentage of outliers) to the aggregator. The aggregator may penalize or give a lower weight to a contribution of a UE with a relatively high number of outliers. This may avoid building an aggregated model affected (e.g. overfitting) by the outliers.

[0149] Missing values - a UE may send information indicating a percentage of missing data in their local data sets. This may be a measure of the number of missing measurements from a local data set.

[0150] Prediction correlation coefficient - this is a statistical measure that quantifies the strength and direction of the relationship between two variables. To evaluate predictive models, the correlation coefficient between ground truth and predictions is computed. This may be provided as the prediction correlation coefficient in some embodiments.

[0151] In the context of classification, the Matthews Correlation Coefficient (MCC) may be useful. MCC handles class imbalance well because it takes into account all four values from a confusion matrix (TP (true positive), TN (true negatives), FP (false positive), FN (false negative)). When dealing with imbalanced datasets, the Fl score may be biased towards the majority class, as it is heavily influenced by the recall of the minority class. MCC provides a more balanced assessment irrespective of class distribution. MCC may be used for measuring correlation in discrete values (classification type of problems)

[0152] In regression analysis, the Spearman Rank Correlation Coefficient (SRCC) may be employed to assess the relationship between the predicted values and the true values. This may be provided as the prediction correlation coefficient in some embodiments. The Spearman Rank Correlation Coefficient is based on the ranks of the values rather than their actual values. This makes it suitable for assessing the monotonicity or non-linear relationships in regression tasks. A higher Spearman correlation indicates that the model's predictions closely follow the ground truth, which is crucial for accurate regression modelling and predicting continuous outcomes. Spearman correlation captures monotonic relationships without distributional assumptions, making it more robust for non-linear or non-normally distributed data.

[0153] In embodiments the Pearson correlation may be used in measuring the correlation for continuous values (regression type of problems). Pearson correlation measures linear relationships between continuous variables assuming normal distribution

[0154] In some embodiments one or more of the SRCC, MCC and Pearson correlation are used In some embodiments a sum corelation coefficient is sent by the UE to the aggregator.

[0155] In some example embodiments, a plurality of predicted values may be determined. The SRCC may be determined for each value and then a sum or other combination metric or value of the SRCC values may be determined. The aggregator may use this determined coefficient when determining a weighting associated with the model parameters from the UE.

[0156] In some embodiments, the aggregator may cluster the UEs based on the assistance information.

[0157] In some embodiments, an evaluation score is determined. This may be determined by each UE and may be sent by each UE instead of or as well as the raw assistance information. In other embodiments, the aggregator may determine the evaluation score for each UE.

[0158] One example of the determination of an evaluation score is as follows: Discrepancy-aware level is denoted as Dsc, features quality is denoted as n_q, number of outliers is denoted as n o, percentage of missing data is denoted as n_m, prediction correlation coefficient is denoted as cc, and accuracy is denoted as s acc. The UEs may be clustered based on an ES determined as follows:

[0159] ES = (wl * Dsc + w2 * n_q + w3 * cc + w4 * acc) / (1 + w5 * n_o + w6 * n_m). Weighting wl-w6 can be adjusted based on the importance of each criterion in grouping.

[0160] In this example, there are the noted six examples of assistance information. In other embodiments, less than six examples of assistance information may be provided. For example there may be one or more examples of assistance information. There may be more than six examples of assistance information.

[0161] In some embodiments, the one or more examples of assistance information may be taken from the noted six examples of assistance information. However, in other embodiments, one or more different examples of assistance information may be used. In some embodiments, the evaluation score may be based on a different equation.

[0162] The aggregator may be configured to divide the UEs into a fixed number of clusters or groups. In other embodiments, the aggregator may be configured to determine the number of groups or cluster using any suitable technique. For example, the aggregator may use an algorithm or technique such as K mean clustering, DBSCAN, Gaussian mixture models, or the like. The technique may depend on the application and / or implementation.

[0163] The aggregator may determine a number of groups based on the distribution of the evaluation scores which can be for example linear or logscale. One grouping approach is that after collecting all the evaluation scores, UEs with evaluation scores (ES) values in the first 25% percentile be placed in the first group, UEs with evaluation score from 25% percentile to 50% percentile in the second group, etc. It should be appreciated the size of the percentile associated with each group will depend on the number of groups. The above example is where there are four groups.

[0164] Thus in some embodiments, UEs can be grouped based on their evaluation score and without the UE needing to disclose any confidential information. UEs with similar evaluation scores will be put in the same group.

[0165] Reference is made to FIG. 5 which shows an example where the UEs are put into different groups based on their evaluation score. In the example shown in FIG. 5, there are three groups. Group 1 comprises UE1, UE2 and UE3. Group 2 comprises UE4 and UE5. Group 3 comprises UE6, UE7, UE8 and UE9.

[0166] In this example, for each group, an average will be computed over the model parameters received from the UEs. Thus each model parameter is associated with an average for the respective group - referenced 500, 502 and 504 in FIG. 5.

[0167] The average weights of each group will be weighted. The weight will be based on the evaluation score of that group. For example, this may be based on an average of the evaluation score, a maximum evaluation score for the group or on any other metric associated with the evaluation scores of the group. This is referenced 506 in FIG. 5.

[0168] The average model parameters of each group are combined in dependence on their weight to update the respective model parameters of the global model.

[0169] It should be noted, that based on evaluation score, those UEs which have a low evaluation score may be requested to perform new measurement or enhance their measurement.

[0170] Thus, in some example embodiments, after the aggregator determines the evaluation score based on the received assistance information from the UEs, the UEs will be grouped by the aggregator based on their evaluation score. For example, the UEs with evaluation scores in the first 25% percentile in group 1, in the 25%-50% percentile in group 2, etc. Within each group, the simple averaging needs to be computed. The simple averaging ensures that slight deviation of the parameters sent from UEs to the aggregator will not make any difference in their contribution. Then, a weight is given to each group which, for example, is equal to the maximum evaluation score found in that group. The aggregator the performs a weighted average over the group-averaged models based on their assigned weights. Hence all UEs, even the ones with deviating weights still will contribute to the final aggregation, considering all their data characteristics.

[0171] The aggregator has the freedom to determine the number of groups using any suitable algorithms such as agglomerative, divisive algorithms, and / or the line., which can vary based upon the use cases and implementation. As an example, consider a scenario where there are 7 UEs with scores 0.0001,0.1, 0.2, 0.4, 0.5, 0.7, 0.9. It is assumed their weights are the normalization (sum up to 1) of their score. Their normalized weights are: 0.000036, 0.0357,0.0714, 0.1786, 0.1429, 0.2500, 0.3214. With the proposed grouping, first the UEs are grouped based on the maximum score of each group as groupl : 0.0001,0.1, 0.2; group2: 0.4,0.5; group3: 0.7, 0.9. The max scores of groups are as: 0.2, 0.5, 0.9. If weights are assigned to the groups based on their maximum group weight score, the normalized weights computed for the groups are as: 0.125, 0.3125, 0.5625. The hierarchical weighting aggregation of some example embodiments avoid some UEs being excluded from the aggregation by assigning extremely low weights to them or dominating the aggregation by obtaining extremely high weights.

[0172] With this hierarchical solution, the UEs will be ranked based on their cluster evaluation score which determines for a respective UE if there is need for doing retraining or data collection. This avoids requesting a relatively high number of UEs to perform new measurements which may introduce delay and a lot of associated signaling. Hence, by the hierarchical solution, the max score of different clusters will be considered.

[0173] Grouping local models enables the aggregator to design and allocate suitable weights to UE groups based on their data characteristics. Grouping is useful where there are many UEs with diverse capabilities, as it avoids overlooking UEs who have access to privileged information which may cause their parameters to deviate from the aggregated model.

[0174] The clustering may provide one or more of improved convergence speed, reduced communication overhead, enhanced privacy and security, robustness to heterogeneity, resource efficiency, adaptive grouping, fault tolerance and / or energy efficiency. Some example procedures of some embodiments are now described with reference to

[0175] FIG. 6.

[0176] As referenced 0, the network entity configures the UE to report its resource profile. This may comprise computational resources and / or memory resources The gNB may configure the UE to report training data set information - that is assistance information about the training data set used with the ML model at the UE. The assistance information may comprise one or more of the assistance information discussed in this document.

[0177] As referenced 1, the UE may report its resource profile and assistance information relating to the training data used to train its machine learning model.

[0178] As referenced 2, the network entity determines the evaluation score for each UE, based on the received assistance information.

[0179] As referenced 3, the network entity will cluster or group the UEs based on the evaluation score such as previously described.

[0180] As referenced 4, the network entity may send a message to the UE. The network entity may select one or more UEs with low evaluation score and request the UE to enhance their measurements. This may be regarded as one example of an instruction in a measuring and reporting configuration message.

[0181] The network entity may send an indication to the UE, of the cause of data quality issues and / or an indication of the relative data quality of the measurements made by the UE, in comparison to those made by other UEs.

[0182] The UE may be informed about the reason of the new request from network entity for further measurements, for example. In response the UE may take action to improve its measurements. For example the UE may increase a sampling rate, change an applied filtering, change the configuration of the measuring hardware and / or the like.

[0183] In some embodiments, the UE may be instructed to use synthetic data instead of its own data. This may be case where the data collected by the UE is considered to be of a relatively low quality.

[0184] In some embodiments, the gNB will inform the UE about options. The options may be to use a use a new measurement configuration or to use synthetic or any other suitable option.

[0185] As referenced 5, the UE may request an updated measurement configuration from the UE. For example, where the gNB has provided the UE with one or more options, the UE may indicate which one or more option is preferred.

[0186] In some embodiments, this part of the procedure may be omitted. For example, the parts of the procedure referenced 4 and 6 may be combined. As referenced 6, the network entity may provide the configuration associated with the measurements to be made by the UE and / or the reporting to be provided. This may define what measurements are to be made and / or one or more UE related configuration parameters which are to be used and / or the reporting to be provided by the UE.

[0187] As referenced 7, the UE performs measurements.

[0188] As referenced 7, the UE sends updated assistance information.

[0189] As referenced 8, the network entity sends the global model parameters to the UE. These may be the aggregated parameters which the network entity has determined.

[0190] As referenced 9, the UE will apply the parameters to it ML model. The UE will then train the local model using local data available to the UE. This will result in the parameters of the local ML model being updated.

[0191] As referenced 10, the UE reports the model parameters of the updated local ML model to the network entity.

[0192] As referenced 11, the UE sends performance information associated with the model. This performance information may indicate if the model is performing well. One example of performance information may be a mean squared error (MSE).

[0193] As reference 12, the network entity uses the data from the UEs to apply a hierarchical weighted aggregation over the grouped UEs. This may be as discussed in relation to FIG. 5.

[0194] In some embodiments, the network entity may compare the evaluation score based on assistance information from different UEs. After evaluation, the network entity may optionally, indicate to the UEs that are below certain evaluation score, that they cannot participate in the FL task. Those UEs can then perform additional measurements and can send updated assistance information to allow the UE to participate in FL task. The network entity may update the clustering information.

[0195] Some example embodiments will now be described in which a ML model is us for beam management. Some embodiments may relate to DL Tx beam prediction. Some embodiments may relate to the DL TX- Rx’ beam pair prediction. Some embodiments may relate to the UE- sided model and / or NW-sided model.

[0196] In these example embodiments, the management instruction may be a result of model / functionality performance monitoring at the network.

[0197] The management instruction may include information about the ML model or functionality.

[0198] As described previously, some embodiments provide mechanisms for a network entity (aggregator) to monitor AI / ML models in the UE. Referring to back to FIG. 4, this can be used in the context of ML model training. For example the ML model training and / or inference are in the UE. The RAN (for example the access node or gNB) configures the UE to report assistance data in the form of measurements and other reports. The UE reports the assistance data to the RAN.

[0199] Hence, based on the assistance information and evaluation score such as described previously, the gNB is enabled to evaluate data quality for both training (e.g. collected RSRPs for training) and / or inference (e.g. input query for inference). Consequently, gNB may be able to evaluate trustfulness of a UE prediction. The gNB may request the UE to enhance its measurement configuration, repeat measurements, and / or retrain.

[0200] Some embodiments may allow the network entity to request the UE to enhance the measurement tools or repeat the measurements before requesting the retraining from UE.

[0201] In some embodiments the aggregator (gNB) can compare the evaluation score based on assistance information from different UEs. After evaluation, gNB can optionally, indicate to the UEs that are below certain evaluation score and cannot participate in FL task. Those UEs can then perform additional measurements and can send updated assistance information to participate in FL task. The clustering may be updated when the UE has provided additional measurements and provided updated assistance information.

[0202] The gNB may indicate to the UE information relating to a measurement deficiency.

[0203] Reference is made to FIG. 7 which shows a method flow of some example embodiments where the ML model is used for beam management.

[0204] As referenced 0, the UE has a machine learning model. In this example, the machine learning model is used for beam management BM. The input to the BM ML model may be measurements, e.g. RSRP measurements on a set of beams. The output of the ML model may be a probability of a beam to be a “top” beam or a predicted RSRP measurement. Beam management may be to provide beam prediction in time, and / or spatial domain. This may be reduce overhead and / or latency and / or to improve beam selection accuracy.

[0205] As referenced 1, the network entity, for example the access node or gNodeB sends a message to the UE with configuration information. This may be in a measurement and / or reporting configuration message. The configuration information may indicate the measurements which are to be made by the UE, data which is to be collected by the UE, and / or the data which is to be reported by the UE. In the case of a BM scenario, the UE may be configured to measure RSRPs associated with different beams. The configuration information may indicate the assistance information to be reported. As referenced 2, the UE collects measurements as defined by the configuration information.

[0206] As referenced 3, the UE sends a report to the network entity. This may comprise the assistance information and optionally ML parameters of the local model.

[0207] As referenced 4a and 4b, the network entity, determines an evaluation score for the UE and clusters the UEs based on the evaluations scores. This may be as discussed previously.

[0208] As referenced 5, the network entity determines that one or more UEs are to provide updated measurements and may need to use a different configuration. The network entity will send updated configuration information to the UE requesting the UE to make further measurements. If the UE is to use a different configuration, this may be included in the updated configuration information, This may be provided in an updated measurement and reporting configuration message.

[0209] As referenced 6, the UE performs additional measurements and the UE sends a report to the network entity. This will comprise the assistance information and optionally ML parameters of the local ML model of the UE.

[0210] As referenced 6a and 6b, the network entity, updates the ML model parameters such as previously described. The network entity may determine a model performance report. The network entity may compare the reported information from the different UEs and based on this comparison, provide a UE with an indication of the performance on that UE compared to other UEs. By way of example, the network entity may indicate in what percentile of performance the UE is, e.g. the UE is among best 20% UEs or indicate its ranking or any other suitable measure of relative performance.

[0211] As referenced 7, the network entity, sends model related data to the UE. This may comprise updated model parameters and optionally the model performance report.

[0212] As referenced 8, the UE applies the parameters to the local ML model.

[0213] As referenced 9, the UE uses the ML model for DL beam management, that is to determine one or more DL beams to be used for communication between the UE and the gNodeB B.

[0214] An example of the discrepancy aware level in the BM case will now be described.

[0215] In some embodiments, the discrepancy aware level may be denoted as Pk which is calculated as follows:

[0216] Pk = nk - a*dk + b where Pk is a function of nk which denotes number of training examples (number of Ll-RSRP measurements per beam k in this use case) of the kth UE, dk denotes discrepancy of the data distribution computed at the UE side (i.e. dk is computed as the difference e.g. L2 distance between the uniform distribution and the current available historic Ll-RSRP measurements distribution in the UE). The values a and b are tuning parameters and depend on the UE in question.

[0217] As an example, consider a setup with B = 10 Beams, t = 100 historic Ll-RSRP measurements that have been collected as the training data at the UE. An ideal data at the UE would follow a uniform distribution, in other words, there is a same number of Ll-RSRP measurements for all the beams, in this case dk will be zero. However, at the UE usually there is a different number of Ll-RSRP measurements for each beam, due to different sampling rates, measurement hardware, etc. In other words, the distribution of the classes (different beams) is not uniform in real scenarios, as it is likely that UE has more data for a specific beam than others (i.e. l<=dk >0).

[0218] In a scenario where a=1000, b=l, dk= 0.7 and nk=1000, then: Pk=1000-1000*0.7+l = 301, whereas if dk is 0 (ideal case) Pk =1000-1000*0+1 = 1001. A higher pk promotes an equal number of Ll-RSRP measurements for all beams which will not be biased towards a specific beam and may be preferred.

[0219] Pk may need to be normalized at the gNB. For example this may be done by using [Pk - (Pk min)] / [Pk max - Pk min], where Pk min is the minimum Pk values received and Pk max is the maximum Pk values received from all UEs.

[0220] As described above, in * Pk range is [0,+l ], where 0 means the data set either does not have high number of training examples (nk: total number of Ll-RSRP measurements) or it has a very biased data set (some of the beams have very few Ll-RSRP measurements while others have more). The metric provided to the gNB may therefore be between 0 and 1.

[0221] Pk is abstracting both the number of training examples and (pseudo-)distance between the distribution of the classes of the data at hand at each UE with the ideal case where there is the same number of Ll-RSRP measurements for each beam.

[0222] In this case, no local data is shared between the local (client) and global (server) models. Therefore, privacy is preserved to some extent, and data leaking can be avoided. Moreover, UE will not give the information about the number of samples and its sampling rate.

[0223] When all clients (UE) use the same local data for the local models and global models, then the discrepancy level is zero. The weights are the same and the only dependency remains on the data size. On the other hand, if some UEs are not able to provide rich datasets or discrepancy level is quite high, then by using assistance information such as discussed, the global model can give the parameters from that UE a low weight. In the example embodiments described previously, the distributed nodes are UE and the aggregator is an access node such as a gNB. In other embodiments, the aggregator may be provided by a server or other network entity. The server or other network entity may be provided in a core network.

[0224] In other embodiments, the distributed nodes may be access nodes such as base stations, for example gNBs. In this example, the aggregator may be a network entity.

[0225] In the example embodiments described in relation to machine learning, the machine learning technique described in FL. It should be appreciated that other embodiments may be used with other machine learning techniques. For example, some embodiments may be used with other machine learning algorithms, such as, supervised learning algorithms or reinforcement learning.

[0226] Other embodiments may be used in multi agent reinforcement learning (MARL). In this scenario, in the collaborative setup, each agent is interacting with the environment and the quality of the collected experimentations, local policies, reward functions, and / or the like significantly impact the global policy update and consequently convergence in solving the underlying optimization problem. According, assistance information such as described previously can be used in determining a weighting of the data used for the global policy update.

[0227] Other embodiments may be used with supervised learning techniques.

[0228] In some of the described examples have been in the context for the training of a ML model. It should be appreciated that alternatively or additionally, the examples may be modified to be used for inference.

[0229] Some example embodiments have been described in relation to BM applications. Other embodiments may be used for CSI (channel state indicator) feedback enhancement, e.g., to provide overhead reduction, improved accuracy, and / or prediction.

[0230] In this example, the measurements and / or data collected may comprise one or more of RSRP, channel-quality indicator (CQI), rank indicator (RI), precoder-matrix indicator (PMI) and / or the like., The UE may measure these on the CSLRS (C Si-Reference Signal) transmitted by the RAN.

[0231] Other embodiments may be used for positioning accuracy enhancements for different scenarios including, e.g., those with heavy NLOS (non-line of sight) conditions. In this example, the measurements and / or data collected may comprise one or more of the UE location itself (as the AIML model output), assistance data that could be used by the network in UE positioning (for example - this could include LOS / NLOS identification), timing, angle of measurement, likelihood of measurement and / or the like. FIG. 2 illustrates an example of an apparatus 900.

[0232] The apparatus may be or provide an aggregator, such a previously described. The aggregator may be in an access node, for example, a gNB or may be provided by a network entity or server. The apparatus 900 may have at least one processor and at least one memory storing instructions that, when executed by at least one of the at least one processor cause operations or actions of at least a part of the aggregator to be provided.

[0233] The apparatus 900 may comprise at least one random access memory (RAM) 91 la, at least one read only memory (ROM) 911b, at least one processor 912, 913 and a network interface 914. The at least one processor 912, 913 may be coupled to the RAM 911a and the ROM 91 lb. The at least one processor 912, 913 may be configured to execute an appropriate software code 915.

[0234] Execution of the software code 915 may for example cause the apparatus to perform operations for providing one or more of the described aspects. The software code 915 may be stored in the ROM 911b.

[0235] FIG. 3 illustrates an example of a communication device 1000, such as the UE illustrated on FIG. 1 (and referenced 202, 204, 208 and 208). The communication device 1000 may be provided by any device capable of sending and receiving radio signals. Non-limiting examples of a communication device 1000 comprise a user equipment, a mobile station (MS) or mobile device such as a mobile phone or what is known as a ’smart phone’, a computer provided with a wireless interface card or other wireless interface facility (e.g., USB dongle), a personal data assistant (PDA) or a tablet provided with wireless communication capabilities, a machine-type communications (MTC) device, an Internet of things (loT) type communication device or any combinations of these or the like. The communication device 1000 may comprise a transceiver for transmitting and / or receiving, for example, wireless signals carrying communications, for example radio signals. The communications may be one or more of voice, electronic mail (email), text messages, multimedia data, machine data and so on.

[0236] The communication device 1000 may receive wireless signals (e.g., radio signals) over an air or radio interface 1007 via appropriate apparatus for receiving and may transmit wireless signals via appropriate apparatus for transmitting radio signals. In FIG. 3, the transceiver is designated schematically by block 1006. The transceiver 1006 may comprise, for example, a radio part and associated antenna arrangement. The antenna arrangement may be arranged internally or externally to the mobile device and may comprise one or more antenna elements. The antenna arrangement may be a multi-input multi output (MIMO) antenna. The communication device 1000 may be provided with at least one processor 1001, at least one memory (for example ROM 1002a and RAM 1002b) and other possible components 1003 for use in software and hardware aided execution of tasks it is designed to perform, including control of access to and communications with access networks and other communication devices. The at least one processor 1001 is coupled to the RAM 1002b and the ROM 1002a. The at least one processor 1001 may be configured to execute an appropriate software code 1008. The software code 1008 may for example allow to perform one or more operations of the communication device 1000. The software code 1008 may be stored in the ROM 1002a. The processor, the ROM, and the RAM, the transceiver and other circuitry of the communication device (e.g., a modem) can be provided on a circuit board, in chipsets, or in a system on chip. This circuit board, chipsets or system on chip is denoted by reference 1004. The communication device 1000 may optionally have a user interface such as keypad 1005, touch sensitive screen or pad, combinations thereof or the like. Optionally one or more of a display, a speaker and a microphone may be provided depending on the type of the communication device.

[0237] Reference is made to FIG. 8 which show a method of some example embodiments.

[0238] This method may be performed by an apparatus.

[0239] The apparatus may comprise or be a network entity. The apparatus may be an access point or a base station.

[0240] The apparatus may comprise suitable means, such as circuitry for providing the method. Alternatively or additionally, the apparatus may comprise at least one processor and at least one memory storing instructions that, when executed by the at least one processor cause the apparatus at least to provide the method below.

[0241] Alternatively or additionally, the apparatus may be such as discussed in relation to FIG. 2.

[0242] The method may be provided by computer program code or computer executable instructions.

[0243] The method may comprise as referenced Al receiving, from a plurality of user equipment, information relating to one or more statistical parameters, the one or more statistical parameters relating to measurements made by the respective user equipment.

[0244] The method may comprise as referenced A2, determining, based on the received information, measurement and reporting configuration information for one or more of the plurality of user equipment. The method may comprise as referenced A3, sending the measurement and reporting configuration information to the one or more of the plurality of user equipment .

[0245] Reference is made to FIG. 9 which show a method of some example embodiments.

[0246] This method may be performed by an apparatus.

[0247] The apparatus may comprise or be a user equipment..

[0248] The apparatus may comprise suitable means, such as circuitry for providing the method.

[0249] Alternatively or additionally, the apparatus may comprise at least one processor and at least one memory storing instructions that, when executed by the at least one processor cause the apparatus at least to provide the method below.

[0250] Alternatively or additionally, the apparatus may be such as discussed in relation to FIG. 3.

[0251] The method may be provided by computer program code or computer executable instructions.

[0252] The method may comprise as referenced Bl, determining one or more statistical parameters, the one or more statistical parameters relating to measurements made by the user equipment.

[0253] The method may comprise as referenced B2, sending information relating to the one or more statistical parameters to a network entity.

[0254] The method may comprise as referenced B3, receiving from the network entity, measurement and reporting configuration information.

[0255] It is noted that whilst some embodiments have been described in relation to 5G networks, similar principles can be applied in relation to other networks and communication systems. Therefore, although certain embodiments were described above by way of example with reference to certain example architectures for wireless networks, technologies and standards, embodiments may be applied to any other suitable forms of communication systems than those illustrated and described herein.

[0256] It is also noted herein that while the above describes example embodiments, there are several variations and modifications which may be made to the disclosed solution without departing from the scope of the present invention.

[0257] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements. In general, the various embodiments may be implemented in hardware or special purpose circuitry, software, logic or any combination thereof. Some aspects of the disclosure may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the disclosure is not limited thereto. While various aspects of the disclosure may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0258] As used herein, the term “circuitry” may refer to one or more or all of the following:

[0259] (a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry) and

[0260] (b) combinations of hardware circuits and software, such as (as applicable):

[0261] (i) a combination of analog and / or digital hardware circuit(s) with software / firmware and

[0262] (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and

[0263] (c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.”

[0264] This definition of circuitry applies to all uses of this term herein, including in any claims. As a further example, as used herein, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0265] The embodiments of this disclosure may be implemented by computer software executable by a data processor of the mobile device, such as in the processor entity, or by hardware, or by a combination of software and hardware. Computer software or program, also called program product, including software routines, applets and / or macros, may be stored in any apparatus-readable data storage medium and they comprise program instructions to perform particular tasks. A computer program product may comprise one or more computerexecutable components which, when the program is run, are configured to carry out embodiments. The one or more computer-executable components may be at least one software code or portions of it.

[0266] Further in this regard it should be noted that any blocks of the logic flow as in the FIGs may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on such physical media as memory chips, or memory blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD. The physical media is a non-transitory media.

[0267] The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

[0268] The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The data processors may be of any type suitable to the local technical environment, and may comprise one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASIC), FPGA, gate level circuits and processors based on multi core processor architecture, as non-limiting examples.

[0269] Various example embodiments of the disclosure may be practiced in various components such as integrated circuit modules. The design of integrated circuits is by and large a highly automated process. Complex and powerful software tools are available for converting a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.

[0270] The scope of protection sought for various example embodiments of the disclosure is set out by the independent claims. The example embodiments and features thereof, if any, described in this disclosure that do not fall under the scope of the independent claims are to be interpreted as examples useful for understanding various example embodiments of the disclosure.

[0271] The foregoing description has provided, by way of non-limiting and illustrative examples, a full and informative description of the various example embodiments of this disclosure. However, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings and the claims. However, all such and similar modifications of the teachings will still fall within the various example embodiments of the disclosure as set forth in the claims. By way of non-limiting and illustrative example, there is a further example embodiment comprising a combination of one or more example embodiments with any of the other example embodiments previously discussed.

Claims

CLAIMS1. A method comprising: receiving, from a plurality of user equipment, information relating to one or more statistical parameters, the one or more statistical parameters relating to measurements made by the respective user equipment; determining, based on the received information, measurement and reporting configuration information for one or more of the plurality of user equpment; and sending the measurement and reporting configuration information to the one or more of the plurality of user equipment.

2. The method as claimed in claim 1, wherein the one or more statistical parameters provide a measure of a quality of the measurements made by the respective user equipment.

3. The method as claimed in any preceding claim, wherein the one or more statistical parameters comprise one or more of: outlier information relating to outlying measurements of the measurements made by a user equipment; feature quality information related to one or more features, the feature quality information providing a measurable property of the measurements made by a user equipment; prediction confidence information relating to a prediction provided by a user equipment; prediction correlation coefficient information relating to a prediction provided by a user equipment; missing values information indicating an amount of missing data from of the measurements made by a user equipment; and / or discrepancy-aware level providing information about a deviation of a distribution of the measurements made by a user equipment from a uniform distribution.

4. The method as claimed in any preceding claim, comprising using the respective information relating to the one or more statistical parameters for a respective one of the plurality of user equipment to determine weight information for the respective user equipment, and determining the measurement and reporting configuration information for one or more of the plurality of user equipment based on the respective weight information.

5. The method as claimed in claim 4, comprising determining the weight information based on an evaluation score for each of the plurality of user equipment, said evaluation score being determined based on a respective weighting applied to the one or more statistical parameters.

6. The method as claimed in claim 5, comprising allocating each user equipment to a group of a plurality of groups based on the determined evaluation score, the measurement and reporting configuration being associated with a respective one or more of the plurality of groups.

7. The method as claimed in any preceding claim, comprising receiving from the plurality of user equipment, updated local machine learning model parameters of a local machine learning model and using at least some of the updated local machine learning model parameters to update global machine learning model parameters of a global machine learning model.

8. The method as claimed in claim 7 when appended to claim 6, comprising determining for each of the plurality of groups a set of updated local machine learning model parameters.

9. The method as claimed in claim 8, wherein the set of updated local machine learning model parameters for a group is an average of the updated local machine learning model parameters provided by the user equipment in that group10. The method as claimed in claim 7 or 8, comprising determining a group weight associated with each group and using the group weight when combining the set of updated local machine learning model parameters of each group to provide updated global machine learning parameters of the global machine learning model.

11. The method as claimed in claim 10 when appended to claim 5, wherein the group weight associated with each group is dependent on a highest evaluation score associated with a group.

12. The method as claimed in any preceding claim, wherein the measurement and reporting configuration information comprises one or more of: information requesting the user equpment to perform further measurements; information about a configuration to be used by the user equipment; information about achanged sampling rate to be used by the user equipment when performing further measurements; information about a filtering to be applied by the user equipment to the further measurements; information about reporting further measurements; and information indicating that the user equipment is to use synthetic data with a machine learning model.

13. The method as claimed in claim 12, comprising providing information to a respective user equipment which has been requested to perform further measurements indicating a cause for the request for the performance of further measurements.

14. The method as claimed in claim 12 or 13, comprising providing information to a respective user equipment about a quality of the measurements of that user equipment compared to one or more other user equipment.

15. The method as claimed in claim 12 when appended to claim 8, comprising receiving, from one or more user equipment of a group requested to perform further measurements, information relating to one or more statistical parameters relating to the further measurements made by the respective user equipment and updated local machine learning model parameters, and using the updated local machine learning model parameters to update the set of updated local machine learning model parameters.

16. The method as claimed in any preceding claim when appended to claim 7, wherein the machine learning model is for providing machine learning enabled beam management and the measurements relate to beams.

17. The method as claimed in claim 16, wherein the machine learning model is for downlink transmission beam prediction or for downlink transmission reception beam pair prediction, and the measurements relate to reference signal received power.

18. The method as claimed in any of claims 1 to 15 when appended to claim 7, wherein the machine learning model is for providing machine learning enabled channel state indicator feedback and the measurements relate to reference signals.

19. The method as claimed in any of claims 1 to 15 when appended to claim 7, wherein the machine learning model is for providing machine learning enabled positioning and the measurements relate to positioning.

20. An apparatus comprising means for receiving, from a plurality of user equipment, information relating to one or more statistical parameters, the one or more different statistical parameters relating to measurements made by the respective user equipment; means for determining, based on the received information, measurement and reporting configuration information for one or more of the plurality of user equpment; and means for sending the measurement and reporting configuration information to the one or more of the plurality of user equipment.

21. A computer program comprising computer executable code which when run cause the method of any one of claims 1 to 19.

Images