Passive self-monitoring in ai / ML positioning

Abstract

A UE has a trained model used for positioning of the UE within a cellular network, and the UE extracts characterization for data that is input to the trained model for ML inference to form an inference data characterization. The UE compares the inference data characterization with training input data characterization from a characterization of training input data that were used to train a model that resulted in the trained model. The UE computes a metric for the trained model based on the comparison and sends indication of the metric from the UE toward an LMF in the cellular network. The LMF receives the indication of the metric, and determines one or more actions to perform based on at least the metric.

Description

PASSIVE SELF-MONITORING IN AI / ML POSITIONINGCROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application claims priority from, and the benefit of, US Provisional Application No. 63 / 625355, filed Jan. 26 2024, the contents of which are herewith incorporated by reference in their entirety.TECHNICAL FIELD

[0002] Examples of embodiments herein relate generally to wireless communications and, more specifically, relate to positioning in wireless networks, where the positioning uses AI / ML models.BACKGROUND

[0003] Artificial intelligence (Al, often also referred to a machine learning, ML, or even AI / ML) is being used for many purposes in wireless networks such as cellular networks. One such purpose is for positioning of a UE (user equipment, a wireless and typically mobile device), where the position of the UE within a cellular network is determined. Another possibility is to use AL / ML for determination of intermediate feature(s), such as a line of sight (LoS, a direct path between transmitter and receiver) or non-line-of-sight (NLoS, which is a path between transmitter and receiver with obstacles in between). The NLoS is also typically characterized by scattering of multiple surfaces and indirect paths. The intermediate feature(s) can be used for assisted positioning, e.g., an intermediate feature is a feature that will subsequently be used to determine a position of a UE.

[0004] As is known, one case of AI / ML is based on supervised learning, which uses a model that is trained using labelled data. These labelled data can correspond to ML inputs (e.g., radio measurements) and the corresponding ML output (e.g., a UE position or an intermediate feature). Thus, the model is trained on data that are known to be true.

[0005] The term “AI / ML” may include software (run on a computer system) or a computer system that aims to perform or solve complex human tasks. A model is software (runon a computer system) defined by known inputs and outputs, usually used to recognize patterns in specific data.

[0006] Once trained, the model is then used for inference. For instance, a model for positioning can be used to determine a UE’s position within the cellular network.

[0007] There is a lot of research in this area, one part of which concerns monitoring the performance of AI / ML models.BRIEF SUMMARY

[0008] This section is intended to include examples and is not intended to be limiting.

[0009] In an exemplary embodiment, a method is disclosed that includes extracting, at a user equipment having a trained model used for positioning of the user equipment within a cellular network, characterization for data that is input to the trained model for machine-learning inference to form an inference data characterization; comparing, by the user equipment, the inference data characterization with training input data characterization from a characterization of training input data that were used to train a model that resulted in the trained model; computing, by the user equipment, a metric for the trained model based on the comparison of the training input data characterization and the inference data characterization; and sending indication of the metric from the user equipment toward a location management function in the cellular network.

[0010] An additional exemplary embodiment includes a computer program, comprising instructions for performing the method of the previous paragraph, when the computer program is run on an apparatus. The computer program according to this paragraph, wherein the computer program is a computer program product comprising a computer-readable medium bearing the instructions embodied therein for use with the apparatus. Another example is the computer program according to this paragraph, wherein the program is directly loadable into an internal memory of the apparatus.

[0011] An exemplary apparatus includes one or more processors and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform: extracting, at a user equipment having a trained model used for positioning of the user equipment within a cellular network, characterization for data that is inputto the trained model for machine-learning inference to form an inference data characterization; comparing, by the user equipment, the inference data characterization with training input data characterization from a characterization of training input data that were used to train a model that resulted in the trained model; computing, by the user equipment, a metric for the trained model based on the comparison of the training input data characterization and the inference data characterization; and sending indication of the metric from the user equipment toward a location management function in the cellular network.

[0012] An exemplary computer program product includes a computer-readable storage medium bearing instructions that, when executed by an apparatus, cause the apparatus to perform at least the following: extracting, at a user equipment having a trained model used for positioning of the user equipment within a cellular network, characterization for data that is input to the trained model for machine-learning inference to form an inference data characterization; comparing, by the user equipment, the inference data characterization with training input data characterization from a characterization of training input data that were used to train a model that resulted in the trained model; computing, by the user equipment, a metric for the trained model based on the comparison of the training input data characterization and the inference data characterization; and sending indication of the metric from the user equipment toward a location management function in the cellular network.

[0013] In another exemplary embodiment, an apparatus comprises means for performing: extracting, at a user equipment having a trained model used for positioning of the user equipment within a cellular network, characterization for data that is input to the trained model for machine-learning inference to form an inference data characterization; comparing, by the user equipment, the inference data characterization with training input data characterization from a characterization of training input data that were used to train a model that resulted in the trained model; computing, by the user equipment, a metric for the trained model based on the comparison of the training input data characterization and the inference data characterization; and sending indication of the metric from the user equipment toward a location management function in the cellular network.

[0014] In an exemplary embodiment, a method is disclosed that includes receiving, by a location management function in a cellular network from a user equipment, indication of ametric, the metric corresponding to a trained model used by the user equipment for positioning of the user equipment within the cellular network and based on a comparison of training input data characterization and inference data characterization, the training input data characterization from a characterization of training input data that were used to train the model, and the inference data characterization from a characterization for data that is input to the trained model for machinelearning inference; and determining one or more actions to perform based on at least the metric.

[0015] An additional exemplary embodiment includes a computer program, comprising instructions for performing the method of the previous paragraph, when the computer program is run on an apparatus. The computer program according to this paragraph, wherein the computer program is a computer program product comprising a computer-readable medium bearing the instructions embodied therein for use with the apparatus. Another example is the computer program according to this paragraph, wherein the program is directly loadable into an internal memory of the apparatus.

[0016] An exemplary apparatus includes one or more processors and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform: receiving, by a location management function in a cellular network from a user equipment, indication of a metric, the metric corresponding to a trained model used by the user equipment for positioning of the user equipment within the cellular network and based on a comparison of training input data characterization and inference data characterization, the training input data characterization from a characterization of training input data that were used to train the model, and the inference data characterization from a characterization for data that is input to the trained model for machine-learning inference; and determining one or more actions to perform based on at least the metric.

[0017] An exemplary computer program product includes a computer-readable storage medium bearing instructions that, when executed by an apparatus, cause the apparatus to perform at least the following: receiving, by a location management function in a cellular network from a user equipment, indication of a metric, the metric corresponding to a trained model used by the user equipment for positioning of the user equipment within the cellular network and based on a comparison of training input data characterization and inference data characterization, the training input data characterization from a characterization of training inputdata that were used to train the model, and the inference data characterization from a characterization for data that is input to the trained model for machine-learning inference; and determining one or more actions to perform based on at least the metric.

[0018] In another exemplary embodiment, an apparatus comprises means for performing: receiving, by a location management function in a cellular network from a user equipment, indication of a metric, the metric corresponding to a trained model used by the user equipment for positioning of the user equipment within the cellular network and based on a comparison of training input data characterization and inference data characterization, the training input data characterization from a characterization of training input data that were used to train the model, and the inference data characterization from a characterization for data that is input to the trained model for machine-learning inference; and determining one or more actions to perform based on at least the metric.BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In the attached drawings:

[0020] FIG. 1 is a block diagram of one example of a proposed passive monitoring scheme for ML based positioning;

[0021] FIG. 2 is a signaling diagram illustrating an example of a signaling enhancement for an LMF-based training case;

[0022] FIG. 3 is a signaling diagram illustrating an example of a signaling enhancement for UE based training case;

[0023] FIG. 4 illustrates an example of PCA with 2 components;

[0024] FIG. 5 illustrates a graph of cumulative explainable variance for PCA;

[0025] FIG. 6 illustrates a plot of data representation with PCA 2 components;

[0026] FIG. 7 illustrates a plot of data representation with PCA 2 components;

[0027] FIG. 8 is a signaling diagram of an example of signaling for the PCA example and training at the LMF side; and

[0028] FIG. 9 is a block diagram of one possible and non-limiting exemplary system in which the exemplary embodiments may be practiced.DETAILED DESCRIPTION OF THE DRAWINGS

[0029] Abbreviations that may be found in the specification and / or the drawing figures are defined below, at the end of the detailed description section.

[0030] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments. All of the embodiments described in this Detailed Description are exemplary embodiments provided to enable persons skilled in the art to make or use the invention and not to limit the scope of the invention which is defined by the claims.

[0031] When more than one drawing reference numeral, word, or acronym is used within this description withand in general as used within this description, the “ / ” may be interpreted as “or”, “and”, or “both”. As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or,” mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

[0032] As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises”, “comprising”, “has”, “having”, “includes” and / or “including”, when used herein, specify the presence of stated features, elements, and / or components etc., but do not preclude the presence or addition of one or more other features, elements, components and / or combinations thereof.

[0033] Any flow diagram or signaling diagram (such as FIGS. 2, 3 and 8) herein is considered to be a logic flow diagram, and illustrates the operation of an exemplary method, results of execution of computer program instructions embodied on a computer readable memory, functions performed by logic implemented in hardware, and / or interconnected means for performing functions in accordance with an exemplary embodiment. Block diagrams (such as FIGS. 1 and 9) also illustrate the operation of an exemplary method, results of execution of computer program instructions embodied on a computer readable memory, functions performed by logic implemented in hardware, and / or interconnected means for performing functions inaccordance with an exemplary embodiment. For methods, flow diagrams, and signaling diagrams, the orders of method steps, blocks in the flow, or signaling are not critical and instead are examples.

[0034] It is noted that capital and lowercase words or phrases are considered to be the same herein. For instance, the words Slice and slice are the same, as are the phrases Network Repository Function and network repository function.

[0035] The examples herein related to certain technical areas, which are described now.

[0036] This section describes motivation for the examples presented herein. In the recent Release 18, a new study item has been approved to explore the benefits of augmenting the air-interface with AI / ML (also, AIML). See Intel Corporation, CATT, “Revised WID on NR Positioning Enhancements”, RP-210897, 3GPP TSGRAN Meeting #91e, Electronic Meeting, March 16 - 26, 2022. One of the use cases highlighted is positioning accuracy enhancements considering AI / ML methodology. For instance, AI / ML based methodology could be used to drive LOS / NLOS classifications in order to achieve higher positioning accuracy. The scope of the SID is not limited to only the LOS / NLOS classification for positioning accuracy and can include any validation of ML model for any positioning measurements. Furthermore, the SI will assess potential specification impact to support AI-ML with different level of collaborations between UE and network.

[0037] Some initial sets of use cases and their impacts are as follows: 1) Positioning accuracy enhancements for different scenarios including, e.g., those with heavy NLOS conditions; 2) Finalize representative sub use cases for each use case for characterization and baseline performance evaluations by RAN#98, where the AI / ML approaches for the selected sub use cases need to be diverse enough to support various requirements on the gNB-UE collaboration levels; 3) Consider AI / ML model, terminology and description to identify common and specific characteristics for framework investigations, including (a) characterize the defining stages of AI / ML related algorithms and associated complexity: Model generation, e.g., model training (including input / output, pre- / post-process, online / offline as applicable), model validation, model testing, as applicable, and (b) identify various levels of collaboration betweenUE and gNB pertinent to the selected use cases, e.g., various levels of UE / gNB collaboration targeting at separate or joint ML operation.

[0038] A pre-requirement in AI / ML supervised learning is that testing and validation data need to be labelled beforehand, which sounds obvious. Data labelling, however, is not for free, as it typically requires external devices to support an in-field measurement. The positioning reference unit (PRU), which was introduced and discussed in 3GPP Release 17 in RANI #105e, is intrinsically suitable to accommodate real-world measurement and labelling for AIML-based learning.

[0039] Currently there are five use-cases under discussion as part of the Rel-18 study item on AIML for air Interface. The agreement from RANI-111-bis-e is as follows:

[0040] Study and provide inputs on benefit(s) and potential specification impact at least for the following cases of AI / ML based positioning accuracy enhancement.

[0041] 1) Case 1 : UE-based positioning with UE-side model, direct AI / ML or AI / ML assisted positioning.

[0042] 2) Case 2a: UE-assisted / LMF-based positioning with UE-side model, AI / ML assisted positioning.

[0043] 3) Case 2b: UE-assisted / LMF-based positioning with LMF-side model, directAI / ML positioning.

[0044] 4) Case 3a: NG-RAN node assisted positioning with gNB-side model, AI / ML assisted positioning.

[0045] 5) Case 3b: NG-RAN node assisted positioning with LMF-side model, directAI / ML positioning.

[0046] Another agreement (from RAN1#112) regards AI / ML model monitoring for AI / ML based positioning, to study and provide inputs on benefit(s), feasibility, necessity and potential specification impact for the following aspects:

[0047] 1) Entity to derive monitoring metric.

[0048] 2) If model monitoring does not require ground truth label (or its approximation).

[0049] a) Monitoring metric, e.g., statistics of measurement, relative displacement, inference output inconsistency, or the like.

[0050] b) Assistance signaling and procedure, e.g., RS configuration(s) for measurement, measurement statistics as compared to the model input statistics of the training data, or the like.

[0051] c) Report of the calculated metric and / or model monitoring decision.

[0052] 2) If model monitoring requires and is provided ground truth label (or its approximation):

[0053] a) Monitoring metric, e.g., statistics of the difference between model output and ground truth label, or the like.

[0054] b) Assistance signaling and procedure, e.g., from LMF to UE / gNB indicating ground truth label and / or measurement, or the like.

[0055] A further agreement from RANI -113 regards ground truth label generation for AI / ML based positioning, the following options of entity to generate ground truth label are identified when beneficial and necessary (e.g., limited PRU availability): UE with estimated / known location generates ground truth label and corresponding label quality indicator; and Network entity generates ground truth label and corresponding label quality indicator.

[0056] A further agreement describes model monitoring without ground truth label:

[0057] 1) Monitoring metric:

[0058] a) FFS: statistics of measurement(s) compared to the statistics associated with the training data, statistics associated with the model output.

[0059] b) FFS details of statistics.

[0060] c) FFS details of what type of measurement(s).

[0061] 2) For monitoring UE-side and gNB-side model:

[0062] a) signaling from LMF to facilitate the monitoring entity to derive the monitoring metric (if needed).

[0063] b) signaling from monitoring entity to request measurement(s) (if needed).

[0064] c) signaling for potential request / report of monitoring metric (if needed).

[0065] d) Note: there may not be any specification impact.

[0066] Now that some motivation has been described for the examples herein, certain problems in these technical areas are described.

[0067] In ML-based positioning, a supervised learning-based approach (such as a neural network) is used in order to estimate either the UE position (referred to as direct AI / ML in Case 1, Case 2b, Case 3b) or intermediate features (such as a LoS / NLoS indicator for Case 1, Case 2a, Case 3a). The training of the supervised model requires labelled datasets corresponding to ML inputs (e.g., radio measurements) and the corresponding ML output (UE position or Intermediate feature).

[0068] Thereafter, this trained model is used to infer the UE position or intermediate features as requested by the LMF (or gNB or UE depending on the case). In the absence of ground truth data, it is difficult task to monitor the performance of the ML model which may degrade for any reason (such as change in the environment, unseen situation, and the like). As is known, ground truth data is the real output that the user aims to model using supervised learning methodologies.

[0069] Thus, one problem addressed herein is the following: How to monitor the ML model used for AI / ML positioning in the absence of ground truth data?

[0070] To address this and other problems, examples herein include methods and related signaling enhancements which target passive monitoring ML models, and which are valid for both direct and assisted AI / ML positioning. One example of an advantage of this approach is that it does not require ground truth labels, as the approach focuses on the model inputs rather than its outputs.

[0071] An overview is provided now. Turning to FIG. 1, this figure is a block diagram of one example of a proposed passive monitoring scheme for ML based positioning. Further, this figure depicts an overview of the different operations and corresponding blocks involved in the AI / ML based positioning:

[0072] Block 1. ML training: This refers to the training of the selected supervised model (typically a neural network-based architecture) based on a training dataset composed of selected inputs (radio measurements such as CIR, ToA and RSRP) and corresponding output (either intermediate feature for assisted positioning or geographical position for direct positioning). The input to block 1 is a training dataset 105 (e.g., inputs and corresponding outputs) and the output is a trained model 115. It is noted that, as a convention used herein, training input data is part of the training data in training dataset 105, such that the training datarefers to both training input data and training output data. In more detail, labelled data is the same as training samples (see block 5 of FIG. 8), and training data in the training dataset 105 is composed of training input data (e.g., radio measurements which would be the input of the ML model used in block 2) and training output data (e.g., UE position which is the expected output of the ML model also called in ML as labels).

[0073] Block 2. ML inference: This corresponds to the use of the previously trained model to infer / estimate the expected model output (either intermediate feature or UE position) based on provided model inputs (e.g., mainly radio measurements). Block 2 relies on model inputs 120 and produces model outputs 125.

[0074] Blocks 3 and 4. A proposed monitoring procedure 140 occurs in these blocks. That is, blocks 3 and 4 in FIG. 1 show possible different operations for a proposed passive monitoring. In fact, the training data used in the training dataset 105 for the model training is analyzed (block 3), in particular the input data (e.g., radio measurements as model inputs 120) are processed in order to extract their inner characterization through statistical analysis. Different approaches can be considered as for this analysis (exemplary implementations are provided below). This characterization of the training input data (via training dataset 105) along with the model inputs 120 (used for ML inference) are used for a self-monitoring operation in block 4 to derive monitoring-related metric(s), referred to as monitoring metric(s) 135, which may be any indicator of performance for a trained model. Examples of the monitoring metric 135 are described below, and the monitoring metric may be simply referred to as a metric. One reasoning behind this approach is to check whether the model inputs 120 are similar enough (in a statistical manner) to the used training dataset 105 to reach the expected model accuracy. In fact, a big deviation from the training data has high probability to induce degraded performance during ML inference. This is generally referred to as the process of monitoring ML models based on drift.

[0075] For clarity purposes, the monitoring procedure is also mapped to the signaling diagrams. That is, the two blocks of analysis and self-monitoring are marked in the signaling diagrams described below.

[0076] Detailed description is now provided of the proposed examples including: the signaling enhancement used to enable the proposed monitoring; and implementation details as for the analysis and self-monitoring depicted in FIG. 1.

[0077] Signaling enhancements are described now. To enable the proposed monitoring, new signaling is used. Hereafter, two embodiments are proposed for the cases where the training is realized (1) at LMF side and (2) at UE side.

[0078] FIG. 2 is a signaling diagram illustrating an example of a signaling enhancement for an LMF-based training case. This figure has three entities involved in the signaling: UE 210; LMF 99-1; and PRU 220. The LMF 99-1 is a function in a 5GCN (5G core network). The LMF 99-1 manages the support of different location services for target UEs, including positioning of UEs and delivery of assistance data to UEs. The LMF may interact with the serving base station for a target UE in order to obtain position measurements for the UE, including measurements used as input to a positioning model that are generally based on PRSs (positioning reference signals). For more information, see 3GPP TS 38.305, §5.4.4. The PRU 220 is a device that, at a known location, can perform positioning measurements and report these measurements to a location server. In addition, the PRU can transmit SRS to enable TRPs to measure and report UL positioning measurements from PRU at a known location. The PRU measurements can be compared by a location server with the measurements expected at the known PRU location to determine correction terms for other nearby target devices. From a location server perspective, the PRU functionality may be realized by a UE with known location. For more information, see 3GPP TS 38.305, §5.4.5. It is noted that the PRU measurements herein would also be used in combination with the known PRU position as ground truth (to constitute the training dataset).

[0079] FIG. 2 depicts the signaling for the case where the training is realized at an LMF 99-1. The different steps are as follows.

[0080] The operations in signaling 1 up to 4 are conventional, which includes collecting labelled data from a PRU 220 and training the selected ML model based on these data. This occurs using signaling 1, which requests from the LMF 99-1 to the PRU 220 labelled data (e.g., radio measurements+ position), a response in signaling 2 sending labelled data from the PRU 220 to the LMF 99-1, and training the model in block 3. It is noted that, for the position, ToA is part of radio measurements and, per definition, the PRU position is also known. Thereafter, the trained model is shared in signaling 4 with UEs (via the signaling from the LMF99-1 to the UE(s) 210) so that inference can be run (either to estimate the intermediate feature or the UE position).

[0081] The operations in block 5 can be realized before or after the operations in block 3 (model training) are performed. Block 5 involves the LMF 99-1 analyzing training data inputs and extracting characterization. This example shows block 5 being performed after block 4, however. One goal of block 5 is to analyze the input data within the collected training dataset (agreed to be used for the model training) and extract a characterization of the input data (e.g., in statistical manner through a projection in a lower dimension, e.g., with PC A or statistical distribution). Additionally, the LMF 99-1 should also set up the monitoring configuration, which includes a definition of the monitoring metric 135 (e.g., a distance metric between the characterized input training data and the input data to be used for inference) and eventually a criterion based on the computed metric to indicate possible degraded performance (e.g., with a threshold value). See block 230.

[0082] The signaling 6 is a new signaling sent from the LMF 99-1 towards the UE 210 to request performing self-monitoring based on the communicated training input dataset characterization (realized by the LMF 99-1) and the input data available at UE 210 (used for the inference). Note that this proposed monitoring is called passive self-monitoring, as the monitoring procedure relies only on the input data (e.g., radio measurements) without requiring ground truth information and the procedure acts to detect whenever there is suspicion of performance degradation of the model (e.g., due to a gap between data used for the training and data used for the inference).

[0083] Block 5 and the signaling in 6 are part of the monitoring procedure 140, and particularly the analysis procedure. The characterization of the input data that is extracted through a dimensionality reduction method e.g., PCA is used below, where a few PCs (e.g., 2D) are used in the characterization. The characterization is not limited to PCA, however, and statistical analyses could be used, such as a probability distribution.

[0084] In block 7, the UE 210 can use the trained model to perform the model inference inferencing (e.g., estimating either intermediate feature(s) or directly the geographical position).

[0085] Blocks 8 and 9 and the signaling in 10 form the rest of the monitoring procedure 140, in particular the self-monitoring. In block 8, the data used as input to realize the inference (performed in block 7) goes through an extraction of its characterization following the indications sent by LMF 99-1 (e.g., projection to extract 2D PCA components). This operation can not only apply for at least one sample data but also apply for a pre-selected number of samples (for more robust and confident decisions).

[0086] In block 9, the monitoring metric 135 is computed based on the characterization of the input data (where the characterization was performed in block 8), which is compared to the characterization of the input training dataset following LMF configurations. The computation of the monitoring metric can use the drift distance described below or other metrics.

[0087] In signaling 10, the UE 210 may report indication of the computed monitoring metric 135 if the reporting condition (indicated by the LMF 99-1 and which could be performed as a comparison to a threshold) is verified.

[0088] Finally in block 11, the LMF 99-1 can aggregate all the received monitoring metrics 135 (e.g., for a given period, and possibly from multiple UEs running the same trained model) and decide actions accordingly (e.g., getting ground truth data and checking the model performance and, in case of confirming model degradation trigger a refinement / retraining).

[0089] Some examples of advantages and technical effects of the proposed passive self-monitoring include one or more of the following:

[0090] 1) The passive self-monitoring relies only on the input model information therefore does not require ground truth data; and / or

[0091] 2) The passive self-monitoring can be used to alleviate the collection of ground truth data by first realizing a first level monitoring on the input model data (e.g., radio measurements) and when evaluating a suspicion on the model degradation, then in response to the evaluation trigger ground truth collection (which could be costly) for more in-depth verification of the model performance.

[0092] Turning to FIG. 3, this figure is a signaling diagram illustrating an example of a signaling enhancement for UE based training case. This figure shows the signaling in the case the training of the model is realized at UE side. In this embodiment, the LMF should indicate tothe UE the monitoring configuration including how the UE should characterize the input training data (see signaling 7) and then compare the data used for inference to the one used for the training (see block 9).

[0093] Signaling 1 and 2 in FIG. 3 are the same as in FIG. 2. The LMF 99-1 delivers in signaling 3 labeled data to the UE 210. The labelled data is assumed to include the training dataset 105 (see FIG. 1), which includes both training input data and training output data. The UE 210 trains a model in block 4 using received and (&) collected labelled data, and performs model inference using the trained model in block 5. Meanwhile, the LMF 99-1 selects a monitoring configuration and (&) related parameters in block 6 and sends signaling 7 to request self-monitoring and corresponding configuration. Block 230 is an example. In this example, the monitoring procedure 140 is performed by the blocks 8 and 9 and the signaling in 10. The UE 210 extracts training input data characterization following LMF configuration in block 8, and in block 9 computes a monitoring metric 135 by comparing input data characterization (from block 8) to determined training data characterization. Block 9a is shown where training data characterization is extracted by the UE 210, but such extraction could be performed earlier, e.g., any time after signaling 3. The computation of the monitoring metric 135 can use the drift distance described below or other metrics. In response to a condition (e.g., the criterion from block 230 and received in signaling 7) being verified, the UE in signaling 10 sends indication of the monitoring metric 135 to the LMF 99-1. The LMF 99-1 aggregates in block 11 received monitoring metric(s) 135 and decides actions accordingly.

[0094] Implementation details are now described. In this part, example implementation details are described regarding the characterization of the input data. As described previously one proposal is based on Principal Component Analysis (PC A). In fact, PCA (Principal Component Analysis) is a data analysis tool conventionally used to reduce the dimensionality (number of variables) of a large number of interrelated variables, while retaining as much of the information (variation) as possible. PCA calculates an uncorrelated set of variables (components or PCs). These components are ordered so that the first few retain most of the variation present in all of the original variables. Thus, PCA will convert a set of possibly correlated measurements into a set of linearly uncorrelated variables with Y = W'X . The weightsmatrix I / Fmay be determined based on historical data using, e.g., an eigenvalues decomposition method. It is noted that Wris the transposer of W and is used to match dimensions.

[0095] The PCA transforms a data set of dimension N into a set of components of size K where each component ykcan be formulated as a weighted sum of the N original data x:

[0096] yk= ^=1wjik Xj.

[0097] For example, considering the case of PCA with two components applied to a data set of size 18, each component is expressed as:

[0098] yq = w1 1x1+ w2 1x2+ ■■■ . . +w18 1x18,- and

[0099] y2= w1>2x1+ wix + ••• . . +w18>2x18.

[0100] There are 18 dimensions in the above equations, and the dimensions can relate to TRPs. The weights have values, and each X corresponds to the measurement of TRP i.

[0101] FIG. 4 provides a representation of the two components (PCI and PC2)). The input vector considered for illustration uses ToAs from 18 TRPs. Alternatively, one can use stacked CIR / PDP / DP from all 18 TRPs as the input vector and obtain PCA decomposition for model monitoring. A TRP is a transmission-reception point, such as a base station or a radio unit (where the radio unit connects through a distributed unit to a central unit) or the like. Note that the example is obtained with PCA function from sklearn (scikit-learn is also known as sklearn and is a free software machine-learning library for the Python programming language). It is assumed that higher values for weights correspond to TRPs that are closer and lower values for weights correspond to TRPs that are farther away.

[0102] In PCA, explainable variance refers to the amount of variance in the original dataset that is captured or explained by each principal component. When performing PCA, the principal components are ordered by the amount of variance they account for. The first principal component captures the highest amount of variance present in the data, and each subsequent component captures less variance in descending order. The “explainable variance” of a principal component refers to the proportion of the total variance in the original data that this specific principal component accounts for. FIG. 5, which is a graph of cumulative explainable variance for PCA, shows the cumulative explainable variance for the example that was considered (ToA from 18 TRPs, which are represented by the 0-17 dimensions in FIGS. 4 and 4A). In thisexample: PCI contains 80% of the variance and PC2 contains 8%: Reference 510 indicates the cumulative explained variance for the two components (PC1,PC2). This means that these two components gather 88% of the information in the original data set (whereas four components case achieve 99% variance). For this reason, use of the two dominant PCA components is selected for the input data characterization in one example, though other numbers of dominant PCA components may also be used, as can other techniques.

[0103] FIG. 6 illustrates a plot of data representation with PCA 2 components, shows the different measurement samples converted to two-dimensional space (PC1,PC2). An initial set (used for training) is shown being darker and a test set (corresponding to different measurements at different positions) is shown being lighter. The initial set can be thought of as being “behind” the test set, and the test set as “overlaying” the initial set. The data shown in FIG. 6 corresponds here to the characterization of an input training data set. In this example, considering the same context / environment, the data is split into initial set used for training and test set (corresponding to different positions). As seen in the figure, when extracting the PCA components, both training and test coincide extremely well in 2D.

[0104] Now considering data from a different context with different NLoS conditions, ToA measurements are translated / pr ejected into PCA components (with PCA trained using the initial set in FIG. 5). FIG. 7, which illustrates a plot of data representation with PCA of two components, shows the two PCA components for both the initial set and the data from the different environment (the other set, which is lighter). The other set in this example is a single point. One can clearly see the gap between both. In this context, it is proposed to calculate the monitoring metric 135 as the drift distance 710 separating the ‘other’ data from the initial set characterized through PCA components.

[0105] It is noted that if additional points are used, metrics other than the drift distance 710 can be used. For instance, one could use a minimum value, mean value, or CDF (cumulative distribution function) value. That is, a metric may be a value of the cumulative distribution function on the distance calculated between the input training data characterization and inference input data characterization such as CDF value at 90%. It is also noted the FIG. 6 has high accuracy relative to FIG. 7, where the accuracy is much poorer.

[0106] For completeness, the signaling (FIG. 8) is provided considering the case where the characterization of the input data is performed with PCA (training in block 5 and input data for inference in block 8). An example is considered where the training is realized at the LMF (as also in FIG. 2). FIG. 8 is a signaling diagram of an example of signaling for the PCA example and training at the LMF side. The first four elements in FIG. 8 are the same as in FIG. 2. In this example, and similar to FIG. 2, the monitoring procedure 140 has an analysis part that is performed by block 5 and signaling 6, and the self-monitoring part of the monitoring procedure is performed by blocks 8 and 9 and signaling 10.

[0107] In FIG. 8, block 5, the LMF 99-1 decides based on the calculated explainable variance to use two dimensions, as two dominant PCA components capture most of the input data information. In this case, N PCA is equal to 2. The number of TRPs is N TRP, which is the original samples dimensions, and N_PC is the number of principal components to consider and which is referred as N PCA. The LMF determines the number of dominant PCE components (where N PCA <= N TRP) from the training samples.

[0108] In signaling 6, the LMF requests UE to perform self-monitoring. The LMF provides then to the UE PCA samples (for two dominant PCA components) of the training input data (which corresponds to the darker samples in the example of FIG. 7). In addition, the LMF 99-1 should indicate to the UE the monitoring rules which correspond to the rules for the drift calculation (see example in FIG. 7 of the drift distance 710, which could be described by a monitoring rule, which could also be used for describing other metric calculations) as well as a threshold value (such that if the drift distance is higher than the threshold, the UE should inform the LMF; otherwise no need to share this information). See block 820 for possible monitoring configuration.

[0109] Signaling 6 also illustrates signaling of a trained PCA function (e.g., pca.transformQ, described now). In more detail, the characterization includes also the learnt PCA obtained through a function pca.fit(d training), which uses training samples. After performing characterization on the inference data, one may apply the learnt PCA as a function pca.transform(d test), which uses test samples (e.g., model inputs). Typically, the PCA function pea. fit will allow one to calculate the weights W, which are then applied on the inference data in order to determine PCI and PC2 and compare to the training data. The LMF 99-1 performspca.fitQ and then sends selected PCA values and the trained PC A function, and the UE 210 can perform pca.transformQ on the inference data.

[0110] In block 7, the UE performs inference with the received trained model.

[0111] In block 8, the UE 210 should calculate the PCA components using the PCA function (trained and estimated by the LMF 99-1). This would correspond to the single point in the example of FIG. 7. That is, the UE calculates N PCA components of input data with a PCA function. In this example, the monitoring metric 135 is the drift distance calculated between the PCA. Therefore, in block 9, the drift distance is calculated, based on the provided rules, between the PCA (training data) and the PCA (inference input data).

[0112] The signaling 10 and the block 11 in FIG. 8 are the same as in FIG. 2. In the example of FIG. 8, the monitoring metric that is sent is the calculated drift distance(or possibly the output of another metric).

[0113] The drift distance 710 is merely one metric that may be used, and other metrics are possible (as in block 830). As described previously, other metrics such as a minimum value, mean value, or CDF (cumulative distribution function), as examples, may be used. The corresponding monitoring rules and threshold values in block 820 would be tailored to the corresponding metric. It is noted that FIG. 8 shows an example similar to FIG. 2. The example of using PCA may also be applied to the example in FIG. 3.

[0114] Turning to FIG. 9, this figure shows a block diagram of one possible and nonlimiting example of a cellular network 1 that is connected to a user equipment (UE) 10. A number of network elements are shown in the cellular network of FIG. 9: a base station 70; and a core network 90. The UE 10 in this example could be a PRU 220 or the UE 210.

[0115] In FIG. 9, a user equipment (UE) 10 is in wireless communication via radio link 11 with the base station 70 of the cellular network 1. A UE 10 is a wireless communication device, such as a mobile device, that is configured to access a cellular network. The UE 10 is illustrated with one or more antennas 28. The ellipses 2 indicate there could be multiple UEs 10 in wireless communication via radio links with the base station 70. The UE 10 includes one or more processors 13, one or more memories 15, and other circuitry 16. The other circuitry 16 includes one or more receivers (Rx(s)) 17 and one or more transmitters (Tx(s)) 18. A program 12is used to cause the UE 10 to perform the operations described herein. For a UE 10, the other circuitry 16 could include circuitry such as for user interface elements (not shown) like a display.

[0116] The base station 70, as a network element of the cellular network 1, provides the UE 10 access to cellular network 1 and to the data network 91 via the core network 90 (e.g., via a user plane function (UPF) of the core network 90). As such, the base station 70 may be considered to be an access node, which provides access by UE(s) 10 to the cellular network 1. The base station 70 is illustrated as having one or more antennas 58. In general, the base station 70 may be referred to as RAN node 70, although many will make reference to this as a gNB (gNode B, a base station for NR, new radio) instead. There are, however, many other examples of RAN nodes including an eNB (evolved Node B) or TRP (Transmission-Reception Point). The term TRP is used mainly herein, and there are multiple options for this, such as a single TRP (of a base station), or distributed unit or radio unit, where multiple such units may be coupled to a central unit. As used herein, a TRP is any device that can transmit and receive signals for positioning.

[0117] The base station 70 (or an individual TRP) includes one or more processors 73, one or more memories 75, and other circuitry 76. The other circuitry 76 includes one or more receivers (Rx(s)) 77 and one or more transmitters (Tx(s)) 78. A program 72 is used to cause the base station 70 to perform the operations described herein.

[0118] It is noted that the base station 70 may instead be implemented via other wireless technologies, such as Wi-Fi (a wireless networking protocol that devices use to communicate without direct cable connections). In the case of Wi-Fi, the link 11 could be characterized as a wireless link.

[0119] Two or more base stations 70 communicate using, e.g., link(s) 79. The link(s) 79 may be wired or wireless or both and may implement, e.g., an Xn interface for 5G (fifth generation), an X2 interface for LTE (Long Term Evolution), or other suitable interface for other standards.

[0120] The cellular network 1 may include a core network 90, as a second network element or elements, that may include core network functionality, and which provide connectivity via a link or links 81 with a data network 91, such as a telephone network and / or a data communications network (e.g., the Internet). The core network 90 includes one or moreprocessors 93, one or more memories 95, and other circuitry 96. The other circuitry 96 includes one or more receivers (Rx(s)) 97 and one or more transmitters (Tx(s)) 98. A program 92 is used to cause the core network 90 to perform the operations described herein.

[0121] The core network 90 could be a 5GC (5G core network). The core network 90 can implement or comprise multiple network functions (NF(s)) 99, and the program 92 may comprise one or more of the NFs 99. A 5G core network may use hardware such as memory and processors and a virtualization layer. It could be a single standalone computing system, a distributed computing system, or a cloud computing system. The NFs 99, as network elements, of the core network could be containers or virtual machines running on the hardware of the computing system(s) making up the core network 90.

[0122] Core network functionality for 5G may include access and mobility management functionality that is provided by a network function 99 such as an access and mobility management function (AMF), session management functionality that is provided by a network function such as a session management function (SMF). Core network functionality for access and mobility management in an LTE (Long Term Evolution) network may be provided by an MME (Mobility Management Entity) and / or SGW (Serving Gateway) functionality, which routes data to the data network. Many others are possible, as illustrated by the examples in FIG. 9: AMF; SMF; MME; SGW; GMLC (Gateway Mobile Location Center); LMF (Location Management Function); UDM (Unified Data Management) / UDR (Unified Data Repository); NRF (Network Repository Function); and / or E-SMLC (Evolved Serving Mobile Location Center). These are merely exemplary core network functionality that may be provided by the core network 90, and note that both 5G and LTE core network functionality might be provided by the core network 90. The base station 70 is coupled via a backhaul link 31 to the core network 90. The base station 70 and the core network 90 may include an NG (Next Generation) interface for 5G, or an SI interface for LTE, or other suitable interface for other radio access technologies for communicating via the backhaul link 31.

[0123] In the data network 91, there is a computer-readable medium 94. The computer-readable medium 94 contains instructions that, when downloaded and installed into the memories 15, 75, or 95 of the corresponding UE 10, base station 70, and / or core network element(s) 90, and executed by processor(s) 13, 73, or 93, cause the respective device to perform1 corresponding actions described herein. The computer-readable medium 94 may be implemented in other forms, such as via a compact disc or memory stick.

[0124] The programs 12, 72, and 92 contain instructions stored by corresponding one or more memories 15, 75, or 95. These instructions, when executed by the corresponding one or more processors 13, 73, or 93, cause the corresponding apparatus 10, 70, or 90, to perform the operations described herein. The computer readable memories 15, 75, or 95 are circuitry and may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, flash memory, firmware, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The computer readable memories 15, 75, and 95 may be means for performing storage functions. The processors 13, 73, and 93, are circuitry and may be of any type suitable to the local technical environment. For example, these processors may include one or more of general-purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), processors based on a multi-core processor architecture, and may also include specialized circuits such as field-programmable gate arrays (FPGAs), application specific circuits (ASICs), signal processing devices and other devices, or combinations of these devices, as non-limiting examples. The processors 13, 73, and 93 may be means for causing their respective apparatus to perform functions, such as those described herein. Particularly, for any apparatus having means to perform functions described herein, the means may include at least one processor, and at least one memory storing instructions that, when executed by at least one processor, cause the performance of the apparatus.

[0125] The receivers 17, 77, and 97, and the transmitters 18, 78, and 98 may implement wired or wireless interfaces. The receivers and transmitters may be grouped together as transceivers.

[0126] The cellular network 1 may implement network virtualization, which is the process of combining hardware and software network resources and network functionality into a single, software-based administrative entity, a virtual network. Network virtualization involves platform virtualization, often combined with resource virtualization. Network virtualization is categorized as either external, combining many networks, or parts of networks, into a virtual unit, or internal, providing network-like functionality to software containers on a single system.Note that the virtualized entities (such as network functions 99) that result from the network virtualization are still implemented, at some level, using hardware such as processors 73 and / or 93 and memories 75 and / or 95, and also such virtualized entities create technical effects.

[0127] In general, the various embodiments of the user equipment 10 can include, but are not limited to, cellular telephones (such as smart phones, mobile phones, cellular phones, voice over Internet Protocol (IP) (VoIP) phones, and / or wireless local loop phones), tablets, portable computers, vehicles or vehicle-mounted devices for, e.g., wireless V2X (vehicle-to- everything) communication, image capture devices such as digital cameras, gaming devices, music storage and playback appliances, Internet appliances (including Internet of Things, loT, devices), loT devices with sensors and / or actuators for, e.g., automation applications, as well as portable units or terminals that incorporate combinations of such functions, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), Universal Serial Bus (USB) dongles, smart devices, wireless customer-premises equipment (CPE), an Internet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an industrial device and applications (e.g., a robot and / or other wireless devices operating in an industrial and / or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and / or industrial wireless networks, and the like. That is, the UE 10 could be any end device that may be capable of wireless communication. By way of example rather than limitation, the UE may also be referred to as a communication device, terminal device (MT), a Subscriber Station (SS), a Portable Subscriber Station, a Mobile Station (MS), or an Access Terminal (AT).

[0128] The following are additional examples.

[0129] Example 1. A method, comprising: extracting, at a user equipment having a trained model used for positioning of the user equipment within a cellular network, characterization for data that is input to the trained model for machine-learning inference to form an inference data characterization; comparing, by the user equipment, the inference data characterization with training input data characterization from a characterization of training input data that were used to train a model that resulted in the trained model; computing, by the user equipment, a metric for the trained model based on the comparison of the training input datacharacterization and the inference data characterization; and sending indication of the metric from the user equipment toward a location management function in the cellular network.

[0130] Example 2. The method according to example 1, wherein: the method further comprises receiving, by the user equipment from the location management function, a request to perform self-monitoring; and performing, by the user equipment, at least the extracting, comparing, and computing, in response to receiving the request.

[0131] Example s. The method according to example 2, further comprising receiving, by the user equipment, indication of configuration from the location management function, the configuration comprising a definition of the metric to be computed and monitored by performing at least the extracting, comparing, and computing, and a criterion based on the computed metric to indicate degraded performance of the trained model.

[0132] Example 4. The method according to example 3, wherein the sending the indication is performed because a condition based on the criterion is verified to be met.

[0133] Example 5. The method according to any one of examples 2 to 4, further comprising receiving, by the user equipment, indication of the training input data characterization from the location management function.

[0134] Example 6. The method according to any one of examples 2 to 4, further comprising: receiving, by the user equipment from the location management function, labelled data used for training the model and comprising the training input data; and performing, by the user equipment, characterization of the training input data to determine the training input data characterization.

[0135] Example 7. The method according to any one of examples 1 to 6, further comprising computing the metric based on the training input data characterization and the inference data characterization.

[0136] Example 8. The method according to example 2, wherein: the request to perform self-monitoring comprises N principal components that are received for principal component analysis, N >1 but less than or equal to a number of transmission-reception points used to determine the N principal components; the method further comprises receiving, by the user equipment, a trained function for principal component analysis and a configuration for monitoring the N principal components; the extracting the characterization for data that is inputto the trained model for inferencing comprises calculating the N principal components based on the trained function for principal component analysis; the comparing the inference data characterization with training input data characterization comprises comparing the calculated N principal components and the received N principal components; and the computing the metric comprises computing a metric based on the comparison of the calculated N principal components and the received N principal components.

[0137] Example 9. The method according to example 8, wherein the metric is one of a drift distance, minimum value, mean value, or a value of a cumulative distribution function on a distance calculated between the input training data characterization and inference input data characterization.

[0138] Example 10. The method according to any one of examples 8 or 9, wherein: the method further comprises receiving, by the user equipment from the location management function, one or more rules for performing the computing of the metric using the calculated N principal components and the received N principal components; and performing, by the user equipment based on the one or more rules, the computing of the metric using the calculated N principal components and the received N principal components.

[0139] Example 11. A method, comprising: receiving, by a location management function in a cellular network from a user equipment, indication of a metric, the metric corresponding to a trained model used by the user equipment for positioning of the user equipment within the cellular network and based on a comparison of training input data characterization and inference data characterization, the training input data characterization from a characterization of training input data that were used to train the model, and the inference data characterization from a characterization for data that is input to the trained model for machinelearning inference; and determining one or more actions to perform based on at least the metric.

[0140] Example 12. The method according to example 11, wherein: the method further comprises aggregating metrics received from multiple user equipment; and the determining the one or more actions to perform is based on the aggregated metrics.

[0141] Example 13. The method according to any one of examples 11 or 12, wherein: the method further comprises sending, by the location management function to the user equipment, a request to perform self-monitoring.

[0142] Example 14. The method according to example 13, further comprising sending by the location management function to the user equipment, indication of configuration comprising a definition of the metric to be computed, and a criterion based on the computed metric to indicate degraded performance of the trained model.

[0143] Example 15. The method according to any one of examples 13 or 14, further comprising: determining, by the location management function, characterization of the training input data to determine the training input data characterization; and sending, from the location management function to the user equipment, indication of the training input data characterization.

[0144] Example 16. The method according to any one of examples 13 or 14, further comprising: sending, by the location management function to the user equipment, labelled data that comprises training input data for training the model by the user equipment.

[0145] Example 17. The method according to example 13, wherein: the method further comprises the location management function determining a number N of principal components for principal component analysis, the number N of the principal components being less than or equal to a number of transmission-reception points used to determine the N principal components, from training samples from the data that is input to the trained model for machinelearning inference; the request to perform self-monitoring comprises indication of the determined N principal components for principal component analysis; and the method further comprises sending, by the location management function to the user equipment, a trained function for principal component analysis and a configuration for monitoring the N principal components using a metric.

[0146] Example 18. The method according to example 17, wherein the metric is one of a drift distance, minimum value, mean value, or a value of a cumulative distribution function on a distance calculated between the input training data characterization and inference input data characterization.

[0147] Example 19. The method according to any one of examples 17 or 18, wherein: the method further comprises sending, by the location management function to the user equipment, one or more rules for performing by the user equipment calculating of the metric using N principal components to be calculated by the user equipment and N principalcomponents sent by the location management function to the user equipment in the configuration for monitoring the N principal components.

[0148] Example 20. A computer program, comprising instructions for performing the methods of any of examples 1 to 19, when the computer program is run on an apparatus.

[0149] Example 21. The computer program according to example 20, wherein the computer program is a computer program product comprising a computer-readable medium bearing instructions embodied therein for use with the apparatus.

[0150] Example 22. The computer program according to example 20, wherein the computer program is directly loadable into an internal memory of the apparatus.

[0151] Example 23. An apparatus, comprising means for performing: extracting, at a user equipment having a trained model used for positioning of the user equipment within a cellular network, characterization for data that is input to the trained model for machine-learning inference to form an inference data characterization; comparing, by the user equipment, the inference data characterization with training input data characterization from a characterization of training input data that were used to train a model that resulted in the trained model; computing, by the user equipment, a metric for the trained model based on the comparison of the training input data characterization and the inference data characterization; and sending indication of the metric from the user equipment toward a location management function in the cellular network.

[0152] Example 24. The apparatus according to example 23, wherein: the means are further configured for performing: receiving, by the user equipment from the location management function, a request to perform self-monitoring; and performing, by the user equipment, at least the extracting, comparing, and computing, in response to receiving the request.

[0153] Example 25. The apparatus according to example 24, wherein the means are further configured for performing: receiving, by the user equipment, indication of configuration from the location management function, the configuration comprising a definition of the metric to be computed and monitored by performing at least the extracting, comparing, and computing, and a criterion based on the computed metric to indicate degraded performance of the trained model.

[0154] Example 26. The apparatus according to example 25, wherein the sending the indication is performed because a condition based on the criterion is verified to be met.

[0155] Example 27. The apparatus according to any one of examples 24 to 26, wherein the means are further configured for performing: receiving, by the user equipment, indication of the training input data characterization from the location management function.

[0156] Example 28. The apparatus according to any one of examples 24 to 26, wherein the means are further configured for performing: receiving, by the user equipment from the location management function, labelled data used for training the model and comprising the training input data; and performing, by the user equipment, characterization of the training input data to determine the training input data characterization.

[0157] Example 29. The apparatus according to any one of examples 23 to 28, wherein the means are further configured for performing: computing the metric based on the training input data characterization and the inference data characterization.

[0158] Example 30. The apparatus according to example 24, wherein: the request to perform self-monitoring comprises N principal components that are received for principal component analysis, N >1 but less than or equal to a number of transmission-reception points used to determine the N principal components; the means are further configured for performing: receiving, by the user equipment, a trained function for principal component analysis and a configuration for monitoring the N principal components; the extracting the characterization for data that is input to the trained model for inferencing comprises calculating the N principal components based on the trained function for principal component analysis; the comparing the inference data characterization with training input data characterization comprises comparing the calculated N principal components and the received N principal components; and the computing the metric comprises computing a metric based on the comparison of the calculated N principal components and the received N principal components.

[0159] Example 31. The apparatus according to example 30, wherein the metric is one of a drift distance, minimum value, mean value, or a value of a cumulative distribution function on a distance calculated between the input training data characterization and inference input data characterization.

[0160] Example 32. The apparatus according to any one of examples 30 or 30, wherein: the means are further configured for performing: receiving, by the user equipment from the location management function, one or more rules for performing the computing of the metric using the calculated N principal components and the received N principal components; and performing, by the user equipment based on the one or more rules, the computing of the metric using the calculated N principal components and the received N principal components.

[0161] Example 33. An apparatus, comprising means for performing: receiving, by a location management function in a cellular network from a user equipment, indication of a metric, the metric corresponding to a trained model used by the user equipment for positioning of the user equipment within the cellular network and based on a comparison of training input data characterization and inference data characterization, the training input data characterization from a characterization of training input data that were used to train the model, and the inference data characterization from a characterization for data that is input to the trained model for machinelearning inference; and determining one or more actions to perform based on at least the metric.

[0162] Example 34. The apparatus according to example 33, wherein: the means are further configured for performing: aggregating metrics received from multiple user equipment; and the determining the one or more actions to perform is based on the aggregated metrics.

[0163] Example 35. The apparatus according to any one of examples 33 or 34, wherein: the means are further configured for performing: sending, by the location management function to the user equipment, a request to perform self-monitoring.

[0164] Example 36. The apparatus according to example 35, wherein the means are further configured for performing: sending by the location management function to the user equipment, indication of configuration comprising a definition of the metric to be computed, and a criterion based on the computed metric to indicate degraded performance of the trained model.

[0165] Example 37. The apparatus according to any one of examples 35 or 36, wherein the means are further configured for performing: determining, by the location management function, characterization of the training input data to determine the training input data characterization; and sending, from the location management function to the user equipment, indication of the training input data characterization.

[0166] Example 38. The apparatus according to any one of examples 35 or 36, wherein the means are further configured for performing: sending, by the location management function to the user equipment, labelled data that comprises training input data for training the model by the user equipment.

[0167] Example 39. The apparatus according to example 35, wherein: the means are further configured for performing: the location management function determining a number N of principal components for principal component analysis, the number N of the principal components being less than or equal to a number of transmission-reception points used to determine the N principal components, from training samples from the data that is input to the trained model for machine-learning inference; the request to perform self-monitoring comprises indication of the determined N principal components for principal component analysis; and the means are further configured for performing: sending, by the location management function to the user equipment, a trained function for principal component analysis and a configuration for monitoring the N principal components using a metric.

[0168] Example 40. The apparatus according to example 39, wherein the metric is one of a drift distance, minimum value, mean value, or a value of a cumulative distribution function on a distance calculated between the input training data characterization and inference input data characterization.

[0169] Example 41. The apparatus according to any one of examples 39 or 40, wherein: the means are further configured for performing: sending, by the location management function to the user equipment, one or more rules for performing by the user equipment calculating of the metric using N principal components to be calculated by the user equipment and N principal components sent by the location management function to the user equipment in the configuration for monitoring the N principal components.

[0170] Example 42. The apparatus of any preceding apparatus example, wherein the means comprises: at least one processor; and at least one memory storing instructions that, when executed by at least one processor, cause the performance of the apparatus.

[0171] Example 43. An apparatus, comprising: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform: extracting, at a user equipment having a trained model used forpositioning of the user equipment within a cellular network, characterization for data that is input to the trained model for machine-learning inference to form an inference data characterization; comparing, by the user equipment, the inference data characterization with training input data characterization from a characterization of training input data that were used to train a model that resulted in the trained model; computing, by the user equipment, a metric for the trained model based on the comparison of the training input data characterization and the inference data characterization; and sending indication of the metric from the user equipment toward a location management function in the cellular network.

[0172] Example 44. The apparatus according to example 43, wherein: the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: receiving, by the user equipment from the location management function, a request to perform self-monitoring; and performing, by the user equipment, at least the extracting, comparing, and computing, in response to receiving the request.

[0173] Example 45. The apparatus according to example 44, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: receiving, by the user equipment, indication of configuration from the location management function, the configuration comprising a definition of the metric to be computed and monitored by performing at least the extracting, comparing, and computing, and a criterion based on the computed metric to indicate degraded performance of the trained model.

[0174] Example 46. The apparatus according to example 3, wherein the sending the indication is performed because a condition based on the criterion is verified to be met.

[0175] Example 47. The apparatus according to any one of examples 44 to 46, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: receiving, by the user equipment, indication of the training input data characterization from the location management function.

[0176] Example 48. The apparatus according to any one of examples 44 to 46, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: receiving, by the user equipment fromthe location management function, labelled data used for training the model and comprising the training input data; and performing, by the user equipment, characterization of the training input data to determine the training input data characterization.

[0177] Example 49. The apparatus according to any one of examples 43 to 48, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: computing the metric based on the training input data characterization and the inference data characterization.

[0178] Example 50. The apparatus according to example 44, wherein: the request to perform self-monitoring comprises N principal components that are received for principal component analysis, N >1 but less than or equal to a number of transmission-reception points used to determine the N principal components; the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: receiving, by the user equipment, a trained function for principal component analysis and a configuration for monitoring the N principal components; the extracting the characterization for data that is input to the trained model for inferencing comprises calculating the N principal components based on the trained function for principal component analysis; the comparing the inference data characterization with training input data characterization comprises comparing the calculated N principal components and the received N principal components; and the computing the metric comprises computing a metric based on the comparison of the calculated N principal components and the received N principal components.

[0179] Example 51. The apparatus according to example 50, wherein the metric is one of a drift distance, minimum value, mean value, or a value of a cumulative distribution function on a distance calculated between the input training data characterization and inference input data characterization.

[0180] Example 52. The apparatus according to any one of examples 50 or 51, wherein: the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: receiving, by the user equipment from the location management function, one or more rules for performing the computing of the metric using the calculated N principal components and the received N principal components; andperforming, by the user equipment based on the one or more rules, the computing of the metric using the calculated N principal components and the received N principal components.

[0181] Example 53. An apparatus, comprising: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform: receiving, by a location management function in a cellular network from a user equipment, indication of a metric, the metric corresponding to a trained model used by the user equipment for positioning of the user equipment within the cellular network and based on a comparison of training input data characterization and inference data characterization, the training input data characterization from a characterization of training input data that were used to train the model, and the inference data characterization from a characterization for data that is input to the trained model for machine-learning inference; and determining one or more actions to perform based on at least the metric.

[0182] Example 54. The apparatus according to example 53, wherein: the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: aggregating metrics received from multiple user equipment; and the determining the one or more actions to perform is based on the aggregated metrics.

[0183] Example 55. The apparatus according to any one of examples 53 or 54, wherein: the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: sending, by the location management function to the user equipment, a request to perform self-monitoring.

[0184] Example 56. The apparatus according to example 55, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: sending by the location management function to the user equipment, indication of configuration comprising a definition of the metric to be computed, and a criterion based on the computed metric to indicate degraded performance of the trained model.

[0185] Example 57. The apparatus according to any one of examples 55 or 56, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: determining, by the location management function, characterization of the training input data to determine the training inputdata characterization; and sending, from the location management function to the user equipment, indication of the training input data characterization.

[0186] Example 58. The apparatus according to any one of examples 55 or 56, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: sending, by the location management function to the user equipment, labelled data that comprises training input data for training the model by the user equipment.

[0187] Example 59. The apparatus according to example 55, wherein: the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: the location management function determining a number N of principal components for principal component analysis, the number N of the principal components being less than or equal to a number of transmission-reception points used to determine the N principal components, from training samples from the data that is input to the trained model for machine-learning inference; the request to perform self-monitoring comprises indication of the determined N principal components for principal component analysis; and the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: sending, by the location management function to the user equipment, a trained function for principal component analysis and a configuration for monitoring the N principal components using a metric.

[0188] Example 60. The apparatus according to example 59, wherein the metric is one of a drift distance, minimum value, mean value, or a value of a cumulative distribution function on a distance calculated between the input training data characterization and inference input data characterization.

[0189] Example 61. The apparatus according to any one of examples 59 or 60, wherein: the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: sending, by the location management function to the user equipment, one or more rules for performing by the user equipment calculating of the metric using N principal components to be calculated by the user equipment and N principal components sent by the location management function to the user equipment in the configuration for monitoring the N principal components.

[0190] As used in this application, the term “circuitry” may refer to one or more or all of the following:

[0191] (a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry) and

[0192] (b) combinations of hardware circuits and software, such as (as applicable): (i) a combination of analog and / or digital hardware circuit(s) with software / firmware and (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and

[0193] (c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0194] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0195] Embodiments herein may be implemented in software (executed by one or more processors), hardware (e.g., an application specific integrated circuit), or a combination of software and hardware. In an example embodiment, the software (e.g., application logic, an instruction set) is maintained on any one of various conventional computer-readable media. In the context of this document, a “computer-readable medium” may be any media or means that can contain, store, communicate, propagate or transport the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer, with one example of a computer described and depicted, e.g., in FIG. 9. A computer-readable medium may comprise a computer-readable storage medium (e.g., memories 15, 75, and 95 or other device) that may be any media or means that can contain, store, and / or transport theinstructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer. A computer-readable storage medium does not comprise propagating signals, and therefore may be considered to be non-transitory. The term “non-transitory”, as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM, random access memory, versus ROM, readonly memory).

[0196] If desired, the different functions discussed herein may be performed in a different order and / or concurrently with each other. Furthermore, if desired, one or more of the above-described functions may be optional or may be combined.

[0197] Although various aspects of the invention are set out in the independent claims, other aspects of the invention comprise other combinations of features from the described embodiments and / or the dependent claims with the features of the independent claims, and not solely the combinations explicitly set out in the claims.

[0198] It is also noted herein that while the above describes example embodiments of the invention, these descriptions should not be viewed in a limiting sense. Rather, there are several variations and modifications which may be made without departing from the scope of the present invention as defined in the appended claims.

[0199] The following abbreviations that may be found in the specification and / or the drawing figures are defined as follows:

[0200] 2D two dimensional

[0201] 5G fifth generation

[0202] 5GCN 5G core network

[0203] Al artificial intelligence

[0204] AIML artificial intelligence / machine learning

[0205] AMF access and mobility management function

[0206] CIR channel impulse response

[0207] DP delay profile

[0208] E-SMLC evolved serving mobile location center

[0209] GMLC Gateway Mobile Location Center

[0210] eNB (or eNodeB) evolved Node B (e.g., an LTE base station)

[0211] FFS for future study

[0212] gNB (or gNodeB) base station for 5G / NR

[0213] I / F interface

[0214] LMF Location Management Function

[0215] LoS or LOS line of sight

[0216] LTE long term evolution

[0217] ML machine learning

[0218] MME mobility management entity

[0219] NLoS or NLOS non-line of sight

[0220] NF network function

[0221] ng or NG next generation

[0222] NR new radio

[0223] NRF Network Repository Function

[0224] N / W or NW network

[0225] PCA principal components analysis

[0226] PC principal component

[0227] PDP power delay profile

[0228] RAN radio access network

[0229] PRS positioning reference signal

[0230] PRU positioning reference unit

[0231] RSRP Reference Signal Received Power

[0232] Rx receiver

[0233] SGW serving gateway

[0234] SI study item

[0235] SID Study Item Description

[0236] SMF session management function

[0237] SRS sounding reference signal

[0238] ToA time of arrival

[0239] TRP transmission-reception point

[0240] Tx transmitter

[0241] UDM unified data management

[0242] UDR unified data repository

[0243] UE user equipment (e.g., a wireless, typically mobile device)

[0244] UPF user plane function

Claims

What is claimed is:

1. A method, comprising: extracting, at a user equipment having a trained model used for positioning of the user equipment within a cellular network, characterization for data that is input to the trained model for machine-learning inference to form an inference data characterization; comparing, by the user equipment, the inference data characterization with training input data characterization from a characterization of training input data that were used to train a model that resulted in the trained model; computing, by the user equipment, a metric for the trained model based on the comparison of the training input data characterization and the inference data characterization; and sending indication of the metric from the user equipment toward a location management function in the cellular network.

2. The method according to claim 1, wherein: the method further comprises receiving, by the user equipment from the location management function, a request to perform self-monitoring; and performing, by the user equipment, at least the extracting, comparing, and computing, in response to receiving the request.

3. The method according to claim 2, further comprising receiving, by the user equipment, indication of configuration from the location management function, the configuration comprising a definition of the metric to be computed and monitored by performing at least the extracting, comparing, and computing, and a criterion based on the computed metric to indicate degraded performance of the trained model.

4. The method according to claim 3, wherein the sending the indication is performed because a condition based on the criterion is verified to be met.

5. The method according to any one of claims 2 to 4, further comprising receiving, by the user equipment, indication of the training input data characterization from the location management function.

6. The method according to any one of claims 2 to 4, further comprising: receiving, by the user equipment from the location management function, labelled data used for training the model and comprising the training input data; and performing, by the user equipment, characterization of the training input data to determine the training input data characterization.

7. The method according to any one of claims 1 to 6, further comprising computing the metric based on the training input data characterization and the inference data characterization.

8. The method according to claim 2, wherein: the request to perform self-monitoring comprises N principal components that are received for principal component analysis, N >1 but less than or equal to a number of transmission-reception points used to determine the N principal components; the method further comprises receiving, by the user equipment, a trained function for principal component analysis and a configuration for monitoring the N principal components; the extracting the characterization for data that is input to the trained model for inferencing comprises calculating the N principal components based on the trained function for principal component analysis;the comparing the inference data characterization with training input data characterization comprises comparing the calculated N principal components and the received N principal components; and the computing the metric comprises computing a metric based on the comparison of the calculated N principal components and the received N principal components.

9. The method according to claim 8, wherein the metric is one of a drift distance, minimum value, mean value, or a value of a cumulative distribution function on a distance calculated between the input training data characterization and inference input data characterization.

10. The method according to any one of claims 8 or 9, wherein: the method further comprises receiving, by the user equipment from the location management function, one or more rules for performing the computing of the metric using the calculated N principal components and the received N principal components; and performing, by the user equipment based on the one or more rules, the computing of the metric using the calculated N principal components and the received N principal components.

11. A method, comprising: receiving, by a location management function in a cellular network from a user equipment, indication of a metric, the metric corresponding to a trained model used by the user equipment for positioning of the user equipment within the cellular network and based on a comparison of training input data characterization and inference data characterization, the training input data characterization from a characterization of training input data that were used to train the model, and the inference data characterization from a characterization for data that is input to the trained model for machine-learning inference; and determining one or more actions to perform based on at least the metric.

12. The method according to claim 11 , wherein: the method further comprises aggregating metrics received from multiple user equipment; and the determining the one or more actions to perform is based on the aggregated metrics.

13. The method according to any one of claims 11 or 12, wherein: the method further comprises sending, by the location management function to the user equipment, a request to perform self-monitoring.

14. The method according to claim 13, further comprising sending by the location management function to the user equipment, indication of configuration comprising a definition of the metric to be computed, and a criterion based on the computed metric to indicate degraded performance of the trained model.

15. The method according to any one of claims 13 or 14, further comprising: determining, by the location management function, characterization of the training input data to determine the training input data characterization; and sending, from the location management function to the user equipment, indication of the training input data characterization.

16. The method according to any one of claims 13 or 14, further comprising: sending, by the location management function to the user equipment, labelled data that comprises training input data for training the model by the user equipment.

17. The method according to claim 13, wherein: the method further comprises the location management function determining a number N of principal components for principal component analysis, the number N of the principal components being less than or equal to a number of transmissionreception points used to determine the N principal components, from trainingsamples from the data that is input to the trained model for machine-learning inference; the request to perform self-monitoring comprises indication of the determined N principal components for principal component analysis; and the method further comprises sending, by the location management function to the user equipment, a trained function for principal component analysis and a configuration for monitoring the N principal components using a metric.

18. The method according to claim 17, wherein the metric is one of a drift distance, minimum value, mean value, or a value of a cumulative distribution function on a distance calculated between the input training data characterization and inference input data characterization.

19. The method according to any one of claims 17 or 18, wherein: the method further comprises sending, by the location management function to the user equipment, one or more rules for performing by the user equipment calculating of the metric using N principal components to be calculated by the user equipment and N principal components sent by the location management function to the user equipment in the configuration for monitoring the N principal components.

20. A computer program, comprising instructions for performing the methods of any of claims 1 to 19, when the computer program is run on an apparatus.

21. A computer program product comprising a computer-readable medium bearing instructions embodied therein, which when executed by at least one processor cause a user equipment to perform the method of any of claims 10 or cause a location management function to perform the method of any of claims 11-19.

22. An apparatus, comprising means for performing the method according to any of claims 1- 10 or means for performing the method according to any of claims 11-19.

23. An apparatus, comprising: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform the method of any of claims 1- 10 or 11-19.

Images