External model monitoring of artificial intelligence / machine learning model

Abstract

A method (1200) performed by a first radio node (102, 202, 302, 502, 602, 702, 802, 902, 1002) for performing inference for positioning includes performing (1202) Artificial Intelligence / Machine Learning, AI / ML, based inference for positioning for a wireless device (1212). The first radio node transmits (1204), to a second radio node (104, 204, 304, 504, 604, 704, 804, 904, 1004) operating as a model monitoring node, model inference output of the AI / ML based inference for positioning. The first radio node receives (1206), from the second radio node, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning.

Description

[0001] EXTERNAL MODEL MONITORING OF ARTIFICIAL INTELLIGENCE / MACHINE LEARNING MODEL

[0002] TECHNICAL FIELD

[0003] The present disclosure relates, in general, to wireless communications and, more particularly, systems and methods for external model monitoring of Artificial Intelligence / Machine Learning model.

[0004] BACKGROUND

[0005] Artificial intelligence (Al) and machine learning (ML) have been investigated, both in academia and industry, as promising tools to optimize the design of the air interface in wireless communication networks. Example use cases include using autoencoders for channel state information (CSI) compression to reduce the feedback overhead and improve channel prediction accuracy; using deep neural networks for classifying line-of-sight (LOS) and non-LOS (NLOS) conditions to enhance positioning accuracy; using reinforcement learning for beam selection at the network side and / or the user equipment (UE) side to reduce the signaling overhead and beam alignment latency; and using deep reinforcement learning to learn an optimal precoding policy for complex multiple input multiple output (MIMO) precoding problems.

[0006] The Third Generation Partnership Project (3GPP) New Radio (NR) standardization work for Release 18 (Rel. 18) included a study item (SI) on AI / ML for the NR air interface. The work explores the benefits of augmenting the air interface with features enabling improved support of AI / ML-based algorithms for enhanced performance and / or reduced complexity / overhead. Through studying and specifying a few selected use cases (CSI feedback, beam management, and positioning), the works aim to design the mechanisms to accommodate AI / ML into the 3rdGeneration Partnership Project (3 GPP) standard.

[0007] Building an AI / ML model includes several development steps where the actual training of the Al model is just one step in a training pipeline. An important part in AI / ML development is the AI / ML model lifecycle management (LCM), which is illustrated in FIGURE 1. Specifically, FIGURE 1 is a flow diagram illustrating training and inference pipelines, and their interactions within a model lifecycle management procedure.

[0008] The Al model lifecycle management typically consists of a training (re-training) pipeline. Data ingestion refers to gathering raw (training) data from a data storage. After data ingestion, there may also be a step that controls the validity of the gathered data. Data pre-processing refers to feature engineering applied to the gathered data, for example, it may include data normalization and possibly a data transformation required for the input data to the AI / ML model.

[0009] The actual model training steps is where a model is obtained using the training dataset.

[0010] Model evaluation refers to benchmarking the performance to a baseline. The iterative steps of model training and model evaluation continues until the acceptable level of performance is achieved.

[0011] Model registration refers to registering the AI / ML model, including any corresponding AI / ML-meta data that provides information on how the AI / ML model was developed, and possibly AI / ML model evaluations performance outcomes.

[0012] The Al model lifecycle management also typically consists of a deployment stage to make the trained (or re-trained) AI / ML model part of the inference pipeline.

[0013] The Al model lifecycle management also typically consists of an inference pipeline. Data ingestion refers to gathering raw (inference) data from a data storage. The data pre-processing stage is typically identical to corresponding processing that occurs in the training pipeline.

[0014] Model operational refers to using the trained and deployed model in an operational mode. Data and model monitoring refers to validating that the inference data are from a distribution that aligns well with the training data, as well as monitoring model outputs for detecting any performance, or operational, drifts.

[0015] The Al model lifecycle management also typically consists of a drift detection stage that informs about any drifts in the model operations.

[0016] One important AI / ML physical (PHY) use case is the positioning of a target UE. Both positioning approaches below have been shown to be effective in obtaining target UE's location.

[0017] One is direct AI / ML positioning, where the AI / ML model output is UE location. Direct AI / ML positioning typically refers to radio fingerprinting, where channel observation is used as the input of AI / ML model.

[0018] Another is AI / ML assisted positioning, where the AI / ML model output is new measurement and / or enhancement of existing measurement. The model output can be, for example, LOS / NLOS identification, timing and / or angle measurement, likelihood or reliability of the measurement. The model input is also channel observations.

[0019] When applying the direct and assisted AI / ML positioning to NR wireless communication network, the following cases are further identified for investigation.

[0020] • Case 1 : UE-based positioning with UE-side model, direct AI / ML or AI / ML assisted positioning • Case 2a: UE-assisted / location management function (LMF)-based positioning with UE-side model, AI / ML assisted positioning

[0021] • Case 2b: UE-assisted / LMF-based positioning with LMF-side model, direct AI / ML positioning

[0022] • Case 3a: NG-RAN node assisted positioning with gNodeB (gNB)-side model, AI / ML assisted positioning

[0023] • Case 3b: NG-RAN node assisted positioning with LMF-side model, direct AI / ML positioning

[0024] For radio signal based positioning methods, conventional methods rely on a sufficient number of LOS links, typically at least three to five LOS links depending on the positioning method, and whether vertical position is estimated in addition to horizontal position.

[0025] In a cluttered environment, there is often a low probability of line-of-sight for a radio link between a UE and a transmission reception point (TRP). For example, for InF-DH (Indoor Factory with Dense clutter and High base station height (Tx or Rx elevated above the clutter)) environment, Table 1 shows the LOS probabilities of a radio link between TRP and UE when assuming different InF-DH clutter parameter settings. It is observed that the LOS probability ranges from 44.9% in a mildly cluttered environment to only 0.8% in a heavily cluttered environment.

[0026] Table 1

[0027] Thus, conventional positioning methods struggle to locate a target UE in a heavily cluttered environment. Evaluations show that the 90%-tile positioning accuracy of conventional positioning methods is more than 15 meters in an InF-DH {60%, 6m, 2m} environment, due to the unavailability of sufficient LOS links. This motivates the application of AI / ML based positioning in such challenging deployment environments. Evaluations show that an AI / ML model can be trained to deliver 90%-tile positioning accuracy below 1 meter. While a well-functioning model can accurately determine the target UE's location in model inference, it has also been observed that the model performance can be sensitive to environment changes. Therefore model monitoring is important in the life-cycle management of AI / ML models for positioning.

[0028] There currently exist certain challenges, however. For example, for AI / ML based positioning, model monitoring has been investigated. However, it is difficult to verify whether the output of the model is accurate, especially in a highly cluttered environment.

[0029] For performance based model monitoring, the monitoring metric is ideally the inference accuracy of model output, which is obtained by comparing the model-generated output with ideal output. However, for AI / ML positioning, when the model is operational, it is often impractical to obtain ideal model output (e.g., true location of the target UE).

[0030] Thus, there is a need to find a good substitute of ideal output for calculating the performance-based model monitoring metric.

[0031] SUMMARY

[0032] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges. For example, particular embodiments support model monitoring by an external entity which is different from the entity which performs model inference. The information of LOS link(s) between the target UE and surrounding TRP(s) are used to calculate the accuracy of model output and provide performance monitoring metric. The information of the LOS link(s) include the classification of the link property (line-of-sight vs non-line-of-sight) and measurement of a LOS link.

[0033] According to certain embodiments, a method performed by a first radio node for performing inference for positioning includes performing AI / ML based inference for positioning for a wireless device. The first radio node transmits, to a second radio node operating as a model monitoring node, model inference output of the AI / ML based inference for positioning. The first radio node receives, from the second radio node, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning.

[0034] According to certain embodiments, a first radio node for performing inference for positioning is configured to perform AI / ML based inference for positioning for a wireless device. The first radio node is configured to transmit, to a second radio node operating as a model monitoring node, model inference output of the AI / ML based inference for positioning. The first radio node is configured to receive, from the second radio node, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning.

[0035] According to certain embodiments, a method performed by a second radio node operating as a model monitoring node includes receiving, from a first radio node, model inference output of an AI / ML based inference for positioning for a wireless device. The second radio node transmits, to the first radio node, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning for the wireless device.

[0036] According to certain embodiments, a second radio node operating as a model monitoring node is configured to receive, from a first radio node, model inference output of an AI / ML based inference for positioning for a wireless device. The second radio node is configured to transmit, to the first radio node, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning for the wireless device.

[0037] Certain embodiments may provide one or more of the following technical advantages. For example, particular embodiments provide a reliable way to calculate performance-based model monitoring metric. Information of the LOS links is recognized as providing reliable positioning related measurements. Even in an environment where LOS links are not abundant, LOS link information is captured where available, and leveraged to support verification of AI / ML model performance.

[0038] Other advantages may be readily apparent to one having skill in the art. Certain embodiments may have none, some, or all of the recited advantages.

[0039] BRIEF DESCRIPTION OF THE DRAWINGS

[0040] For a more complete understanding of the disclosed embodiments and their features and advantages, reference is now made to the following description, taken in conjunction with the accompanying drawings, in which:

[0041] FIGURE 1 illustrates the AI / ML model LCM;

[0042] FIGURE 2 illustrates a general setup of external monitoring embodiments with a second node performing model monitoring using assistant information located at the same second node, according to certain embodiments;

[0043] FIGURE 3 illustrates a general setup of external monitoring embodiments with a second node performing model monitoring using assistant information located provided by a third node, according to certain embodiments;

[0044] FIGURE 4 illustrates a general setup of external monitoring embodiments with a second node performing model monitoring using assistant information provided by the first node, according to certain embodiments;

[0045] FIGURE 5 illustrates an example of sliding windows for model performance monitoring, according to certain embodiments;

[0046] FIGURE 6 illustrates an example of an external monitoring, where ME model in UE (Entity#A) is monitored by LMF (Entity#B) using assistant information located at the same Entity#B, according to certain embodiments;

[0047] FIGURE 7 illustrates an example of an external monitoring, where ML model in UE (Entity#A) is monitored by LMF (Entity#B) using assistant information provided by a gNB (Entity#C), according to certain embodiments;

[0048] FIGURE 8 illustrates an example of an external monitoring, where ML model in UE (Entity# A) is monitored by LMF or a gNB (Entity#B) using assistant information provided by the UE (Entity# A), according to certain embodiments;

[0049] FIGURE 9 illustrates an example of an external monitoring, where ML model in gNB (Entity#A) is monitored by LMF (Entity#B) using assistant information provided by the gNB (Entity#A), according to certain embodiments;

[0050] FIGURE 10 illustrates an example of an external monitoring, where ML model in LMF (Entity#A) is monitored by a UE (Entity#B) using assistant information provided by a gNB or another UE (Entity#C), according to certain embodiments;

[0051] FIGURE 11 illustrates an example of an external monitoring, where ML model in LMF (Entity#A) is monitored by a gNB (Entity#B) using assistant information located at the same Entity#B, according to certain embodiments;

[0052] FIGURE 12 illustrates the cumulative distribution function (CDF) of the difference of the Time of Arrival (ToA) reports generated by the inference model and the monitoring model for the links classified as LOS by the monitoring model, according to certain embodiments;

[0053] FIGURE 13 illustrates an example method performed by a first radio node for performing inference for positioning, according to certain embodiments;

[0054] FIGURE 14 illustrates a method performed by a second radio node operating as a model monitoring node, according to certain embodiments;

[0055] FIGURE 15 illustrates an example communication system, according to certain embodiments;

[0056] FIGURE 16 illustrates an example UE, according to certain embodiments;

[0057] FIGURE 17 illustrates an example network node, according to certain embodiments; and FIGURE 18 illustrates a virtualization environment in which functions implemented by some embodiments may be virtualized, according to certain embodiments.

[0058] DETAILED DESCRIPTION

[0059] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

[0060] In some embodiments, the non-limiting terms UE or a wireless device are used interchangeably. The UE herein may be any type of wireless device capable of communicating with a network node or another UE over radio signals. The UE may also be a radio communication device, target device, device to device (D2D) UE, machine type UE or UE capable of machine to machine communication (M2M), low-cost and / or low-complexity UE, a sensor equipped with UE, tablet, mobile terminals, smart phone, laptop embedded equipment (LEE), laptop mounted equipment (LME), USB dongles, Customer Premises Equipment (CPE), an Internet of Things (loT) device, or a Narrowband loT (NB-IOT) device, etc.

[0061] In some embodiments the generic term “anchor node” is used, which are used as reference points for determining the location of a target UE. In general, the anchor nodes for positioning can be a variety of nodes in the wireless network. For positioning using the radio link between the target UE and a radio network node, the anchor node can be any kind of a radio network node which may comprise any of base station, radio base station, base transceiver station, Node B, evolved Node B (eNB), Next-Generation Node B (gNodeB or gNB), NG-RAN node, Transmission Point (TP), Transmission-Reception Point (TRP), Multi-cell / multicast Coordination Entity (MCE), relay node, access point (AP), Antenna Reference Point (ARP), radio access point, Remote Radio Unit (RRU), Remote Radio Head (RRH). For positioning using sidelink between two UEs, the anchor node is a UE or a wireless device.

[0062] In general, for AI / ML based positioning, particular embodiments opportunistically leverage line-of-sight links between a target UE and surrounding TRPs, even when LOS links are scarce and insufficient to pinpoint the target UE's location.

[0063] Certain embodiments described herein support model monitoring by an external entity that is different from the entity which performs model inference. The information of LOS link(s) between the target UE and surrounding TRP(s) are used to calculate the accuracy of model output and provide a performance monitoring metric. The information of the LOS link(s) include the classification of the link property (line-of-sight vs non-line-of-sight) and measurement of a LOS link. In a particular embodiment, the information of the LOS link(s) is provided by a third node, distinct from the inference and monitoring nodes / entities, as a type of assistance information.

[0064] In a particular embodiment, and in response to detecting performance deterioration beyond a threshold, the monitoring entity takes action itself to correct the poor performance. The possible actions include: deactivate the current model from performing further model inference; perform model switching so that a more appropriate model is used instead of the current model; trigger model re-training (either offline training or online training) to update the model.

[0065] For ease of discussion, the methods and systems are described using the radio links between a UE and an anchor node. TRP is used as a representative example of the anchor node, where the TRP is connected to a gNB. It is understood by those skilled in the art, however, that the same methodology may be applied to many other wireless communication scenarios such as, for example, sidelink-based positioning.

[0066] For target UEs in a highly NLOS environment, there is often insufficient LOS links to support conventional positioning methods, where triangulation or tri-lateration is used to calculate the location of the target UE. For determining horizontal location, a minimum of three or four LOS links between the target UE and anchor nodes are required, depending on the method used.

[0067] On the other hand, it is observed that LOS links still exist, though with low probability. Thus opportunistic LOS link information may be leveraged for performance monitoring of A EM L model for positioning, even if the number of LOS links is fewer than the minimum required for conventional methods. If a link is found to be LOS, measurements may be used to obtain the distance (dL0S, in meters) and / or angular information (aL0S, in degree) of the given wireless link.

[0068] The determination of the LOS property may be determined by AI / ML models or by conventional, non-AI / ML methods. Regardless of non-AI / ML or AI / ML based methods, in general, the LOS / NLOS indicator may be hard-valued or soft-valued. For example, the Release- 17 LOS-NLOS-Indicator provides information on the likelihood of a LOS propagation path from the source to the receiver with a value of 1 corresponding to LOS and a value of 0 corresponding to NLOS. When the LOS / NLOS indicator takes soft value, integer value 'O' indicates likelihood 0, integer value TO' indicates likelihood 1. Thus, a scaling factor 0.1 is applied to the integer value to arrive at the LOS probability. When the LOS / NLOS indicator takes hard value, then FALSE indicates likelihood 'O', TRUE indicates likelihood 'I'.

[0069] LOS-NLOS-Indicator-rl7 ::= SEQUENCE { indicator-rl7 CHOICE { soft-rl7 INTEGER (O..1O), hard-rl7 BOOLEAN },

[0070] }

[0071] Accordingly, a link can be classified as LOS if the LOS / NLOS indicator has value TRUE or T when the indicator is provided by hard value, or the LOS / NLOS indicator has value higher than a predefined threshold PLOS, thresh, when the indicator is provided by soft value. For example, PLOS, thresh=0-9, i.e., the link is determined to be LOS if likelihood of the TRP-UE link being LOS is estimated to be above 90%.

[0072] If a TRP-UE link is determined to be LOS, then measurements of the link may be used to obtain positioning related information.

[0073] In one example, the measurements may be timing information of the LOS link, for example, ToA, Reference Signal Time Difference (RSTD), Relative Time of Arrival (RTOA), Rx - Tx time difference (RxTxTimeDiff). The timing information may be obtained by UE via measurement of downlink (DL) reference signals (typically positioning reference signal (PRS)), for example, ToA, DL RSTD, UE RxTxTimeDiff. Alternatively or additionally, the timing information may be obtained by gNB via measurement of uplink (UL) reference signals (typically sounding reference signal (SRS)), for example, ToA, UL-RTOA (UL Relative Time of Arrival), gNB RxTxTimeDiff. Using the timing information of the LOS link, the distance dL0S(meters) between the UE and the TRP can be calculated using speed-of-light.

[0074] In another example, the measurements may be angular information aL0S(degrees) of the LOS link, for example, angle-of-arrival (AoA), angle -of-departure (AoD). The angular information may be obtained by UE via measurement of DL reference signals, for example, DL AoD (DL AoD). Alternatively or additionally, the angular information may be obtained by gNB via measurement of UL reference signals, for example, UL AoA (UL AoA).

[0075] The positioning related information (e.g., dL0Sand / or aL0S) of a LOS link may be obtained using conventional (i.e., non-AI / ML) methods or AI / ML models.

[0076] The following is a rationale to use information of the LOS link to verify AI / ML model performance. For a UE in a cluttered environment (e.g., {60%, 6m, 2m}), the LOS probability is low but non-zero (see Table 1). When an AI / ML model performs model inference continuously, LOS link can be experienced from time to time. For example, for a UE in an environment with cluter {60%, 6m, 2m}, if a UE observes PRS from 16 TRP at a time, on average one LOS link is experienced for every 8 observations.

[0077] Therefore, even when an AI / ML model is used to support UE positioning in heavily NLOS environment, information of LOS links can be captured when available, and used to check the accuracy of the AI / ML model output. This is applicable for various methods and approaches of AI / ML based positioning, including direct AI / ML positioning and AI / ML assisted positioning approaches.

[0078] To provide independent performance checking as much as possible, the classification of LOS link(s) and the information of identified LOS links are preferably provided by a mechanism separate from the AI / ML for UE positioning. Moreover, the LOS link information is obtained by a different entity (Entity#B) using different data. Lor example, in AI / ML positioning Case 1 where the UE is the model inference entity, the gNB provides LOS link information to the model monitoring entity based on gNB observation of SRS transmission on the UL, where the information includes the LOS / NLOS indicator and positioning related information of the LOS link.

[0079] In external model monitoring, for a given AI / ML model, one entity (entity#A) performs model inference, while another entity (entity#B) conducts model performance monitoring.

[0080] EIGURE 2 is a block diagram illustrating an example 100 of external monitoring embodiments with a first node (entity# A) 102 and a second node (entity#B) 104 where the second node (entity) 104 performs model monitoring using assistant information located at the same second node, according to certain embodiments. Thus, in the illustrated embodiment, the LOS link information is generated by the second node (entity#B) 104, which may also be referred to as a monitoring entity as second node (entity#B) 104 performs model monitoring using the LOS link information.

[0081] In another embodiment, the LOS link information is generated by a third entity (entity#C). EIGURE 3 is a block diagram illustrating an example 200 of external monitoring embodiments with a first node (entity#A) 202 and a second node (entity#B) 204 where the second node (entity#B) 204 performs model monitoring using assistant information located provided by a third node (entity#C) 206, according to certain embodiments.

[0082] In still another embodiment, the LOS link information is provided to the monitoring entity (entity#B) by entity#A. FIGURE 4 is a block diagram illustrating an example setup 300 of external monitoring embodiments with a first node (entity# A) 302 and a second node (entity#B) 304 with the second node (entity#B) 304 performing model monitoring using assistant information provided by the first node (entity# A) 302, according to certain embodiments.

[0083] When entity#A provides the LOS link information (as illustrated in FIGURE 4) or when entity#C provides the LOS link information (as illustrated in FIGURE 3), the LOS link information may be sent as a type of assistance information. In particular embodiments, the assistance information includes both (a) the classification that the link is LOS; and (b) positioning related information of the LOS link (e.g., distance dL0S(meters) between the UE and a TRP; angular information aL0S(degrees) of the LoS link).

[0084] The following describes how certain embodiments described herein use information of the LOS link to verify AI / ML model performance. For example, according to certain embodiments, the independently obtained positioning information of LOS link(s) is compared with the counterpart from AI / ML model output to calculate the accuracy of model output.

[0085] In a particular embodiments, a sliding window is defined to collect the comparison data. FIGURE 5 illustrates an example 400 of sliding windows for model performance monitoring, according to certain embodiments. The occasions with detected LOS links between the target UE and TRP(s) are used to obtain model monitoring metrics.

[0086] Different windowing techniques may be applied. For example, a fixed window size is used for monitoring, in a particular embodiment. Alternatively, in another embodiment, an adaptive window size is used, where the window size varies depending on the perceived model performance. For example, the window size shrinks when the performance metric degrades, and the window size grows when the performance metric improves.

[0087] It is understood by those skilled in the art that FIGURE 5 only illustrates one representative example. In actual operation, many factors need to be considered in determining the final schedule of collecting comparison data and performing model monitoring. Such factors may include: the design target of detection accuracy and detection latency; the extra burden (e.g., power consumption, storage size, computation complexity, signaling overhead) due to model monitoring; the capability of availability of assisting nodes to detect and measure LOS links to support the AI / ML positioning in Entity#A; the need to coordinate activities among multiple AI / ML models (if present). It is understood by those skilled in the art that such variations are still covered by the disclosed methods herein.

[0088] Within a given window j, assume occasions k = 0,1, — 1 have LOS links detected by a method separate from the AI / ML inference model for positioning. In the L-th occasion of window j, NjkLOS links are detected at occasion k between the UE and NjkTRPs. Then model performance monitoring metric may be calculated by comparing the independently obtained positioning related information (e.g., distance dL0S) of LOS link(s) with the corresponding information (e.g., distance dML) provided by the AI / ML inference model. While the same principle applies to any type of positioning related information (e.g., timing information, angular information), the discussion here mainly uses distance information as a representative example.

[0089] Note the positioning related information (e.g., distance dML) provided by the AI / ML inference model may be obtained in two different ways. In a first way, the AI / ML inference model outputs the positioning related information (e.g., distance dML) to the TRP with identified LOS link status. In a second way, the positioning related information (e.g., distance dL0S) is computed based on the output of the AI / ML inference model.

[0090] As a nonlimiting example, the AI / ML inference model outputs the estimate of the UE position. If the required positioning related information is a distance dMLto the TRP with identified LOS link status, then dMLis computed as the distance between the UE position estimate and the known position of the TRP.

[0091] As another nonlimiting example, the AI / ML inference model outputs the estimate of the UE position. If the required positioning related information is an angle aMLtoward the TRP with identified LOS link status, then aMLmay be calculated based on the UE position estimate and the known position of the TRP.

[0092] With a list of independently obtained positioning related information (e.g., distance dL0S) and their AI / ML counterparts for the LOS links, the model performance monitoring metrics may be calculated. Several representative methods are provided below.

[0093] For example, if distance dL0S(meters) is the positioning information of the LOS links, then for window j, the model performance monitoring metric can be calculated by:

[0094] If Mj is greater than a threshold Mthrsh(meters), then significant performance degradation is declared.

[0095] In another example, if angle aL0S(degree) is the positioning information of the LOS links, then for window j, the model performance monitoring metric can be calculated by:

[0096] If Mj is greater than a threshold Mthrsh(degree), then significant performance degradation is declared. It is understood by those skilled in the art that the metric calculations above are just examples to illustrate the concept. Other variations may be used without departing from the principle of the methodology. In one example, instead of using averages in (1) and (2), other statistical measures such as min, max may be applied. An example of using max may be:

[0097] In another example, all the LOS links are used in the calculation equally, for example, (4) may be used instead of (1),

[0098] In another alternative, the LOS links detected are assigned different weights in the calculation, e.g., (5) is used instead of (1),

[0099] Different weights w7-fe nmay be assigned to different LOS links to obtain a more accurate performance metric. For example, LOS link identified with higher confidence (e.g., LOS probability >0.95) is given higher weights than those with lower confidence (e.g., 0.5 < LOS probability <0.95), and / or LOS link with higher measurement quality (e.g., more accurate UE- TRP distance estimation) is given higher weights than those with lower quality.

[0100] For making performance monitoring decisions, the thresholds Mthrshmay be determined based on various factors, e.g., acceptable error range of model output (e.g., higher threshold value Mthrsflif acceptable error is larger), the reliability of LOS link detection (e.g., lower threshold value Mthrshif LOS detection reliability is higher), etc.

[0101] Moreover, the metric may be compared to more than one threshold value to provide a soft indicator of the inference model performance reliability.

[0102] Furthermore, model performance monitoring decisions may be made by observing metrics in one or more windows consecutively, or over a period of time. If Mj > Mthrsflin at least Nwdwindows, then a decision may be made that model performance has deteriorated significantly.

[0103] Some embodiments include action by the model monitoring entity (entity#B) 104, 204, 304. In a first embodiment, after evaluating the model performance reliability, the model monitoring entity (Entity#B) 104, 204, 304 may provide an indicator to the model inference node (Entity#A) 102, 202, 302, which is the result ofthe model performance monitoring procedure. The model performance reliability indicator may be hard-valued or soft- valued. In a particular embodiment, the model performance reliability indicator is provided as a hard value with choice between “reliable” and “not reliable.” As one nonlimiting example, the “reliable” value may be represented as 1 and the “not reliable” value may be represented as 0.

[0104] In another nonlimiting example embodiment, the model performance reliability indicator is provided as a soft value. As one nonlimiting example, the indicator may have an integer value ranging from 0 to 10. The integer value of 0 indicates the inference model is not reliable. Higher values indicate higher likelihood that the inference model is reliable.

[0105] A nonlimiting signaling format for the model performance reliability indicator supporting both types of indicators is shown below. The signaling format enables the model monitoring entity to choose to provide a hard or a soft indicator to the model inference entity. inference-model-reliability-Indicator ::= SEQUENCE { indicator CHOICE { soft INTEGER (0..10), hard BOOLEAN

[0106] },

[0107] }

[0108] In a second embodiment, when the model monitoring entity (Entity#B 104, 204, 304) determines that the model performance (in Entity# A 102, 202, 302) has deteriorated significantly, then Entity#B 104, 204, 304 may trigger action to correct the performance degradation in Entity# A 102, 202, 302. The possible actions include one or more of the following:

[0109] • Entity#B 104, 204, 304 triggers deactivation of the AI / ML model for positioning in Entity# A 102, 202, 302.

[0110] • Entity#B 104, 204, 304 triggers switching to a different AI / ML model for positioning in Entity# A 102, 202, 302.

[0111] • Entity#B 104, 204, 304 sends an error message that Entity#A 102, 202, 302 cannot provide positioning related reports due to failure of the AI / ML model. In a particular embodiment, for example, the error message is sent to a node that manages the positioning service, for example, a location server or a 0AM function.

[0112] • Entity#B 104, 204, 304 triggers a model update of the model in Entity# A 102, 202, 302 via fine-tuning or re-training. The model update can be carried out online or offline.

[0113] When Entity#B 104, 204, 304 triggers actions to address the performance degradation problem in Entity#A 102, 202, 302 (e.g., adapt the model to better match the deployment environment), Entity#B 104, 204, 304 may or may not notify other entities (e.g., a network node) beyond Entity# A and #B about the actions.

[0114] In one example, if Entity#B 104, 204, 304 triggers model switching or model updating in Entity#A 102, 202, 302 while continuously maintaining the functionality of AI / ML based positioning, then no service disruption is visible to external nodes and no notification to external nodes is necessary.

[0115] In another example, if Entity#B 104, 204, 304 takes actions that may affect other nodes (e.g., deactivate AI / ML functionality), or need external support to adapt the model (e.g., need assistance information for re-training the model), then Entity#B 104, 204, 304 may notify and / or exchange information with external nodes.

[0116] Some embodiments include actions in response to detecting AI / ML model performance degradation by the model inference entity. When the model inference entity (Entity#A 102, 202, 302) receives an indicator that its model performance has deteriorated significantly, then Entity#A 102, 202, 302 may take action itself to correct the problem. The possible actions include one or more of the following:

[0117] • Entity#A 102, 202, 302 deactivates its AI / ML model for positioning.

[0118] • Entity#A 102, 202, 302 switches to a different AI / ML model for positioning.

[0119] • Entity#A 102, 202, 302 sends an error message that it cannot provide positioning related reports due to failure of its AI / ML model. The error message may be sent to a node connected to Entity#A 102, 202, 302 in providing the positioning service, for example, a location server or an 0AM function.

[0120] • Entity#A 102, 202, 302 sends a confidence indicator on the model performance of its AI / ML model. The confidence indicator may be sent to a management node connected to Entity#A 102, 202, 302 in providing the positioning service, for example, a location server or an 0AM function.

[0121] • Entity#A 102, 202, 302 triggers a model update via fine-tuning or retraining. The model update may be carried out online or offline. • When Entity#A 102, 202, 302 takes actions to address the performance degradation problem (e.g., adapt its model to better match the deployment environment), Entity# A 102, 202, 302 may or may not notify other entities (e.g., a network node) about the actions.

[0122] In one example, if Entity#A 102, 202, 302 performs model switching or model updating while continuously maintaining the functionality of AI / ML based positioning, then no service disruption is visible to external nodes and no notification to external nodes is necessary.

[0123] In another example, if Entity#A 102, 202, 302 takes actions that affect other nodes (e.g., deactivate AI / ML functionality), or need external support to adapt its model (e.g., need assistance information for re-training its model), then Entity# A 102, 202, 302 may notify and / or exchange information with external nodes.

[0124] Particular embodiments may apply the method described above to an AI / ML model where the UE is the model inference entity.

[0125] Case 1: UE-Based Positioning with UE-Side Model, Direct AI / ML Positioning

[0126] According to certain embodiments, a network node (e.g., an LML) monitors the direct positioning model using the LOS link information obtained by itself or another network node (e.g., a gNB) or the UE. Lor example, the gNB identifies LOS link(s) and measurement information of the LOS links by observing UL SRS transmission and reports it to the LML. Lor example, LIGURE 6 is a block diagram illustrating an example 500 of an external monitoring, where ML model in UE (Entity# A 502) is monitored by LML (Entity#B 504) using assistant information located at the same Entity#B 504. As another example, LIGURE 7 is a block diagram illustrating an example 600 of an external monitoring, where ML model in UE (Entity#A 602) is monitored by LMF (Entity#B 604) using assistant information provided by a gNB (Entity#C 606). As still another example, FIGURE 8 is a block diagram illustrating an example 700 of an external monitoring, where ML model in UE (Entity#A 702) is monitored by LMF or a gNB (Entity#B 704) using assistant information provided by the UE (Entity# A 702).

[0127] In another example embodiment, a network node (e.g., a gNB) monitors the direct positioning model using the LOS link information obtained by a different UE, where the different UE identifies LOS link(s) and measurement information of the LOS links by observing sidelink PRS transmission.

[0128] Case 2a: UE-Assisted / IMF-Based Positioning with UE-Side Model, AI / ML Assisted Positioning According to certain embodiments, the LMF monitors the UE-assisted positioning model using the LOS link information obtained by a method collocated at the UE. The method for detecting LOS link information may be via conventional signal processing or an AI / ML model at the UE. The UE may send the LOS link information to LMF as a part of the assistance information.

[0129] For example, in a particular embodiment, the LMF monitors the UE-assisted positioning model using assistance information provided by another network node (e.g., a gNB).

[0130] In one embodiment, the LMF monitors the UE-assisted positioning model using the LOS link information obtained by a radio network node (e.g., a gNB), where the radio network node identifies LOS link(s) and measurement information of the LOS links by observing UL SRS transmission. In another embodiment, the LMF computes the position estimate of the UE. After that LMF may monitor the UE-assisted positioning model performance and determine the quality of reports from UE-assisted positioning model using its position estimate. In this case, the LMF may or may not additionally use LOS link information to monitor the model performance.

[0131] Some embodiments apply the method to an AI / ML model where the gNB is the model inference entity.

[0132] Case 3a: NG-RAN Node Assisted Positioning with gNB-Side Model, AI / ML Assisted Positioning

[0133] According to certain embodiments, the LMF monitors the NG-RAN node-assisted positioning model using the LOS link information obtained by a method collocated at the gNB. The method for detecting LOS link information may be via conventional signal processing or an AI / ML model at the gNB. The gNB may send the LOS link information to LMF as a part of the assistance information.

[0134] For example, FIGURE 9 is a block diagram illustrating an example 800 of an external monitoring, where ML model in gNB (Entity#A 802) is monitored by LMF (Entity#B 804) using assistant information provided by the gNB (Entity#A 802).

[0135] In a particular embodiment, the LMF (Entity#B 804) monitors the NG-RAN node-assisted positioning model using assistance information provided by another node (e.g., a UE or another gNB).

[0136] In one embodiment, the LMF (Entity#B 804) monitors the NG-RAN node-assisted positioning model using the LOS link information obtained by the target UE, where the target UE identifies LOS link(s) and measurement information of the LOS links by observing DL PRS transmission.

[0137] In another embodiment, the LMF (Entity#B 804) computes the position estimate of the target UE. After that the LMF (Entity#B 804) may monitor the gNB assisted positioning model performance using the position estimate. In this case, the LMF (Entity#B 804) node may or may not additionally use LOS link information to monitor the model performance.

[0138] Some embodiments apply the method to an AI / ML model where the LMF (Entity#B 804) is the model inference entity.

[0139] Case 2b: UE-Assisted / IMF-Based Positioning with IMF-Side Model, Direct AI / ML Positioning

[0140] According to certain embodiments, the UE monitors the LMF-side direct positioning model using the LOS link information obtained by a method located at the UE. The method for detecting LOS link information can be via conventional signal processing or an AI / ML model. The LOS link information may be provided by the UE itself or a radio network node (e.g., gNB) and / or the target UE. For example, FIGURE 10 is a block diagram illustrating an example 900 of an external monitoring, where ML model in LMF (Entity# A 902) is monitored by a UE (Entity#B 904) using assistant information provided by a gNB or another UE (Entity#C 906).

[0141] Case 3b: NG-RAN Node Assisted Positioning with IMF-Side Model, Direct AI / ML Positioning

[0142] According to certain embodiments, the NG-RAN node monitors the LMF-side direct positioning model using the LOS link information obtained by a method located at the NG-RAN node. The method for detecting LOS link information may be via conventional signal processing or an AI / ML model. The LOS link information may be provided by the NG-RAN node (e.g., gNB) itself and / or the target UE. For example, FIGURE 11 is a block diagram illustrating an example 1000 of an external monitoring, where ML model in LMF (Entity# A 1002) is monitored by a gNB (Entity#B 1004) using assistant information located at the same Entity#B.

[0143] FIGURE 12 is a graph 1100 illustrating the cumulative distribution function (CDF) of the difference of the ToA reports generated by the inference model and the monitoring model for the links classified as LOS by the monitoring model in two different realizations of InF-DH with clutter parameter {60%, 6m, 2m}, according to certain embodiments.

[0144] The inference model is an AI / ML model trained using the data collected from the indoor factor scenario with 60% clutter density, and clutter height and width of 6 m and 2 m, respectively (i.e., InF-DH with clutter parameter {60%, 6m, 2m}). The monitoring model is another AI / ML model trained using the data collected from the indoor factor scenario with 40% clutter density, and clutter height and width of 2 m and 2 m, respectively (i.e., InF-DH with clutter parameter {40%, 2m, 2m}). Two curves are shown in FIGURE 12. One curve is for “same environment”, which means that the inference environment is the same as the training environment for the inference model. The inference model performs normally in “same environment”, i.e., no performance degradation. The other curve is for “new environment”, which means that the inference environment is different from the training environment for the inference model, though both environments belong to InF- DH with clutter parameter {60%, 6m, 2m}. The inference model performs poorly in “new environment”, i.e., performance degradation has occurred, and model performance monitoring method should detect it.

[0145] The CDF of the LOS link ToA difference of “same environment” is much smaller and easily distinguishable from the CDF of “new environment." Thus, model performance monitoring decision may be made based on the easily distinguishable behavior.

[0146] Because the model inference environment is the heavily NLOS environment of InF-DH with clutter parameter {60%, 6m, 2m}, the LOS probability is low and inadequate for conventional positioning methods, as expected. However, approximately 10% of observed TRP-UE links may be opportunistically leveraged for performing model monitoring. This is adequate for the purpose of model performance monitoring.

[0147] This evaluation result demonstrates that it is effective to monitor AI / ML model performance using opportunistic LOS links even in a heavily NLOS environment like InF-DH with clutter parameter {60%, 6m, 2m}.

[0148] Some embodiments include signaling of the assistance information. As discussed above, there are cases where Entity#B, performance monitoring entity, receives LOS link information from entity#A, model inference entity, or a separate Entity#C, which provides assistant information. The LOS link information may be requested and delivered in information elements (IE) that convey the LOS / NLOS identification and positioning related information of the identified LOS link(s).

[0149] In the example below, the location server (e.g., LMF) makes a request to the target UE for information on the LOS links as observed by the UE.

[0150] The IE NR-DL-TDOA-RequestLocationlnformation is used by the location server to request NR DL-TDOA location measurements from a target device.

[0151] - ASN1 START

[0152] NR-DL-TDOA-RequestLocationInformation-rl6 ::= SEQUENCE { nr-DL-PRS-RstdMeasurementlnfoRequest-r 16 ENUMERATED { true } OPTIONAL,- Need ON nr-RequestedMeasurements-r 16 BIT STRING { prsrsrpReq (0), firstPathRsrpReq-rl7 (1), jointMeasurementsReq-rl8 (2)

[0153] }

[0154] (SIZE(1..8)), nr-AssistanceAvailability-r 16 BOOLEAN, nr-DL-TDOA-ReportConfig-r 16 NR-DL-TDOA-ReportConfig- rl6 OPTIONAL, - Need ON additionalPaths-r 16 ENUMERATED { requested }

[0155] OPTIONAL, - Need ON

[0156] [[ nr-UE-RxTEG-Request-r 17 ENUMERATED { requested }

[0157] OPTIONAL, - Need ON nr-los-nlos-IndicatorRequest-rl7 { type-rl7 LOS-

[0158] NLOS-IndicatorType 1 -r 17, granularity-rl7 LOS-NLOS-

[0159] IndicatorGranularity 1 -r 17,

[0160] }

[0161] OPTIONAL, - Need ON Measurementlnfo-Lo S , Measurementlnfo-Lo S -Quality, OPTIONAL,

[0162] }

[0163] The components of requested information include:

[0164] • nr-los-nlos-indicator. This provides indication on whether the link is detected to be LOS or not. The indication can be hard-valued or soft-valued and can be provided per TRP or per reference signal (e.g., PRS for DL) source.

[0165] • Measurementlnfo-LoS: this provides positioning related measurement information of the LOS link, e.g., timing information, distance information, angle information.

[0166] • Measurementlnfo-LoS-Quality: this provides quality estimation of the measurement information.

[0167] Upon receiving such a request, the target UE may respond to the location server (e.g., LMF) with measurement report that includes information on the detected LOS links. FIGURE 13 illustrates an example method 1200 performed by a first radio node 102, 202, 302, 502, 602, 702, 802, 902, 1002 for performing inference for positioning, according to certain embodiments. In the illustrated embodiment, the method 1200 begins at step 1202 when the first radio node 102, 202, 302, 502, 602, 702, 802, 902, 1002 performs AI / ML based inference for positioning for a wireless device. At step 1204, the first radio node 102, 202, 302, 502, 602, 702, 802, 902, 1002 transmits, to a second radio node 104, 204, 304, 504, 604, 704, 804, 904, 1004 operating as a model monitoring node, model inference output of the AI / ML based inference for positioning. At step 1206, the first radio node 102, 202, 302, 502, 602, 702, 802, 902, 1002 receives, from the second radio node 104, 204, 304, 504, 604, 704, 804, 904, 1004, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning.

[0168] In a particular embodiment, performing the AI / ML based inference for positioning for the wireless device includes using an AI / ML model to generate the model inference output.

[0169] In a particular embodiment, a number of available LOS links at the first radio node is less than a number of LOS links needed to perform the non-AI / ML based inference for positioning.

[0170] In a particular embodiment, the model inference output comprises a position of the wireless device.

[0171] In a particular embodiment, the model inference output comprises positioning related information for determining by the second radio node or another node a position of the wireless device.

[0172] In a particular embodiment, the first radio node transmits model monitoring assistance information to the second radio node.

[0173] In a further particular embodiment, the assistance information comprises at least one LOS NLOS indicator.

[0174] In a further particular embodiment, the at least one LOS / NLOS indicator is associated with a sequence of LOS / NLOS indicator values, wherein each value is associated with a respective one of a plurality of links.

[0175] In a particular embodiment, the assistance information includes a classification of each of at least one LOS / NLOS links as LOS or NLOS and positioning related information for each of the at least one LOS / NLOS links.

[0176] In a particular embodiment, the first radio node is the wireless device, and the second radio node is a gNB or an LMF.

[0177] In a particular embodiment, the first radio node is a gNB and the second radio node is an

[0178] LMF. In a particular embodiment, the first radio node is an LMF, and the second radio node is a UE or gNB.

[0179] In a particular embodiment, the feedback comprises an indication of at least one of: an error message; an accuracy of an A EM L model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is inaccurate; a level of reliability of the AI / ML model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is unreliable; the AI / ML model used for the AI / ML based inference for positioning is to be deactivated; the AI / ML model used for the AI / ML based inference for positioning is to be retrained; and subsequent AI / ML based inference for positioning is to be performed by a different AI / ML model.

[0180] In a particular embodiment, the first radio node performs at least one of: deactivating the AI / ML model used for the AI / ML based inference for positioning; switching to the different AI / ML model for subsequent performance of AI / ML based inference for positioning; transmitting an error message indicating that the first radio node cannot perform AI / ML based inference for positioning; transmitting a confidence indicator to a third radio node; and retraining the AI / ML model used for performing the AI / ML based inference for positioning.

[0181] FIGURE 14 illustrates a method 1300 performed by a second radio node 104, 204, 304, 504, 604, 704, 804, 904, 1004 operating as a model monitoring node, according to certain embodiments. As illustrated, the method 1300 begins at step 1302 when the second radio node 104, 204, 304, 504, 604, 704, 804, 904, 1004 receives, from a first radio node 102, 202, 302, 502, 602, 702, 802, 902, 1002, model inference output of an AI / ML based inference for positioning for a wireless device. At step 1304, the second radio node 104, 204, 304, 504, 604, 704, 804, 904, 1004 transmits, to the first radio node 102, 202, 302, 502, 602, 702, 802, 902, 1002, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning for the wireless device.

[0182] In a particular embodiment, the first radio node is the wireless device, and the second radio node is a gNB or an LMF.

[0183] In a particular embodiment, the first radio node is a gNB and the second radio node is an LMF.

[0184] In a particular embodiment, the first radio node is an LMF, and the second radio node is a UE or gNB.

[0185] In a particular embodiment, the model inference output comprises a position of the wireless device.

[0186] In a particular embodiment, the model inference output comprises positioning related information, and the second radio node determines a position of the wireless device.

[0187] In a particular embodiment, the model inference output includes positioning related information, and the second radio node receives, from a third radio node, a position of the wireless device.

[0188] In a further particular embodiment, the third radio node comprises at least one of a gNB or a UE.

[0189] In a particular embodiment, the second radio node obtains assistance information and determines the accuracy and / or reliability of the Al / based inference for positioning based on at least the assistance information.

[0190] In a particular embodiment, the assistance information comprises at least one measurement of at least one available LOS link, and the second radio node determines the accuracy and / or reliability of the Al / based inference for positioning by comparing the model inference output received from the first radio node to the at least one measurement of the at least one available LOS link at the second radio node.

[0191] In a further particular embodiment, the assistance information comprises at least one LOS / NLOS indicator.

[0192] In a further particular embodiment, the at least one LOS / NLOS indicator is associated with a sequence of LOS / NLOS indicator values, wherein each value is associated with a respective one of a plurality of links.

[0193] In a particular embodiment, the assistance information includes a classification of each of at least one LOS / NLOS links as LOS or NLOS and positioning related information for each of the at least one LOS / NLOS links.

[0194] In a particular embodiment, obtaining the assistance information includes determining the assistance information by the second radio node.

[0195] In a particular embodiment, obtaining the assistance information includes receiving the assistance information from the first radio node.

[0196] In a particular embodiment, obtaining the assistance information includes receiving the assistance information from a third radio node (206, 606, 906).

[0197] In a particular embodiment, the feedback comprises an indication of at least one of: an error message; an accuracy of an A EM L model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is inaccurate; a level of reliability of the AI / ML model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is unreliable; the AI / ML model used for the AI / ML based inference for positioning is to be deactivated; the AI / ML model used for the AI / ML based inference for positioning is to be retrained; and subsequent AI / ML based inference for positioning is to be performed by a different AI / ML model.

[0198] FIGURE 15 shows an example of a communication system 1400 in accordance with some embodiments. In the example, the communication system 1400 includes a telecommunication network 1402 that includes an access network 1404, such as a radio access network (RAN), and a core network 1406, which includes one or more core network nodes 1408. The access network 1404 includes one or more access network nodes, such as network nodes 1410a and 1410b (one or more of which may be generally referred to as network nodes 1410), or any other similar 3rd Generation Partnership Project (3GPP) access node or non-3GPP access point. The network nodes 1410 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 1412a, 1412b, 1412c, and 1412d (one or more of which may be generally referred to as UEs 1412) to the core network 1406 over one or more wireless connections.

[0199] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 1400 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 1400 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0200] The UEs 1412 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 1410 and other communication devices. Similarly, the network nodes 1410 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 1412 and / or with other network nodes or equipment in the telecommunication network 1402 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 1402.

[0201] In the depicted example, the core network 1406 connects the network nodes 1410 to one or more hosts, such as host 1416. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 1406 includes one more core network nodes (e.g., core network node 1408) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 1408. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0202] The host 1416 may be under the ownership or control of a service provider other than an operator or provider of the access network 1404 and / or the telecommunication network 1402 and may be operated by the service provider or on behalf of the service provider. The host 1416 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0203] As a whole, the communication system 1400 of FIGURE 15 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0204] In some examples, the telecommunication network 1402 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 1402 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 1402. For example, the telecommunications network 1402 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC)ZMassive loT services to yet further UEs.

[0205] In some examples, the UEs 1412 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 1404 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 1404. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi -radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

[0206] In the example, the hub 1414 communicates with the access network 1404 to facilitate indirect communication between one or more UEs (e.g., UE 1412c and / or 1412d) and network nodes (e.g., network node 1410b). In some examples, the hub 1414 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 1414 may be a broadband router enabling access to the core network 1406 for the UEs. As another example, the hub 1414 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 1410, or by executable code, script, process, or other instructions in the hub 1414. As another example, the hub 1414 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 1414 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 1414 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 1414 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 1414 acts as a proxy server or orchestrator for the UEs, in particular in if one or more of the UEs are low energy loT devices.

[0207] The hub 1414 may have a constant / persi stent or intermittent connection to the network node 1410b. The hub 1414 may also allow for a different communication scheme and / or schedule between the hub 1414 and UEs (e.g., UE 1412c and / or 1412d), and between the hub 1414 and the core network 1406. In other examples, the hub 1414 is connected to the core network 1406 and / or one or more UEs via a wired connection. Moreover, the hub 1414 may be configured to connect to an M2M service provider over the access network 1404 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 1410 while still connected via the hub 1414 via a wired or wireless connection. In some embodiments, the hub 1414 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 1410b. In other embodiments, the hub 1414 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 1410b, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0208] FIGURE 16 shows a UE 1500 in accordance with some embodiments. As used herein, a UE refers to a device capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle-mounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-IoT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE.

[0209] A UE may support device -to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

[0210] The UE 1500 includes processing circuitry 1502 that is operatively coupled via a bus 1504 to an input / output interface 1506, a power source 1508, amemory 1510, a communication interface 1512, and / or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in FIGURE 16. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0211] The processing circuitry 1502 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 1510. The processing circuitry 1502 may be implemented as one or more hardware -implemented state machines (e.g., in discrete logic, field- programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 1502 may include multiple central processing units (CPUs).

[0212] In the example, the input / output interface 1506 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 1500. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

[0213] In some embodiments, the power source 1508 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 1508 may further include power circuitry for delivering power from the power source 1508 itself, and / or an external power source, to the various parts of the UE 1500 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 1508. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 1508 to make the power suitable for the respective components of the UE 1500 to which power is supplied.

[0214] The memory 1510 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 1510 includes one or more application programs 1514, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 1516. The memory 1510 may store, for use by the UE 1500, any of a variety of various operating systems or combinations of operating systems.

[0215] The memory 1510 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUICC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 1510 may allow the UE 1500 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 1510, which may be or comprise a device-readable storage medium.

[0216] The processing circuitry 1502 may be configured to communicate with an access network or other network using the communication interface 1512. The communication interface 1512 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 1522. The communication interface 1512 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network). Each transceiver may include a transmitter 1518 and / or a receiver 1520 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 1518 and receiver 1520 may be coupled to one or more antennas (e.g., antenna 1522) and may share circuit components, software or firmware, or alternatively be implemented separately.

[0217] In the illustrated embodiment, communication functions of the communication interface 1512 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / intemet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

[0218] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 1512, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected, an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).

[0219] As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.

[0220] A UE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or itemtracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and / or software in dependence of the intended application of the loT device in addition to other components as described in relation to the UE 1500 shown in FIGURE 16.

[0221] As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements and transmits the results of such monitoring and / or measurements to another UE and / or a network node. The UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-IoT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.

[0222] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.

[0223] FIGURE 17 shows a network node 1600 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with a UE and / or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NRNodeBs (gNBs)).

[0224] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0225] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).

[0226] The network node 1600 includes a processing circuitry 1602, a memory 1604, a communication interface 1606, and a power source 1608. The network node 1600 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 1600 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 1600 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1604 for different RATs) and some components may be reused (e.g., a same antenna 1610 may be shared by different RATs). The network node 1600 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1600, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1600.

[0227] The processing circuitry 1602 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 1600 components, such as the memory 1604, to provide network node 1600 functionality.

[0228] In some embodiments, the processing circuitry 1602 includes a system on a chip (SOC). In some embodiments, the processing circuitry 1602 includes one or more of radio frequency (RF) transceiver circuitry 1612 and baseband processing circuitry 1614. In some embodiments, the radio frequency (RF) transceiver circuitry 1612 and the baseband processing circuitry 1614 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 1612 and baseband processing circuitry 1614 may be on the same chip or set of chips, boards, or units.

[0229] The memory 1604 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processing circuitry 1602. The memory 1604 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions capable of being executed by the processing circuitry 1602 and utilized by the network node 1600. The memory 1604 may be used to store any calculations made by the processing circuitry 1602 and / or any data received via the communication interface 1606. In some embodiments, the processing circuitry 1602 and memory 1604 is integrated.

[0230] The communication interface 1606 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, the communication interface 1606 comprises port(s) / terminal(s) 1616 to send and receive data, for example to and from a network over a wired connection. The communication interface 1606 also includes radio front-end circuitry 1618 that may be coupled to, or in certain embodiments a part of, the antenna 1610. Radio front-end circuitry 1618 comprises filters 1620 and amplifiers 1622. The radio frontend circuitry 1618 may be connected to an antenna 1610 and processing circuitry 1602. The radio front-end circuitry may be configured to condition signals communicated between antenna 1610 and processing circuitry 1602. The radio front-end circuitry 1618 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 1618 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 1620 and / or amplifiers 1622. The radio signal may then be transmitted via the antenna 1610. Similarly, when receiving data, the antenna 1610 may collect radio signals which are then converted into digital data by the radio front-end circuitry 1618. The digital data may be passed to the processing circuitry 1602. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0231] In certain alternative embodiments, the network node 1600 does not include separate radio front-end circuitry 1618, instead, the processing circuitry 1602 includes radio front-end circuitry and is connected to the antenna 1610. Similarly, in some embodiments, all or some of the RF transceiver circuitry 1612 is part of the communication interface 1606. In still other embodiments, the communication interface 1606 includes one or more ports or terminals 1616, the radio frontend circuitry 1618, and the RF transceiver circuitry 1612, as part of a radio unit (not shown), and the communication interface 1606 communicates with the baseband processing circuitry 1614, which is part of a digital unit (not shown).

[0232] The antenna 1610 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. The antenna 1610 may be coupled to the radio front-end circuitry 1618 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, the antenna 1610 is separate from the network node 1600 and connectable to the network node 1600 through an interface or port.

[0233] The antenna 1610, communication interface 1606, and / or the processing circuitry 1602 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a UE, another network node and / or any other network equipment. Similarly, the antenna 1610, the communication interface 1606, and / or the processing circuitry 1602 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.

[0234] The power source 1608 provides power to the various components of network node 1600 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 1608 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 1600 with power for performing the functionality described herein. For example, the network node 1600 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 1608. As a further example, the power source 1608 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0235] Embodiments of the network node 1600 may include additional components beyond those shown in FIGURE 17 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, the network node 1600 may include user interface equipment to allow input of information into the network node 1600 and to allow output of information from the network node 1600. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 1600.

[0236] FIGURE 18 is a block diagram illustrating a virtualization environment 1700 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1700 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized.

[0237] Applications 1702 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment Q400 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0238] Hardware 1704 includes processing circuitry, memory that stores software and / or instructions executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1706 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1708a and 1708b (one or more of which may be generally referred to as VMs 1708), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 1706 may present a virtual operating platform that appears like networking hardware to the VMs 1708.

[0239] The VMs 1708 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1706. Different embodiments of the instance of a virtual appliance 1702 may be implemented on one or more of VMs 1708, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[0240] In the context of NFV, a VM 1708 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each of the VMs 1708, and that part of hardware 1704 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1708 on top of the hardware 1704 and corresponds to the application 1702.

[0241] Hardware 1704 may be implemented in a standalone network node with generic or specific components. Hardware 1704 may implement some functions via virtualization. Alternatively, hardware 1704 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1710, which, among others, oversees lifecycle management of applications 1702. In some embodiments, hardware 1704 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 1712 which may alternatively be used for communication between hardware nodes and radio units.

[0242] Although the computing devices described herein (e.g., UEs, network nodes, hosts) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and / or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

[0243] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionalities may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer-readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.

[0244] EXAMPLE EMBODIMENTS

[0245] Group A Embodiments

[0246] Example Embodiment 1. A method performed by a wireless device for performing positioning, the method comprising: performing AI / ML based positioning for the wireless device; transmitting model inference output of the AI / ML based positioning to a model monitoring node; and receiving feedback from the model monitoring node regarding an accuracy of the AI / ML based positioning based on a measurement of an available LOS link at the model monitoring node.

[0247] Example Embodiment 2. The method of the previous embodiment, wherein a number of available LOS links is less than a number of LOS links needed to perform non-AI / ML based positioning.

[0248] Example Embodiment 3. The method of any one of the previous embodiments, wherein the model monitoring node comprises an LMF or a base station.

[0249] Example Embodiment 4. The method of any one of the previous embodiments, when, based on the received feedback, an accuracy of the AI / ML based positioning is determined to be below a threshold value, and the method further comprises performing one or more of the following actions: deactivating the AI / ML based positioning; switching to a different AI / ML based positioning model; transmitting an error message; and trigger retraining of the AI / ML based positioning.

[0250] Example Embodiment 5. A method performed by a wireless device, the method comprising: any of the wireless device steps, features, or functions described above, either alone or in combination with other steps, features, or functions described above.

[0251] Example Embodiment 6. The method of the previous embodiment, further comprising one or more additional wireless device steps, features or functions described above.

[0252] Example Embodiment 7. The method of any of the previous two embodiments, further comprising: providing user data; and forwarding the user data to a host computer via the transmission to the base station.

[0253] Group B Embodiments

[0254] Example Embodiment 8. A method performed by a base station for performing positioning for a wireless device, the method comprising: performing AI / ML based positioning for the wireless device; transmitting model inference output of the AI / ML based positioning to a model monitoring node; and receiving feedback from the model monitoring node regarding an accuracy of the AI / ML based positioning based on a measurement of an available LOS link at the model monitoring node.

[0255] Example Embodiment 9. The method of the previous embodiment, wherein a number of available LOS links is less than a number of LOS links needed to perform non-AI / ML based positioning. Example Embodiment 10. The method of any one of the previous two embodiments, wherein the model monitoring node comprises an LMF or a base station.

[0256] Example Embodiment 11. The method of any one of the previous three embodiments, when, based on the received feedback, an accuracy of the AI / ML based positioning is below a threshold value, and the method further comprises performing one or more of the following actions: deactivating the AEML based positioning; switching to a different AI / ML based positioning model; transmitting an error message; and trigger retraining of the AI / ML based positioning.

[0257] Example Embodiment 12. A method performed by a base station, the method comprising: any of the steps, features, or functions described above with respect to base stations, either alone or in combination with other steps, features, or functions described above.

[0258] Example Embodiment 13. The method of the previous embodiment, further comprising one or more additional base station steps, features or functions described above.

[0259] Example Embodiment 14. The method of any of the previous embodiments, further comprising: obtaining user data; and forwarding the user data to a host computer or a wireless device.

[0260] Group C Embodiments

[0261] Example Embodiment 15. A mobile terminal comprising: processing circuitry configured to perform any of the steps of any of the Group A embodiments; and power supply circuitry configured to supply power to the wireless device.

[0262] Example Embodiment 16. A base station comprising: processing circuitry configured to perform any of the steps of any of the Group B embodiments; power supply circuitry configured to supply power to the wireless device.

[0263] Example Embodiment 17. A user equipment (UE) comprising: an antenna configured to send and receive wireless signals; radio front-end circuitry connected to the antenna and to processing circuitry, and configured to condition signals communicated between the antenna and the processing circuitry; the processing circuitry being configured to perform any of the steps of any of the Group A embodiments; an input interface connected to the processing circuitry and configured to allow input of information into the UE to be processed by the processing circuitry; an output interface connected to the processing circuitry and configured to output information from the UE that has been processed by the processing circuitry; and a battery connected to the processing circuitry and configured to supply power to the UE. Example Embodiment 18. A communication system including a host computer comprising: processing circuitry configured to provide user data; and a communication interface configured to forward the user data to a cellular network for transmission to a user equipment (UE), wherein the cellular network comprises a base station having a radio interface and processing circuitry, the base station’s processing circuitry configured to perform any of the steps of any of the Group B embodiments.

[0264] Example Embodiment 19. The communication system of the pervious embodiment further including the base station.

[0265] Example Embodiment 20. The communication system of the previous 2 embodiments, further including the UE, wherein the UE is configured to communicate with the base station.

[0266] Example Embodiment 21. The communication system of the previous 3 embodiments, wherein: the processing circuitry of the host computer is configured to execute a host application, thereby providing the user data; and the UE comprises processing circuitry configured to execute a client application associated with the host application.

[0267] Example Embodiment 22. A method implemented in a communication system including a host computer, a base station and a user equipment (UE), the method comprising: at the host computer, providing user data; and at the host computer, initiating a transmission carrying the user data to the UE via a cellular network comprising the base station, wherein the base station performs any of the steps of any of the Group B embodiments.

[0268] Example Embodiment 23. The method of the previous embodiment, further comprising, at the base station, transmitting the user data.

[0269] Example Embodiment 24. The method of the previous 2 embodiments, wherein the user data is provided at the host computer by executing a host application, the method further comprising, at the UE, executing a client application associated with the host application.

[0270] Example Embodiment 25. A user equipment (UE) configured to communicate with a base station, the UE comprising a radio interface and processing circuitry configured to performs any of the previous 3 embodiments.

[0271] Example Embodiment 26. A communication system including a host computer comprising: processing circuitry configured to provide user data; and a communication interface configured to forward user data to a cellular network for transmission to a user equipment (UE), wherein the UE comprises a radio interface and processing circuitry, the UE’s components configured to perform any of the steps of any of the Group A embodiments.

[0272] Example Embodiment 27. The communication system of the previous embodiment, wherein the cellular network further includes a base station configured to communicate with the UE.

[0273] Example Embodiment 28. The communication system of the previous 2 embodiments, wherein: the processing circuitry of the host computer is configured to execute a host application, thereby providing the user data; and the UE’s processing circuitry is configured to execute a client application associated with the host application.

[0274] Example Embodiment 29. A method implemented in a communication system including a host computer, a base station and a user equipment (UE), the method comprising: at the host computer, providing user data; and at the host computer, initiating a transmission carrying the user data to the UE via a cellular network comprising the base station, wherein the UE performs any of the steps of any of the Group A embodiments.

[0275] Example Embodiment 30. The method of the previous embodiment, further comprising at the UE, receiving the user data from the base station.

[0276] Example Embodiment 31. A communication system including a host computer comprising: communication interface configured to receive user data originating from a transmission from a user equipment (UE) to a base station, wherein the UE comprises a radio interface and processing circuitry, the UE’s processing circuitry configured to perform any of the steps of any of the Group A embodiments.

[0277] Example Embodiment 32. The communication system of the previous embodiment, further including the UE.

[0278] Example Embodiment 33. The communication system of the previous 2 embodiments, further including the base station, wherein the base station comprises a radio interface configured to communicate with the UE and a communication interface configured to forward to the host computer the user data carried by a transmission from the UE to the base station.

[0279] Example Embodiment 34. The communication system of the previous 3 embodiments, wherein: the processing circuitry of the host computer is configured to execute a host application; and the UE’s processing circuitry is configured to execute a client application associated with the host application, thereby providing the user data.

[0280] Example Embodiment 35. The communication system of the previous 4 embodiments, wherein: the processing circuitry of the host computer is configured to execute a host application, thereby providing request data; and the UE’s processing circuitry is configured to execute a client application associated with the host application, thereby providing the user data in response to the request data. Example Embodiment 36. A method implemented in a communication system including a host computer, a base station and a user equipment (UE), the method comprising: at the host computer, receiving user data transmitted to the base station from the UE, wherein the UE performs any of the steps of any of the Group A embodiments.

[0281] Example Embodiment 37. The method of the previous embodiment, further comprising, at the UE, providing the user data to the base station.

[0282] Example Embodiment 38. The method of the previous 2 embodiments, further comprising: at the UE, executing a client application, thereby providing the user data to be transmitted; and at the host computer, executing a host application associated with the client application.

[0283] Example Embodiment 39. The method of the previous 3 embodiments, further comprising: at the UE, executing a client application; and at the UE, receiving input data to the client application, the input data being provided at the host computer by executing a host application associated with the client application, wherein the user data to be transmitted is provided by the client application in response to the input data.

[0284] Example Embodiment 40. A communication system including a host computer comprising a communication interface configured to receive user data originating from a transmission from a user equipment (UE) to a base station, wherein the base station comprises a radio interface and processing circuitry, the base station’s processing circuitry configured to perform any of the steps of any of the Group B embodiments.

[0285] Example Embodiment 41. The communication system of the previous embodiment further including the base station.

[0286] Example Embodiment 42. The communication system of the previous 2 embodiments, further including the UE, wherein the UE is configured to communicate with the base station.

[0287] Example Embodiment 43. The communication system of the previous 3 embodiments, wherein: the processing circuitry of the host computer is configured to execute a host application; the UE is configured to execute a client application associated with the host application, thereby providing the user data to be received by the host computer.

[0288] Example Embodiment 44. A method implemented in a communication system including a host computer, a base station and a user equipment (UE), the method comprising: at the host computer, receiving, from the base station, user data originating from a transmission which the base station has received from the UE, wherein the UE performs any of the steps of any of the Group A embodiments.

[0289] Example Embodiment 45. The method of the previous embodiment, further comprising at the base station, receiving the user data from the UE.

[0290] Example Embodiment 46. The method of the previous 2 embodiments, further comprising at the base station, initiating a transmission of the received user data to the host computer.

Claims

CLAIMS1. A method (1200) performed by a first radio node (102, 202, 302, 502, 602, 702, 802, 902, 1002) for performing inference for positioning, the method comprising: performing (1202) Artificial Intelligence / Machine Learning, AI / ML. based inference for positioning for a wireless device (1212); transmitting (1204), to a second radio node (104, 204, 304, 504, 604, 704, 804, 904, 1004) operating as a model monitoring node, model inference output of the AI / ML based inference for positioning; and receiving (1206), from the second radio node, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning.

2. The method of Claim 1, wherein performing the AI / ML based inference for positioning for the wireless device comprises using an AI / ML model to generate the model inference output.

3. The method of any one of Claims 1 to 2, wherein a number of available LOS links at the first radio node is less than a number of LOS links needed to perform the non-AI / ML based inference for positioning.

4. The method of any one of Claims 1 to 3, wherein the model inference output comprises a position of the wireless device.

5. The method of any one of Claims 1 to 3, wherein the model inference output comprises positioning related information for determining by the second radio node or another node a position of the wireless device.

6. The method of any one of Claims 1 to 5, comprising transmitting model monitoring assistance information to the second radio node.

7. The method of Claim 6, wherein the assistance information comprises at least one LOS / Non-Line of Sight, NLOS, indicator.

8. The method of Claim 7, wherein the at least one LOS / NLOS indicator is associated with a sequence of LOS / NLOS indicator values, wherein each value is associated with a respective one of a plurality of links.

9. The method of any one of Claims 6 to 8, wherein the assistance information comprises: a classification of each of at least one Line of Sight, LOS, / Non-Line of Sight, NLOS, links as LOS or NLOS, andpositioning related information for each of the at least one LOS / NLOS links.

10. The method of any one of Claims 1 to 9, wherein: the first radio node is the wireless device, and the second radio node is a gNodeB, gNB, or a Location Management Function, LMF.

11. The method of any one of Claims 1 to 9, wherein: the first radio node is a gNodeB, and the second radio node is a Location Management Function, LMF.

12. The method of any one of Claims 1 to 9, wherein: the first radio node is a Location Management Function, LMF, and the second radio node is a User Equipment, UE, or gNodeB, gNB.

13. The method of any one of Claims 1 to 12, wherein the feedback comprises an indication of at least one of: an error message; an accuracy of an AI / ML model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is inaccurate; a level of reliability of the AI / ML model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is unreliable; the AI / ML model used for the AI / ML based inference for positioning is to be deactivated; the AI / ML model used for the AI / ML based inference for positioning is to be retrained; and subsequent AI / ML based inference for positioning is to be performed by a different AI / ML model.

14. The method of Claim 13, comprising performing at least one of: deactivating the AI / ML model used for the AI / ML based inference for positioning; switching to the different AI / ML model for subsequent performance of AI / ML based inference for positioning; transmitting an error message indicating that the first radio node cannot perform AI / ML based inference for positioning; transmitting a confidence indicator to a third radio node; and retraining the AI / ML model used for performing the AI / ML based inference forpositioning.

15. A method (1300) performed by a second radio node (104, 204, 304, 504, 604, 704, 804, 904, 1004) operating as a model monitoring node, the method comprising: receiving (1302), from a first radio node (102, 202, 302, 502, 602, 702, 802, 902, 1002), model inference output of an Artificial Intelligence / Machine Learning, AI / ML, based inference for positioning for a wireless device; and transmitting (1304), to the first radio node, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning for the wireless device.

16. The method of Claim 15, wherein at least one of: the first radio node is the wireless device, and the second radio node is a gNodeB, gNB, or a Location Management Function, LMF.

17. The method Claim 15, wherein: the first radio node is a gNodeB, and the second radio node is a Location Management Function, LMF.

18. The method of Claim 15, wherein: the first radio node is a Location Management Function, LMF, and the second radio node is a User Equipment, UE, or gNodeB, gNB.

19. The method of any one of Claims 15 to 18, wherein the model inference output comprises a position of the wireless device.

20. The method of any one of Claims 15 to 18, wherein the model inference output comprises positioning related information, and wherein the method comprises determining, by the second radio node, a position of the wireless device.

21. The method of any one of Claims 15 to 18, wherein the model inference output comprises positioning related information, and wherein the method comprises receiving, from a third radio node, a position of the wireless device.

22. The method of Claim 21, wherein the third radio node comprises at least one of a gNode B, gNB, or a User Equipment, UE.

23. The method of any one of Claims 15 to 22, comprising obtaining assistance information, anddetermining the accuracy and / or reliability of the Al / based inference for positioning based on at least the assistance information.

24. The method of Claim 23, wherein: the assistance information comprises at least one measurement of at least one available LOS link, and determining the accuracy and / or reliability of the Al / based inference for positioning comprises comparing the model inference output received from the first radio node to the at least one measurement of the at least one available LOS link at the second radio node.

25. The method of Claim 24, wherein assistance information comprises at least one LOS / Non- Line of Sight, NLOS, indicator.

26. The method of Claim 25 wherein the at least one LOS / NLOS indicator is associated with a sequence of LOS / NLOS indicator values, wherein each value is associated with a respective one of a plurality of links.

27. The method of any one of Claims 23 to 26, wherein the assistance information comprises: a classification of each of at least one Line of Sight, LOS, / Non-Line of Sight, NLOS, links as LOS or NLOS, and positioning related information for each of the at least one LOS / NLOS links.

28. The method of any one of Claims 23 to 27, wherein obtaining the assistance information comprises determining the assistance information by the second radio node.

29. The method of any one of Claims 23 to 27, wherein obtaining the assistance information comprises receiving the assistance information from the first radio node.

30. The method of any one of Claims 23 to 27, wherein obtaining the assistance information comprises receiving the assistance information from a third radio node (206, 606, 906).

31. The method of any one of Claims 15 to 30, wherein the feedback comprises an indication of at least one of: an error message; an accuracy of an AI / ML model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is inaccurate; a level of reliability of the AI / ML model used for the AI / ML based inference for positioning is below a threshold value;the AI / ML model used for the AI / ML based inference for positioning is unreliable; the AI / ML model used for the AI / ML based inference for positioning is to be deactivated; the AI / ML model used for the AI / ML based inference for positioning is to be retrained; and subsequent AI / ML based inference for positioning is to be performed by a different AI / ML model.

32. A first radio node for performing inference for positioning, the first radio node configured to: perform Artificial Intelligence / Machine Learning, AI / ML. based inference for positioning for a wireless device; transmit, to a second radio node operating as a model monitoring node, model inference output of the AI / ML based inference for positioning ; and receive, from the second radio node, feedback indicating an accuracy and / or reliability of the AI / ML based inference for positioning.

33. The first radio node of Claim 32, wherein performing the AI / ML based inference for positioning for the wireless device comprises using an AI / ML model to generate the model inference output.

34. The first radio node of any one of Claims 32 to 33 wherein at least one of: a number of available LOS links at the first radio node is less than a number of LOS links needed to perform the non-AI / ML based inference for positioning, the model inference output comprises a position of the wireless device, and the model inference output comprises positioning related information for determining by the second radio node or another node a position of the wireless device.

35. The first radio node of any one of Claims 32 to 34, comprising transmitting model monitoring assistance information to the second radio node.

36. The first radio node of Claim 35, wherein the assistance information comprises at least one of: at least one LOS / Non-Line of Sight, NLOS, indicator, the at least one LOS / NLOS indicator is associated with a sequence of LOS / NLOS indicator values, each value being associated with a respective one of a plurality of links, and a classification of each of at least one Line of Sight, LOS, / Non-Line of Sight, NLOS, links as LOS or NLOS and positioning related information for each of the at least one LOS / NLOS links.

37. The first radio node of any one of Claims 32 to 36, wherein: the first radio node is the wireless device and the second radio node is a gNodeB, gNB, or a Location Management Function, LMF, or the first radio node is a gNB and the second radio node is an LMF, or the first radio node is a LMF and the second radio node is a User Equipment, UE, or gNB.

38. The first radio node of any one of Claims 32 to 37, wherein the feedback comprises an indication of at least one of: an error message; an accuracy of an AI / ML model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is inaccurate; a level of reliability of the AI / ML model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is unreliable; the AI / ML model used for the AI / ML based inference for positioning is to be deactivated; the AI / ML model used for the AI / ML based inference for positioning is to be retrained; and subsequent AI / ML based inference for positioning is to be performed by a different AI / ML model.

39. The first radio node of Claim 38, comprising performing at least one of: deactivating the AI / ML model used for the AI / ML based inference for positioning; and switching to the different AI / ML model for subsequent performance of AI / ML based inference for positioning; transmitting an error message indicating that the first radio node cannot perform AI / ML based inference for positioning; transmitting a confidence indicator to a third radio node; and retraining the AI / ML model used for performing the AI / ML based inference for positioning.

40. A second radio node operating as a model monitoring node, the second radio node configured to: receive, from a first radio node, model inference output of an Artificial Intelligence / Machine Learning, AI / ML, based inference for positioning for a wireless device; and transmit, to the first radio node, feedback indicating an accuracy and / or reliability of theAI / ML based inference for positioning for the wireless device.

41. The second radio node of Claim 40, wherein: the first radio node is the wireless device and the second radio node is a gNodeB, gNB, or a Location Management Function, LMF, or the first radio node is a gNB and the second radio node is an LMF, or the first radio node is a LMF and the second radio node is a User Equipment, UE, or gNB.

42. The second radio node of any one of Claims 40 to 41, wherein the model inference output comprises: a position of the wireless device, or positioning related information for determining, by the second radio node, a position of the wireless device, or positioning related information a position of the wireless device is received from a third radio node that comprises at least one of a gNB or a UE.

43. The second radio node of any one of Claims 40 to 42, comprising obtaining assistance information, and determining the accuracy and / or reliability of the Al / based inference for positioning based on at least the assistance information.

44. The second radio node of Claim 43, wherein at least one of: the assistance information comprises at least one measurement of at least one available LOS link, and the accuracy and / or reliability of the Al / based inference for positioning is determined by comparing the model inference output received from the first radio node to the at least one measurement of the at least one available LOS link at the second radio node, the assistance information comprises at least one LOS / Non-Line of Sight, NLOS, indicator associated with a sequence of LOS / NLOS indicator values, each value being associated with a respective one of a plurality of links, the assistance information comprises a classification of each of at least one LOS, NLOS links as LOS or NLOS and positioning related information for each of the at least one LOS / NLOS links.

45. The second radio node of any one of Claims 40 to 44, wherein obtaining the assistance information comprises at least one of: determining the assistance information by the second radio node; obtaining the assistance information comprises receiving the assistance information fromthe first radio node; receiving the assistance information from a third radio node (206, 606, 906).

46. The second radio node of any one of Claims 40 to 45, wherein the feedback comprises an indication of at least one of: an error message; an accuracy of an AI / ML model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is inaccurate; a level of reliability of the AI / ML model used for the AI / ML based inference for positioning is below a threshold value; the AI / ML model used for the AI / ML based inference for positioning is unreliable; the AI / ML model used for the AI / ML based inference for positioning is to be deactivated; the AI / ML model used for the AI / ML based inference for positioning is to be retrained; and subsequent AI / ML based inference for positioning is to be performed by a different AI / ML model.

Images