Reporting applicability information based on performance monitoring by a user equipment

By implementing UE-based performance monitoring and network-controlled activation/deactivation of AI/ML models, the patent addresses the lack of performance monitoring in 5G networks, ensuring reliable and accurate AI/ML predictions.

WO2026235602A1PCT designated stage Publication Date: 2026-11-12TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/SE2026/050314
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-08
Filing Date
2026-05-07
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

The 3GPP standards lack performance monitoring for AI/ML-based mobility predictions in 5G networks, leading to potential inaccuracies and the need for proactive mechanisms to ensure reliable decision-making by wireless devices.

Method used

A UE monitors AI/ML model performance and sends indications of prediction accuracy to the network, allowing the network to activate or deactivate the inference configuration based on performance metrics, with the UE optionally making autonomous decisions.

Benefits of technology

Ensures accurate AI/ML predictions by enabling proactive management of inference configurations, preventing inaccurate decisions and improving network performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure SE2026050314_12112026_PF_FP_ABST
    Figure SE2026050314_12112026_PF_FP_ABST
Patent Text Reader

Abstract

Monitoring performance of artificial-intelligence / machine-learning, AI / ML, predictions performed by the wireless device operating in a wireless network. An example method, carried out by the wireless device, comprises calculating (620) at least one performance metric characterizing prediction accuracy for a plurality of measurement and / or event predictions made by one or more AI / ML models running in the wireless device and sending (630), to a network node in the wireless network, an indication of prediction accuracy for the measurement and / or event predictions, based on the at least one performance metric.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] REPORTING APPLICABILITY INFORMATION BASED ON PERFORMANCE MONITORING BY A USER EQUIPMENT

[0002] TECHNICAL FIELD

[0003] The present document is generally directed to wireless communications, and is more particularly directed to techniques for monitoring performance of artificial-intelligence / machine-learning (AI / ML) predictions performed by a wireless device operating in a wireless communications network.

[0004] BACKGROUND

[0005] The 3rd-Generation Partnership Project (3GPP) is currently developing Release 19 (Rel-19) of their industry standards for wireless communications. As part of this development, 3GPP has agreed upon a study item regarding the use of artificial intelligence / machine-learning (AI / ML) for improving mobility for a wireless device (referred to as a “user equipment” or“UE,” in 3GPP terminology) operating in a wireless network. This study item explores the benefits of augmenting mobility functionality in wireless networks with features enabling improved support of AI / ML-based algorithms for enhanced performance and / or reduced complexity / overhead in such use cases in mobility as Radio Resource Management (RRM) measurement prediction, measurement event prediction, and prediction of Radio Link Failure (RLF) and Handover Failure (HOF) events.

[0006] The goal of the study is to focus on mobility enhancements for wireless devices operating in connected mode (formally, “RRC_CONNECTED mode”), i.e. , in a mode where the wireless device has an active connection with an access node, e.g., a New Radio (NR) base station, referred to as a “gNB” in 3GPP terminology. More particularly, the study is concerned with following the existing mobility framework for mobility in RRC_CONNECTED mode, i.e., a framework in which the handover decision is always made on the network side. Mobility use cases in the study focus on standalone NR PCell (Primary Cell) change. In the study, both UE-side and network-side usage of AI / ML models can be considered.

[0007] The objectives of the study item are to study and evaluate potential benefits and gains of AI / ML -aided mobility for network-triggered Layer 3 (L3)-based handover, considering the following aspects:

[0008] • AI / ML-based Radio Resource Management (RRM) measurement and event prediction o Cell-level measurement prediction including intra and inter-frequency (UE sided and NW sided model) [RAN2]■ Inter-cell Beam-level measurement prediction for L3 Mobility (UE sided and NW sided model) [RAN2]

[0009] o HO failure / RLF prediction (UE sided model) [RAN2]

[0010] o Measurement events prediction (UE sided model) [RAN2]

[0011] • Study the need for and / or benefits of any other UE assistance information for the network side model [RAN2]

[0012] According to the study item:

[0013] • The evaluation of the AI / ML aided mobility benefits should consider HO performance KPIs (e.g., Ping-pong HO, HOF / RLF, Time of stay, Handover interruption, prediction accuracy, and measurement reduction) etc.) and complexity tradeoffs [RAN2]

[0014] • Potential Al mobility specific enhancement should be based on the Rel-19 AI / ML-air interface Work Item Description (WID) general framework (e.g., Life Cycle Management (LCM), performance monitoring, etc.) [RAN2]

[0015] o NOTE: This would only be treated after sufficient progress is made in the Rel-19 AI / ML air interface WID

[0016] • Potential specification impacts of AI / ML-aided mobility [RAN2]

[0017] • Evaluate testability, interoperability, and impacts on RRM requirements and performance [RAN4]

[0018] The following agreement has been taken during the Study Item in Rel-19 -this agreement indicates thatthe performance monitoringwill be part of the study in Rel-19and Rel-20 as well:

[0019] • The general Life Cycle Management (LCM) framework for beam management can be the baseline (where applicable) for Al mobility, such as the following aspects:

[0020] o Data collection for model training

[0021] o UE capability

[0022] o Applicability reporting

[0023] o Inference configuration and reporting

[0024] o Performance monitoring and management

[0025] RRM measurement prediction in this study item includes RRM measurement prediction in temporal domain, spatial domain and frequency domain. For temporal-domain prediction, an Al model predicts RRM measurement results for the future time instances, and / or may predict unavailable RRM measurement results for partial time instances in the past and / or present, based on available RRM results measured by the UE. For spatial-domain prediction, an Almodel predicts RRM measurement results for the beams / cells in Set A according to available RRM measurement results of the beams / cells in Set B, or predicts cell-level RRM measurement results based on RRM measurement results of one set of beams. In frequency-domain prediction, one Al model predicts RRM measurement results for the frequency(ies) in Set A according to available RRM measurement results for the frequency(ies) in Set B. Some examples for intra-frequency temporal RRM measurement prediction are shown in each of Figures 1 , 2, and 3.

[0026] SUMMARY

[0027] In various AI / ML-based solutions for RRM measurement prediction and / or event prediction, the AI / ML model or models may be deployed at the UE side or at the network side, or at both UE and network sides. If the AI / ML models are at the UE side, then the network needs to monitor the performance of the inference configuration related to the AI / ML models for functionality management purposes. For example, if the performance of the inference configuration becomes worse, the network may wish to deactivate the inference configuration. Information regarding the performance of the inference configuration may be used by the network for other purposes, as well.

[0028] However, performance monitoring for AI / ML-based mobility is not yet supported in the 3GPP standards for fifth-generation (5G) networks. If the performance of predictions produced by the AI / ML modes cannot be monitored, it is impossible to know whether the predictions are good enough to be used fortheir intended purpose. Furthermore, in time-limited inference situations, where the network depends on UE prediction, a proactive mechanism from the UE side is needed to avoid providing the network with inaccurate decisions.

[0029] Still further, performance monitoring is generally a reactive mechanism, e.g., the UE performs prediction and sends a report to the network and the network makes the decision based on the prediction and then the performance is measured by the UE. However, a prediction made by the UE and reported to the network may negatively influence the network performance if the prediction performance at the UE is not good. Hence, performance monitoring in particular for measurement and / or event prediction or for the decisions that require immediate actions based on UE reports may require some proactive decisions at the UE which is currently not possible.

[0030] Techniques, apparatuses, and systems described herein address these problems. According to several of these techniques, a UE monitors the performance of an AI / ML model (or an AI / ML functionality, or an AI / ML inference configuration) in terms of prediction accuracy or othermetrics. (Note that one AI / ML functionality may include one or more AI / ML models, and one AI / ML inference configuration may mean one inference configuration for one AI / ML functionality or for one AI / ML model.)

[0031] In some embodiments or instances, if the model performance(s) (or functionality performance or inference performance) is (are) below a certain threshold (their individual certain thresholds), the UE informs the network, and the network decides whether to deactivate the AI / ML model (or the AI / ML functionality or the AI / ML inference configuration).

[0032] In some embodiments or instances, if the model performance(s) (or functionality performance or inference performance) is (are) below a certain threshold (their individual certain thresholds), the UE autonomously deactivates the AI / ML model (orthe AI / MLfunctionality orthe AI / ML inference configuration) and notifies the network about that.

[0033] Embodiments of the presently disclosed techniques and apparatuses for monitoring performance of AI / ML predictions performed by the wireless device operating in a wireless network include methods carried out by wireless device and a network node, such as a base station. An example method, carried out by the wireless device, comprises calculating at least one performance metric characterizing prediction accuracy for a plurality of measurement and / or event predictions made by one or more AI / ML models running in the wireless device and sending, to a network node in the wireless network, an indication of prediction accuracy for the measurement and / or event predictions, based on the at least one performance metric.

[0034] An example method performed by the corresponding network device comprises receiving, from the wireless device, an indication of prediction accuracy for measurement and / or event predictions made by one or more AI / ML models running in the wireless device. This method further comprises determining whether to activate or deactivate an inference configuration in the wireless device, based on the indication.

[0035] Apparatuses corresponding to the methods summarized above, and their many variants, are described in detail below. These apparatuses include a wireless device comprising radio circuitry and processing circuitry operatively coupled to the radio circuitry, the processing circuitry being configured, e.g., with program instructions stored in memory operatively coupled to the processing circuitry, to calculate at least one performance metric characterizing prediction accuracy for a plurality of measurement and / or event predictions made by one or more AI / ML models running in the wireless device, and send, to a network node in the wirelessnetwork, an indication of prediction accuracy for the measurement and / or event predictions, based on the at least one performance metric.

[0036] Likewise, other apparatuses described herein include a network node comprising radio circuitry and processing circuitry operatively coupled to the radio circuitry, where the processing circuitry is configured, e.g., with program instructions stored in memory operatively coupled to the processing circuitry, to receive, from the wireless device, an indication of prediction accuracy for measurement and / or event predictions made by one or more AI / ML models running in the wireless device, and to determine whether to activate or deactivate an inference configuration in the wireless device, based on the indication.

[0037] Variations of these techniques and apparatuses and additional details are provided below. An advantage of various embodiments is that they provide the ability to control the performance of an AI / ML model inference configuration in the UE. This guarantees that predictions sent to the network node are accurate enough and can be trusted. When the predictions are accurate enough, they can be used for the intended purpose.

[0038] BRIEF DESCRIPTION OFTHE FIGURES

[0039] Figure 1 , Figure 2, and Figure 3 each illustrate an example of intra-frequency temporal RRM measurement predictions.

[0040] Figure 4 illustrates an example flow of operations and signaling according to some embodiments.

[0041] Figure 5 an alternative or complementary flow of operations and signaling, according to some embodiments.

[0042] Figure 6 illustrates an example method as carried out by a wireless device, according to some embodiments.

[0043] Figure 7 illustrates an example method as carried out be a network node, according to some embodiments.

[0044] Figure 8 shows a communication system according to various embodiments of the present disclosure.

[0045] Figure 9 shows a UE according to various embodiments of the present disclosure.

[0046] Figure 10 shows a network node according to various embodiments of the present disclosure.Figure 11 shows a host computing system according to various embodiments of the present disclosure.

[0047] Figure 12 is a block diagram of a virtualization environment in which functions implemented by some embodiments of the present disclosure may be virtualized.

[0048] Figure 13 illustrates communication between a host computing system, a network node, and a UE via multiple connections, at least one of which is wireless, according to various embodiments of the present disclosure.

[0049] DETAILED DESCRIPTION

[0050] The present disclosure involves the prediction of various RRM measurements, based on actual measurements taken at different times and / or at different frequencies. Events related to these measurements, such as Radio Link Failure (RLF) or Handover Failure (HOF), may also be predicted, in various embodiments. Various techniques for performing these predictions may be used, with many of these techniques falling within the domain of Machine Learning and Artificial Intelligence, which terms are used interchangeably herein to refer to computerized systems that are able to learn and adapt, without following explicit instructions, by using algorithms and statistical models to analyze and draw inferences from patterns in data. The inferences in the present context are the predicted measurements and / or events.

[0051] Thus, the predictions referred to herein may be Artificial intelligence (Al) / Machine Learning (ML) based predictions, namely the results of the inference of an AI / ML engine, that is performed based on measurements and / or some additional information or any non-AI / ML based predictions, and reported to the network. However, prediction techniques not based on AI / ML models may be used, in some embodiments or instances of the claimed techniques. Thus, references herein to “AI / ML model” may be replaced by “non-AI / ML model”, and “AI / ML functionality” may be replaced by“non-AI / MLfunctionality”, and “AI / ML inference configuration” may be replaced by “non-AI / ML inference configuration,” when considering those possible embodiments.

[0052] The terms “configuration” and “configuration information” are used herein to refer to information provided to a wireless device, by the wireless network, where that information defines elements and / or parameters of a task (or repeated tasks) to be carried out by the wireless devices and / or operational modes to be used by the wireless device. The configurations and configuration information described herein may be provided in formalized configuration messages, which may be modifications of various data structures defined by3GPP standards for such things as measurement configurations and measurement reporting configurations.

[0053] Techniques for addressing the problems discussed above include methods, performed by a User Equipment (UE), in which the UE receives an inference configuration based on which the UE performs and reports measurement prediction(s) and / or event predictions of one or more cells of a measurement object (measObject). The UE additionally receives a performance monitoring configuration based on which the UE calculates the AI / ML model performance during running time of the inference configuration, in terms of prediction accuracy or other metrics (e.g., F1 score, MSE (Mean Squared Error), etc.). This is shown in Figure 4, in the “calculate performance monitoring” box.

[0054] In one option, if the performance monitoring indicates bad performance of the AI / ML model (e.g., the prediction accuracy is lower than a certain configured threshold), the UE sends to the network (NW) an inapplicability report (e.g., in the RRC UEAssistancelnformation message). This is also shown in Figure 4. The “inapplicability report” may comprise, for example, simply an indication that the prediction accuracy or other performance metric calculated by the UE has fallen below a threshold. In one option, an event is triggered if the prediction accuracy is below a certain threshold and a message is sent to the network, e.g. in a MeasurementReport.

[0055] Based on the report, the network makes a decision on deactivating the inference configuration (or AI / ML model) and commands the UE to deactivate the inference (e.g., using a RRCReconfiguration message). These operations are shown in Figure 4, with the latter being labeled a “LCM [Life Cycle Management] operation indication. It will be appreciated, of course, that this indication may carry different names, in various implementations.

[0056] In some embodiments or instances, even though the inference is deactivated, the UE can continue monitoring the performance of the inference configuration (or AI / ML model performance). If the performance monitoring indicates the performance is good again, the UE can send an applicability report to NW and the network can activate the inference again. In another case, if the performance monitoring indicates the performance is good again, the UE can send one indication to NW about the performance of the deactivated inference configuration meeting a configured condition and the network may activate the inference again.

[0057] In another option, based on the performance monitoring, the UE deactivates the inference autonomously. The UE notifies the NW that the inference is deactivated via an inapplicability report (e.g., in the RRC UEAssistancelnformation message). The inapplicability report mayinclude an additional cause, indicating the reason why the inference has been deactivated. This approach is shown generally in Figure 5, which shows the calculating of the performance monitoring metric, an autonomous decision to activate / deactivate, and the sending of an indication of whether prediction is applicable or inapplicable, i.e., an indication that the UE has activated or deactivated an AI / ML model, an inference configuration, etc.

[0058] In some embodiments or instances where the UE autonomously deactivates an inference configuration, the UE does not remove the inference configuration received by the network but continues monitoring the AI / ML model performance. When the performance is good enough, the UE can autonomously activate the inference and notify the network by sending an applicability report including a cause. In one option, an event is triggered if the prediction accuracy is above a certain threshold and a message is sent to the network, e.g., in a MeasurementReport.

[0059] UE implementations

[0060] Embodiments of the presently disclosed techniques include methods carried out by a UE (or wireless device) configured with an inference configuration as well as with a performance monitoring configuration that is associated with the inference configuration. Examples of such methods may comprise one, several, or all of the steps of:

[0061] • Receiving from a NW node a configuration comprising performance monitoring parameters.

[0062] • Monitoring the performance of the inference configuration while performing predictions • Sending an inapplicability report to the NW when the performance is below a first threshold and / or

[0063] • Sending an applicability report to the NW when the performance is above a second threshold.

[0064] In some embodiments or instances, the performance monitoring is done by calculating a prediction accuracy (e.g., the difference between the predicted cell quality and the actual measured cell quality such as e.g. the RSRP difference) or other metric or metrics (e.g., F1 score, MSE (Mean Squared Error), etc.).

[0065] In some embodiments or instances, the performance configuration may include an indication of whether the UE can autonomously deactivate / activate the AI / ML model (or the inference configuration). This indication may be an explicit indication or may be implicitly indicated, e.g.by the inclusion of a configuration to apply when the model (orthe inference configuration) is deactivated.

[0066] In some embodiments or instances, the inference configuration may include the configuration to use when the UE autonomously deactivates the AI / ML model (orthe inference configuration). The configuration to use may be defined as a pre-configured RRC message, e.g., an RRCReconfigu ration message which the UE applies when the AI / ML model (orthe inference configuration) is deactivated.

[0067] In some embodiments or instances, when the performance monitoring indicates bad performance, e.g., as indicated by the calculated prediction accuracy or other performance metric falling below a threshold value, the UE sends an inapplicability report or other indication of poor performance to the network (e.g., included in a RRC UEAssistancelnformation IE).

[0068] In some embodiments or instances, the bad performance triggers an event, the event indicating that the prediction quality is worse than a threshold. The event can be sent in a MeasurementReport, for example. In some of these instances or embodiments, the UE may then receive from the network, in response, a command to deactivate the AI / ML model (or the inference configuration) (e.g., in a RRCReconfiguration message or in a MAC CE (Control Element)).

[0069] In other embodiments or instances, when the performance monitoring indicates bad performance, the UE may autonomously deactivate the AI / ML model (orthe inference configuration), e.g. by applying an alternative configuration to use when the model (or the inference configuration) is deactivated, as described above, and sends an inapplicability report (or other indication of the deactivation and / or change in inference configuration) to the network (e.g., included in a RRC UEAssistancelnformation IE).

[0070] In some embodiments or instances, the UE, even when the AI / ML model is deactivated, keeps the inference configuration as received by the network.

[0071] In some embodiments or instances, the inapplicability report or other indication of poor performance of predictions sent to the network includes a cause indicating a reason for the inapplicability or poor performance.

[0072] In some embodiments or instances, when the performance monitoring indicates good performance, e.g., as indicated by the predicted accuracy or other performance metriccalculated by the UE exceeding a threshold value, the UE sends an applicability report or other indication of good prediction performance to the network (e.g., included in an RRC UEAssistancelnformation IE).

[0073] In some embodiments or instances, the good performance triggers an event, the event indicating that the prediction quality is better than a threshold. The event can be sent in a MeasurementReport, for example. In some embodiments or instances, the UE may receive from the network, in response a command to activate the AI / ML model (or the inference configuration) (e.g., in an RRCReconfiguration message).

[0074] In some embodiments or instances, the good performance may apply to a deactivated inference configuration forwhich the UE is still monitoring performance. In some such embodiments or instances, the UE may apply an inference configuration when the performance is good again. This may comprise applying a pre-configured configuration which the UE has previously received.

[0075] In some embodiments or instances, when the performance monitoring indicates good performance, the UE may autonomously activate an AI / ML model (orthe inference configuration), e.g., a previously deactivated AI / ML model (or previously deactivated inference configuration) and send an applicability report to the network (e.g., included in an RRC UEAssistancelnformation IE).

[0076] In various embodiments or instances, “good performance” means a performance metric calculated by the UE is above a threshold or below a threshold, wherein the threshold is configured by the network or is pre-defined e.g., based on some performance metric requirements.

[0077] In various embodiments or instances, an applicability report or other indication of good performance sent to the network may include a cause indicating the reason for the applicability.

[0078] Likewise, when sending an indication of good performance or bad performance, when the UE sends to the network a reason (cause) for declaring applicability or inapplicability, it may do so as an extension to the applica bility / inapplicability message, e.g., by sending an indication that inference error is high (low), or that confidence is low (high), or that inference model input RSRP below (above) a detectable threshold, etc.In various embodiments or instances, where the network has configured the UE with multiple inference configurations (either for frequency, spatial or temporal prediction), upon evaluating each inference configuration, the UE may send, to the network, a report with a set of elements each indicating applicability or inapplicability of the corresponding inference configuration.

[0079] Network-side implementations

[0080] Embodiments of the presently disclosed techniques include methods carried out by a network node (or group of network nodes) operating in a wireless network, where the network node configures a UE with an inference configuration and a performance monitoring configuration. The methods may comprise, in various embodiments or instances, one or more of:

[0081] • Sending, to the UE, a configuration comprising performance monitoring parameters, such as a threshold value for prediction accuracy to be used by the UE for evaluating prediction performance;

[0082] • Receiving an inapplicability report or other indication of poor prediction performance from the UE when the performance of the inference configuration (or the AI / ML model performance) is below a certain threshold; and

[0083] • Receiving an applicability report or other indication of good prediction performance from the UE when the performance of the inference configuration (or the AI / ML model performance) is above a certain threshold.

[0084] In some embodiments or instances, the network configures the UE as to whether autonomously deactivating / activating the AI / ML model (or the inference configuration) is allowed or not, in the performance monitoring configuration. In some embodiments or instances, the network includes another inference configuration to use when the UE autonomously deactivates the AI / ML model (or the inference configuration)

[0085] In some embodiments or instances, the network decides to deactivate the AI / ML model (or the inference configuration) if an inapplicability report is received, and sends to the UE a command to deactivate the AI / ML model (or the inference configuration) (e.g., in an RRCReconfigu ration message). Similarly, the network may decide to deactivate the AI / ML model (or the inference configuration) if it receives an event indicating that the prediction quality is below a threshold.

[0086] In some embodiments or instances, the network may decide to activate the AI / ML model (or the inference configuration) if an applicability report is received, and send to the UE a command to activate the AI / ML model (or the inference configuration) (e.g. in the RRCReconfigu ration message). This may be a previously deactivated AI / ML model or inference configuration, insome embodiments or instances. Similarly, the network may decide to activate an AI / ML model (or the inference configuration) if it receives an event indicating that the prediction quality is higher a threshold.

[0087] In some embodiments or instances, if the (i n)a lica bility report received from the UE includes a cause indicatingwhy the model (orthe inference configuration) has been deactivated / activated by the UE), the network node receiving the report stores the cause. In some embodiments or instances, the network node may then reconfigure the UE with a new configuration with the aim of making the model (orthe prediction) applicable.

[0088] Figure 6 is a process flow diagram illustrating an example method, as carried out by a wireless device operating in a wireless network, for supporting performance monitoring of artificial-intelligence / machine-learning (AI / ML) predictions performed by the wireless device. As shown at block 620, the method comprises calculating at least one performance metric characterizing prediction accuracy for a plurality of measurement and / or event predictions made by one or more AI / ML models running in the wireless device. As shown at block 630, the method further comprises sending, to a network node in the wireless network, an indication of prediction accuracy for the measurement and / or event predictions, based on the at least one performance metric. This indication may be, for example, an inapplicability or applicability report as discussed above.

[0089] Note that the illustrated method is intended to represent a generalization of the various UE-based techniques described in detail above. Accordingly, where there are minor differences between the terminology used to describe the method in Figure 6 and the terminology used in the detailed examples above, it should be understood that the terminology used to describe Figure 6 is intended to at least encompass the similar or clearly related terminology used above, in such a way that the steps and details described for Figure 6 can be mapped to corresponding steps and details in the examples provided above.

[0090] In some instances or embodiments of the method shown in Figure 6, the indication of prediction accuracy comprises an indication that the performance metric or other measure of prediction accuracy for the measurement and / or event predictions has fallen below a threshold value. In some of embodiments or instances, the method may further comprise receiving, from the network node, an instruction to deactivate an AI / ML inference configuration previously activated for the wireless device, in response to sending the indication of prediction accuracy. This is shown as one of the alternatives in block 640.In some embodiments or instances, the method further comprises autonomously deactivating an AI / ML inference configuration previously provided to the wireless device, based on the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions having fallen below the threshold value. This is shown as another alternative in block 640.

[0091] In some embodiments or instances, the indication of prediction accuracy comprises an indication that the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions has risen above a threshold value. In some of these and in some other instances, the method may further comprise receiving, from the network node, an instruction to activate an AI / ML inference configuration previously provided to the wireless device, in response to sending the indication of prediction accuracy. Again, this is shown as an alternative in block 640.

[0092] Likewise, in other embodiments or instances, the method may further comprise autonomously activating an AI / ML inference configuration previously provided to the wireless device, based on the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions having risen above the threshold value. In some of these embodiments or instances, the AI / ML inference configuration previously provided to the wireless device may have been deactivated, prior to said sending of the indication of prediction accuracy for the measurement and / or event predictions, in response to a previous report of prediction accuracy for the AI / ML inference configuration.

[0093] In various embodiments or instances, the method may further comprise receiving, prior to said calculating, an inference configuration comprising one or more performance monitoring parameters for monitoring performance of the one or more AI / ML models. This is shown at block 610 in Figure 6. In some of these embodiments or instances, the inference configuration comprises a threshold parameter indicative of a threshold performance for predictive accuracy of the one or more AI / ML models. Note that the “inference configuration” referred to here may be labeled an “inference configuration” or it may be labeled differently, e.g., referred to as a performance monitoring configuration, or it may form part of a measurement configuration, in various embodiments or instances. In some embodiments or instances, the inference configuration may comprise an indication of whether the wireless device is permitted to autonomously deactivate and / or active the inference configuration.Figure 7 is another process flow diagram, this time illustrating an example method, as carried out by a network node in a wireless network, for monitoring performance of artificial-intelligence / machine-learning (AI / ML) predictions performed by a wireless device operating in the wireless network. As shown at block 720, this method comprises receiving, from the wireless device, an indication of prediction accuracy for measurement and / or event predictions made by one or more AI / ML models running in the wireless device. As shown at block 730, the method further comprises determining whether to activate or deactivate an inference configuration in the wireless device, based on the indication.

[0094] In the same manner described above for Figure 6, the method shown in Figure 7 is intended to represent a generalization of the various network-side techniques described in detail above. Accordingly, where there are minor differences between the terminology used to describe the method in Figure 7 and the terminology used in the detailed examples above, it should be understood that the terminology used to describe Figure 7 is intended to at least encompass the similar or clearly related terminology used above, in such a way that the steps and details described for Figure 7 can be mapped to corresponding steps and details in the examples provided above.

[0095] In some embodiments or instances of the illustrated method, the indication of prediction accuracy comprises an indication that a performance metric or other measure of prediction accuracy for the measurement and / or event predictions has fallen below a threshold value, and wherein the method comprises sending, to the wireless device, an instruction to deactivate the inference configuration, in response to receiving the indication. In some embodiments or instances, the indication of prediction accuracy comprises an indication that a performance metric or other measure of prediction accuracy for the measurement and / or event predictions has risen above a threshold value, and wherein the method comprises sending, to the wireless device, an instruction to activate the inference configuration, in response to receiving the indication. Note that the “indication of prediction accuracy” referred to here may take the form (or name) of the applicability report or inapplicability report discussed above.

[0096] In some embodiments or instances of the method shown in Figure 7, the method further comprises sending to the wireless device, prior to the receiving step shown in block 720, an inference configuration comprising one or more performance monitoring parameters for monitoring performance of the one or more AI / ML models. This is shown at block 710 in the figure. Again, the “inference configuration” referred to here may be labeled an “inference configuration” or it may be labeled differently, e.g., referred to as a performance monitoringconfiguration, or it may form part of a measurement configuration, in various embodiments or instances. In some embodiments or instances, the inference configuration comprises a threshold parameter indicative of a threshold performance for predictive accuracy of the one or more AI / ML models. In some embodiments or instances, the inference configuration may comprise an indication of whether the wireless device is permitted to autonomously deactivate and / or active the inference configuration. In some embodiments or instances, the inference configuration may comprise an indication of a second inference configuration for the wireless device to use in the event that the wireless device autonomously deactivates the inference configuration.

[0097] Embodiments of the presently disclosed techniques include apparatuses, such as UE apparatuses and gNB apparatuses, configured to carryout any of the methods described above. Below is a detailed description of an example wireless system and several of its nodes - it should be understood that the method described above may be implemented in one or more of these nodes, in various embodiments and instances.

[0098] Figure 8 shows an example of a communication system 800 in accordance with some embodiments. This provides a context for the techniques described herein, which can be implemented by devices operating in such a communication system 800. In this example, communication system 800 includes telecommunication network 802 that includes access network 804 (e.g., RAN) and a core network 806, which includes one or more core network nodes 808. Access network 804 includes one or more access network nodes, such as network nodes 81 Oa-b (one or more of which may be generally referred to as network nodes 810), or any other similar 3GPP access node or non-3GPP access point. Network nodes 810 facilitate direct or indirect connection of UEs, such as by connecting UEs 812a-d (one or more of which may be generally referred to as UEs 812) to core network 806 over one or more wireless connections.

[0099] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, communication system 800 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired orwireless connections. Communication system 800 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.UEs 812 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with network nodes 810 and other communication devices. Similarly, network nodes 810 are arranged, capable, configured, and / or operable to communicate directly or indirectly with UEs 812 and / or with other network nodes or equipment in telecommunication network 802 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in telecommunication network 802.

[0100] In the depicted example, core network 806 connects network nodes 810 to one or more hosts, such as host 816. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. Core network 806 includes one or more core network nodes (e.g., 808) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of core network node 808. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0101] Host 816 may be under the ownership or control of a service provider other than an operator or provider of access network 804 and / or telecommunication network 802, and may be operated by the service provider or on behalf of the service provider. Host 816 may host a variety of applications to provide one or more service. Examples of such applications include live and prerecorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0102] As a whole, communication system 800 of Figure 8 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); LongTerm Evolution (LTE), and / or other suitable 2G, 3G,4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0103] In some examples, telecommunication network 802 is a cellular network that implements 3GPP standardized features. Accordingly, telecommunication network 802 may support network slicing to provide different logical networks to different devices that are connected to telecommunication network 802. For example, telecommunication network 802 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC) / Massive loT services to yet further UEs.

[0104] In some examples, UEs 812 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to access network 804 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from access network 804. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with anyone orcombination of Wi-Fi, NR (New Radio) and LTE, i.e., being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

[0105] In the example, hub 814 communicates with access network 804 to facilitate indirect communication between one or more UEs (e.g., UE 812c and / or 812d) and network nodes (e.g., network node 810b). In some examples, hub 814 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, hub 814 may be a broadband router enabling access to core network 806 for the UEs. As another example, hub 814 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 810, or by executable code, script, process, or other instructions in hub 814. As another example, hub 814 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, hub 814 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, hub 814 may retrieve VR assets, video, audio, orother media or data related to sensory information via a network node, which hub 814 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, hub 814 acts as a proxy server or orchestrator for the UEs, in particular in if one or more of the UEs are low energy loT devices.

[0106] Hub 814 may have a constant / persistent or intermittent connection to network node 810b. Hub 814 may also allow for a different communication scheme and / or schedule between hub 814 and UEs (e.g., UE 812c and / or 812d), and between hub 814 and core network 806. In other examples, hub 814 is connected to core network 806 and / or one or more UEs via a wired connection. Moreover, hub 814 may be configured to connect to an M2M service provider over access network 804 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with network nodes 810 while still connected via hub 814 via a wired orwireless connection. In some embodiments, hub 814 may be a dedicated hub-that is, a hub whose primary function is to route communications to / from the UEs from / to network node 810b. In other embodiments, hub 814 may be a non-dedicated hub-that is, a device which is capable of operating to route communications between the UEs and network node 810b, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0107] Figure 9 shows a UE 900 in accordance with some embodiments. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptopmounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle-mounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by 3GPP, including a narrow band internet of things (NB-loT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE.

[0108] A UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, orwhich may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent adevice that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

[0109] UE 900 includes processing circuitry 902 that is operatively coupled via bus 904 to input / output interface 906, power source 908, memory 910, communication interface 912, and possibly other components not explicitly shown. Certain UEs may utilize all or a subset of the components shown in Figure 9. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0110] Processing circuitry 902 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in memory 910. Processing circuitry 902 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, processing circuitry 902 may include multiple central processing units (CPUs).

[0111] In the example, input / output interface 906 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices.

[0112] Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into UE 900. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.In some embodiments, power source 908 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. Power source 908 may further include power circuitry for delivering power from power source 908 itself, and / or an external power source, to the various parts of UE 900 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging power source 908. Power circuitry may perform any formatting, converting, or other modification to the power from power source 908 to make the power suitable for the respective components of UE 900 to which power is supplied.

[0113] Memory 910 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, memory 910 includes one or more application programs 914, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 916. Memory 910 may store, for use by UE 900, any of a variety of various operating systems or combinations of operating systems.

[0114] Memory 910 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUlCC), integrated UICC (iUICC) ora removable UICC commonly known as ‘SIM card.’ Memory 910 may allow UE 900 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in memory 910, which may be or comprise a device-readable storage medium.

[0115] Processing circuitry 902 may be configured to communicate with an access network or other network using communication interface 912. Communication interface912 may comprise one or more communication subsystems and may include or be communicatively coupled toantenna 922. Communication interface 912 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE ora network node in an access network). Each transceiver may include transmitter 918 and / or receiver 920 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, transmitter 918 and receiver 920 may be coupled to one or more antennas (e.g., 922) and may share circuit components, software or firmware, or alternatively be implemented separately.

[0116] In the illustrated embodiment, communication functions of communication interface 912 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in accordingto one or more communication protocols and / or standards, such as IEEE 802.11 , Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / internet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

[0117] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 912, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 19 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., an alert is sent when moisture is detected), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).

[0118] As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.A UE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smartwatch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and / or software in dependence of the intended application of the loT device in addition to other components as described in relation to UE 900 shown in Figure 9.

[0119] As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements, and transmits the results of such monitoring and / or measurements to another UE and / or a network node. The UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-loT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.

[0120] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g., by controlling an actuator) to increase or decrease the drone’s speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.Figure 10 shows a network node 1000 in accordance with some embodiments. Examples of network nodes include, but are not limited to, access points (e.g., radio access points) and base stations (e.g., radio base stations, Node Bs, eNBs, and gNBs).

[0121] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0122] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).

[0123] Network node 1000 includes processing circuitry 1002, memory 1004, communication interface 1006, and power source 1008. Network node 1000 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which network node 1000 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, network node 1000 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1004 for different RATs) and some components may be reused (e.g., a same antenna 1010 may be shared by different RATs). Network node 1000 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1000, for example GSM, WCDMA, LTE, NR, WiFi,Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chipsand other components within network node 1000.

[0124] Processing circuitry 1002 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 1000 components, such as memory 1004, to provide network node 1000 functionality.

[0125] In some embodiments, processing circuitry 1002 includes a system on a chip (SOC). In some embodiments, processing circuitry 1002 includes one or more of radio frequency (RF) transceiver circuitry 1012 and baseband processing circuitry 1014. In some embodiments, RF transceiver circuitry 1012 and baseband processing circuitry 1014 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 1012 and baseband processing circuitry 1014 may be on the same chip or set of chips, boards, or units.

[0126] Memory 1004 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) ora Digital Video Disk (DVD)), and / or any other volatile or nonvolatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by processing circuitry 1002. Memory 1004 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions (collectively denoted computer program product 1004a) capable of being executed by processing circuitry 1002 and utilized by network node 1000. Memory 1004 may be used to store any calculations made by processing circuitry 1002 and / or any data received via communication interface 1006. In some embodiments, processing circuitry 1002 and memory 1004 is integrated.

[0127] Communication interface 1006 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, communicationinterface 1006 comprises port(s) / terminal(s) 1016 to send and receive data, for example to and from a network over a wired connection. Communication interface 1006 also includes radio front-end circuitry 1018 that may be coupled to, or in certain embodiments a part of, antenna 1010. Radio front-end circuitry 1018 comprises filters 1020 and amplifiers 1022. Radio frontend circuitry 1018 may be connected to antenna 1010 and processing circuitry 1002. The radio front-end circuitry may be configured to condition signals communicated between antenna 1010 and processing circuitry 1002. Radio front-end circuitry 1018 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. Radio front-end circuitry 1018 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 1020 and / or amplifiers 1022. The radio signal may then be transmitted via antenna 1010. Similarly, when receiving data, antenna 1010 may collect radio signals which are then converted into digital data by radio front-end circuitry 1018. The digital data may be passed to processing circuitry 1002. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0128] In certain alternative embodiments, network node 1000 does not include separate radio frontend circuitry 1018, instead, processing circuitry 1002 includes radio front-end circuitry and is connected to antenna 1010. Similarly, in some embodiments, all or some of RF transceiver circuitry 1012 is part of communication interface 1006. In still other embodiments, communication interface 1006 includes one or more ports or terminals 1016, radio front-end circuitry 1018, and RF transceiver circuitry 1012, as part of a radio unit (not shown), and communication interface 1006 communicates with the baseband processing circuitry 1014, which is part of a digital unit (not shown).

[0129] Antenna 1010 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. Antenna 1010 may be coupled to radio front-end circuitry 1018 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, antenna 1010 is separate from network node 1000 and connectable to network node 1000 through an interface or port.

[0130] Antenna 1010, communication interface 1006, and / or processing circuitry 1002 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a UE, another network node and / or any other network equipment. Similarly, antenna 1010, communication interface 1006, and / or processing circuitry 1002 may beconfigured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.

[0131] Power source 1008 provides power to the various components of network node 1000 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). Power source 1008 may further comprise, or be coupled to, power management circuitry to supply the components of network node 1000 with power for performing the functionality described herein. For example, network node 1000 maybe connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of power source 1008. As a further example, power source 1008 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0132] Embodiments of network node 1000 may include additional components beyond those shown in Figure 10 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, network node 1000 may include user interface equipment to allow input of information into network node 1000 and to allow output of information from network node 1000. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for network node 1000.

[0133] Figure 11 is a block diagram of a host 1100, which may be an embodiment of host 816 of Figure 8, in accordance with various aspects described herein. Host 1100 may be or comprise various combinations of hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. Host 1100 may provide one or more services to one or more UEs.

[0134] Host 1100 includes processing circuitry 1102 that is operatively coupled via a bus 1104 to an input / output interface 1106, a network interface 1108, a power source 1110, and a memory 1112. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 9 and 10, such that the descriptions thereof are generally applicable to the corresponding components of host 1100.Memory 1112 may include one or more computer programs including one or more host application programs 1114 and data 1116, which may include user data, e.g., data generated by a UE for host 1100 or data generated by host 1100 for a UE. Embodiments of host 1100 may utilize only a subset or all of the components shown. Host application programs 1114 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (WC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711 ), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). Host application programs 1114 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, host 1100 may select and / or indicate a different host for over-the-top services for a UE. Host application programs 1114 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0135] Figure 12 is a block diagram illustrating a virtualization environment 1200 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1200 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized.

[0136] Applications 1202 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment 1200 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.Hardware 1204 includes processing circuitry, memory that stores software and / or instructions (collectively denoted computer program product 1204a) executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1206 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1208a-b (one or more of which may be generally referred to as VMs 1208), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 1206 may present a virtual operating platform that appears like networking hardware to VMs 1208.

[0137] VMs 1208 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1206. Different embodiments of the instance of a virtual appliance 1202 may be implemented on one or more of VMs 1208, and the implementations may differ. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[0138] In the context of NFV, each VM 1208 may be a software implementation of a physical machine that runs programs as if they were executingon a physical, non-virtualized machine. Each of VMs 1208, and that part of hardware 1204 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1208 on top of hardware 1204 and corresponds to application 1202.

[0139] Hardware 1204 may be implemented in a standalone network node with generic or specific components. Hardware 1204 may implement some functions via virtualization. Alternatively, hardware 1204 may be part of a larger cluster of hardware (e.g., such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1210, which, among others, oversees lifecycle management of applications 1202. In some embodiments, hardware 1204 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components toprovide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of control system 1212 which may alternatively be used for communication between hardware nodes and radio units.

[0140] Figure 13 shows a communication diagram of a host 1302 communicating via a network node 1304 with a UE 1306 overa partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 812a of Figure 8 and / or UE 900 of Figure 9), network node (such as network node 810a of Figure 8 and / or network node 1000 of Figure 10), and host (such as host 816 of Figure 8 and / or host 1100 of Figure 11) discussed in the preceding paragraphswill now be described with reference to Figure 13.

[0141] Like host 1100, embodiments of host 1302 include hardware, such as a communication interface, processing circuitry, and memory. Host 1302 also includes software, which is stored in or accessible by host 1302 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as UE 1306 connecting via an over-the-top (OTT) connection 1350 extending between UE 1306 and host 1302. In providing the service to the remote user, a host application may provide user data which is transmitted using OTT connection 1350.

[0142] Network node 1304 includes hardware enabling it to communicate with host 1302 and UE 1306. Connection 1360 may be direct or pass through a core network (like core network 806 of Figure 8) and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.

[0143] UE 1306 includes hardware and software, which is stored in or accessible by UE 1306 and executable by the UE’s processing circuitry. The software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 1306 with the support of host 1302. In host 1302, an executing host application may communicate with the executing client application via OTT connection 1350 terminating at UE 1306 and host 1302. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. OTT connection 1350 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through OTT connection 1350.OTT connection 1350 may extend via a connection 1360 between host 1302 and network node 1304 and via wireless connection 1370 between network node 1304 and UE 1306 to provide the connection between host 1302 and UE 1306. Connection 1360 and wireless connection 1370, over which OTT connection 1350 may be provided, have been drawn abstractly to illustrate the communication between host 1302 and UE 1306 via network node 1304, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

[0144] As an example of transmitting data via OTT connection 1350, in step 1308, host 1302 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interactingwith UE 1306. In other embodiments, the user data is associated with a UE 1306 that shares data with host 1302 without explicit human interaction. In step 1310, host 1302 initiates a transmission carryingthe user data towards UE 1306. Host 1302 may initiate the transmission responsive to a request transmitted by UE 1306. The request may be caused by human interaction with UE 1306 or by operation of the client application executing on UE 1306. The transmission may pass via network node 1304, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1312, network node 1304 transmits to UE 1306 the user data that was carried in the transmission that host 1302 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1314, UE 1306 receives the user data carried in the transmission, which may be performed by a client application executed on UE 1306 associated with the host application executed by host 1302.

[0145] In some examples, UE 1306 executes a client application which provides user data to host 1302. The user data may be provided in reaction or response to the data received from host 1302. Accordingly, in step 1316, UE 1306 may provide user data, which may be performed by executing the client application. In providingthe user data, the client application may further consider user input received from the user via an input / output interface of UE 1306. Regardless of the specific manner in which the user data was provided, UE 1306 initiates, in step 1318, transmission of the user data towards host 1302 via network node 1304. In step 1320, in accordance with the teachings of the embodiments described throughout this disclosure, network node 1304 receives user data from UE 1306 and initiates transmission of the received user data towards host 1302. In step 1322, host 1302 receives the user data carried in the transmission initiated by UE 1306.

[0146] One or more of the various embodiments improve the performance of OTT services provided to UE 1306 using OTT connection 1350, in which wireless connection 1370 forms the last segment.More precisely, embodiments can reduce and / or prevent undesired recovery actions by UEs. For example, due to the conditions for considering an LTM cell switch procedure successful (causing UE to stop a supervision timer), undesired recovery actions due to supervision timer expiration are prevented at the UE. This is especially an issue in the scenarios where LTM cell switch needs to be performed without a RA procedure (i.e., “RACH-less”). Preventing undesired recovery actions makes LTM RACH-less solutions more efficient, which reduces the delay to access an LTM candidate cell. Embodiments can facilitate predictable UE behavior in LTM execution failures and can reduce and / or eliminate ambiguity for UE actions in the event of LTM failures that are concurrent other failures such as radio link failure (RLF). By improving operation of UEs and RANs in this manner, embodiments increase the value of OTT services delivered to / from the UE via the RAN, to both end users and service providers.

[0147] In an example scenario, factory status information may be collected and analyzed by host 1302. As another example, host 1302 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, host 1302 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, host 1302 may store surveillance video uploaded by a UE. As another example, host 1302 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, host 1302 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and / or transmitting data.

[0148] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring OTT connection 1350 between host 1302 and UE 1306, in response to variations in the measurement results. The measurement procedure and / or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of host 1302 and / or UE 1306. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which OTT connection 1350 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of OTT connection 1350 may include message format,retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of network node 1304. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by host 1302. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or ‘dummy’ messages, using OTT connection 1350 while monitoring propagation times, errors, etc.

[0149] The foregoing merely illustrates the principles of the disclosure. Various modifications and alterations to the described embodiments will be apparent to those skilled in the art in view of the teachings herein. It will thus be appreciated that those skilled in the art will be able to devise numerous systems, arrangements, and procedures that, although not explicitly shown or described herein, embody the principles of the disclosure and can be thus within the spirit and scope of the disclosure. Various embodiments can be used together with one another, as well as interchangeably therewith, as should be understood by those having ordinary skill in the art.

[0150] The term unit, as used herein, can have conventional meaning in the field of electronics, electrical devices and / or electronic devices and can include, for example, electrical and / or electronic circuitry, devices, modules, processors, memories, logic solid state and / or discrete devices, computer programs or instructions for carrying out respective tasks, procedures, computations, outputs, and / or displaying functions, and so on, as such as those that are described herein.

[0151] Any appropriate steps, methods, features, functions, or benefits disclosed herein may be performed through one or more functional units or modules of one or more virtual apparatuses. Each virtual apparatus may comprise a number of these functional units. These functional units may be implemented via processing circuitry, which may include one or more microprocessor or microcontrollers, as well as other digital hardware, which may include Digital Signal Processor (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as Read Only Memory (ROM), Random Access Memory (RAM), cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory includes program instructions for executing one or more telecommunications and / or data communications protocols as well as instructions for carrying out one or more of the techniques described herein. In some implementations, the processing circuitry may be used tocause the respective functional unit to perform corresponding functions according to one or more embodiments of the present disclosure.

[0152] As described herein, device and / or apparatus can be represented by a semiconductor chip, a chipset, or a (hardware) module comprising such chip or chipset; this, however, does not exclude the possibility that a functionality of a device or apparatus, instead of being hardware implemented, be implemented as a software module such as a computer program or a computer program product comprising executable software code portions for execution or being run on a processor. Furthermore, functionality of a device or apparatus can be implemented by any combination of hardware and software. A device or apparatus can also be regarded as an assembly of multiple devices and / or apparatuses, whether functionally in cooperation with or independently of each other. Moreover, devices and apparatuses can be implemented in a distributed fashion throughout a system, so long as the functionality of the device or apparatus is preserved. Such and similar principles are considered as known to a skilled person.

[0153] Furthermore, functions described herein as being performed by a wireless device or a network node may be distributed over a plurality of wireless devices and / or network nodes. In other words, it is contemplated that the functions of the network node and wireless device described herein are not limited to performance by a single physical device and, in fact, can be distributed among several physical devices.

[0154] In addition, certain terms used in the present disclosure, includingthe specification, drawings and embodiments thereof, can be used synonymously in certain instances, including, but not limited to, e.g., data and information. It should be understood that, while these words and / or other words that can be synonymous to one another, can be used synonymously herein, that there can be instances when such words can be intended to not be used synonymously.

[0155] Further, to the extent that the prior art knowledge has not been explicitly incorporated by reference herein above, it is explicitly incorporated herein in its entirety. All publications referenced are incorporated herein by reference in their entireties.

[0156] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification andthe relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0157] In addition, certain terms used in the present disclosure, including the specification and drawings, can be used synonymously in certain instances (e.g., “data” and “information”). It should be understood, that although these terms (and / or other terms that can be synonymous with one another) can be used synonymously herein, there can be instances when such words can be intended to not be used synonymously.

[0158] EXAMPLE EMBODIMENTS

[0159] Embodiments of the techniques, apparatuses, and systems described herein include, but are not limited to, the following enumerated examples:

[0160] A1. A method, in a wireless device operating in a wireless network, for monitoring performance of artificial-intelligence / machine-learning (AI / ML) predictions performed by the wireless device, the method comprising:

[0161] calculating at least one performance metric characterizing prediction accuracy for a plurality of measurement and / or event predictions made by one or more AI / ML models running in the wireless device; and

[0162] sending, to a network node in the wireless network, an indication of prediction accuracy for the measurement and / or event predictions, based on the at least one performance metric.

[0163] A2. The method of example embodiment A1, wherein the indication of prediction accuracy comprises an indication that the performance metric or other measure of prediction accuracy for the measurement and / or event predictions has fallen below a threshold value.

[0164] A3. The method of example embodiment A1 or A2, wherein the method further comprises receiving, from the network node, an instruction to deactivate an AI / ML inference configuration previously activated for the wireless device, in response to sending the indication of prediction accuracy.A4. The method of example embodiment A2, wherein the method further comprises autonomously deactivating an AI / ML inference configuration previously provided to the wireless device, based on the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions having fallen below the threshold value.

[0165] A5. The method of example embodiment A1 , wherein the indication of prediction accuracy comprises an indication that the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions has risen above a threshold value.

[0166] A6. The method of example embodiment A1 or A5, wherein the method further comprises receiving, from the network node, an instruction to activate an AI / ML inference configuration previously provided to the wireless device, in response to sending the indication of prediction accuracy.

[0167] A7. The method of example embodiment A5, wherein the method further comprises autonomously activating an AI / ML inference configuration previously provided to the wireless device, based on the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions having risen above the threshold value.

[0168] A6. The method of example embodiment A5, wherein the AI / ML inference configuration previously provided to the wireless device was deactivated, prior to said sending of the indication of prediction accuracy for the measurement and / or event predictions, in response to a previous report of prediction accuracy for the AI / ML inference configuration.

[0169] A7. The method of any of example embodiments A1-A6, wherein the method further comprises receiving, prior to said calculating, an inference configuration comprising one or more performance monitoring parameters for monitoring performance of the one or more AI / ML models.

[0170] A8. The method of example embodiment A7, wherein the inference configuration comprises a threshold parameter indicative of a threshold performance for predictive accuracy of the one or more AI / ML models.A9. The method of example embodiment A7 or A8, wherein the inference configuration comprises an indication of whether the wireless device is permitted to autonomously deactivate and / or active the inference configuration.

[0171] A10. The method of anyone of example embodiments A1-A9, wherein the at least one performance metric comprises an F1 score or a mean-squared error (MSE) metric.

[0172] A11. A wireless device adapted to carryout a method according to anyone of example embodiments A1-A10.

[0173] A12. A wireless device comprising radio circuitry and processing circuitry operatively coupled to the radio circuitry, the processing circuitry being configured, e.g., with program instructions stored in memory operatively coupled to the processing circuitry, to carry out a method according to anyone of example embodiments A1-A11.

[0174] B1. A method, in a network node in a wireless network, for monitoring performance of artificial-intelligence / machine-learning (AI / ML) predictions performed by a wireless device operating in the wireless network, the method comprising:

[0175] receiving, from the wireless device, an indication of prediction accuracy for measurement and / or event predictions made by one or more AI / ML models running in the wireless device; and

[0176] determining whether to activate or deactivate an inference configuration in the wireless device, based on the indication.

[0177] B2. The method of example embodiment B1 , wherein the indication of prediction accuracy comprises an indication that a performance metric or other measure of prediction accuracy for the measurement and / or event predictions has fallen below a threshold value, and wherein the method comprises sending, to the wireless device, an instruction to deactivate the inference configuration, in response to receiving the indication.

[0178] B3. The method of example embodiment B1 , wherein the indication of prediction accuracy comprises an indication that a performance metric or other measure of prediction accuracy for the measurement and / or event predictions has risen above a threshold value, and wherein themethod comprises sending, to the wireless device, an instruction to activate the inference configuration, in response to receiving the indication.

[0179] B4. The method of any of example embodiments B1-B3, wherein the method further comprises sending to the wireless device, prior to said receiving, an inference configuration comprising one or more performance monitoring parameters for monitoring performance of the one or more AI / ML models.

[0180] B5. The method of example embodiment B4, wherein the inference configuration comprises a threshold parameter indicative of a threshold performance for predictive accuracy of the one or more AI / ML models.

[0181] B6. The method of example embodiment B4 or B5, wherein the inference configuration comprises an indication of whether the wireless device is permitted to autonomously deactivate and / or active the inference configuration.

[0182] B7. The method of example embodiment B6, wherein the inference configuration comprises an indication of a second inference configuration for the wireless device to use in the event that the wireless device autonomously deactivates the inference configuration.

[0183] B8. A network node adapted to carryout a method according to anyone of example embodiments B1-B6.

[0184] B8. A network node comprising radio circuitry and processing circuitry operatively coupled to the radio circuitry, the processing circuitry being configured, e.g., with program instructions stored in memory operatively coupled to the processing circuitry, to carryout a method according to anyone of example embodiments B1-B6.

[0185] REFERENCES

[0186] 1. RP-242393, Study on Al (Artificial lntelligence) / ML (Machine Learning) for mobility in NR 2. TR 38.843 v.18.0.0, Study on Artificial Intelligence (Al) / Machine Learning (ML) for NR air interface

[0187] 3. TS 38.331 , v.18.5.1 , NR; Radio Resource Control (RRC); Protocol specificationABBREVIATIONS GPP 3rd Generation Partnership Project 5GC / 5GCN 5G Core Network

[0188] 5GS 5G System

[0189] AF Application Function

[0190] Al Artificial Intelligence

[0191] AMF Access and Mobility Management Function AN Access Network

[0192] ARQ Automatic Repeat Request

[0193] API Application Programming Interface ARFCN Absolute Radio-Frequency Channel Number ASN.1 Abstract Syntax Notation One

[0194] BM Beam Management

[0195] CA Carrier Aggregation

[0196] CE Control Element

[0197] CGI Cell Global Identity

[0198] CHO Conditional Handover

[0199] CN Core Network

[0200] CP Control Plane

[0201] CPC Conditional PSCell Change

[0202] CSI-RS Channel State Information reference signal CU Central Unit

[0203] CU-UP Central Unit User Plane

[0204] DAPS Dual Active Protocol Stacks

[0205] DC Dual Connectivity

[0206] DCAF Data Collection Application Function

[0207] DCI Downlink Control Information

[0208] DRB Data Radio Bearer

[0209] DRX Discontinuous reception

[0210] DU Distributed Unit

[0211] eNB E-UTRAN NodeB

[0212] EN-DC E-UTRA-NR Dual Connectivity

[0213] E-UTRA Evolved UTRA

[0214] E-UTRAN Evolved UTRANgNB Radio base station in NR

[0215] GNSS Global Navigation Satellite System GPS Global Positioning System

[0216] HARQ Hybrid ARQ

[0217] HOF Handover Failure

[0218] ID Identifier / ldentity

[0219] IE Information Element

[0220] L2 Layer 2

[0221] LCM Life Cycle Management

[0222] LTE Long Term Evolution

[0223] MAC Medium Access Control

[0224] MBS Multicast Broadcast Service

[0225] MCE Measurement Collector Entity

[0226] MCG Master Cell Group

[0227] MDT Minimization of Drive Test

[0228] MIMO Multiple Input Multiple Output

[0229] ML Machine Learning

[0230] MME Mobility Management Entity

[0231] MN Master Node

[0232] MNO Mobility Network Operator

[0233] MR-DC Multi-Radio Dual Connectivity

[0234] NE-DC NR-E-UTRA Dual Connectivity

[0235] NEF Network Exposure Function

[0236] NG Next Generation

[0237] NGEN-DC NG-RAN E-UTRA-NR Dual Connectivity NG-RAN NG Radio Access Network

[0238] NR New Radio

[0239] NW Network

[0240] NWDAF Network Data Analytics Function OAM / O&M Operation and Maintenance

[0241] OTT Over-the-Top

[0242] PBCH Physical Broadcast Channel

[0243] PCell Primary Cell

[0244] PCF Policy Control FunctionPCI Physical Cell Identity

[0245] PDCCH Physical Downlink Control Channel

[0246] PDCP Packet Data Convergence Protocol

[0247] PDU Protocol Data Unit

[0248] PLMN Public Land Mobile Network

[0249] PSCell Primary Secondary Cell

[0250] PTM Pointto Multipoint

[0251] PTP Pointto Point

[0252] QCI QoS Class Identifier

[0253] QFI QoS Flow Identifier

[0254] QMC QoE Measurement Collection

[0255] QoE Quality of Experience

[0256] QoS Quality of Service

[0257] RACH Random Access Channel

[0258] RAN Radio Access Network

[0259] RAT Radio Access Technology

[0260] RLF Radio Link Failure

[0261] RLM Radio Link Monitoring

[0262] RRC Radio Resource Control

[0263] RRM Radio Resource Management

[0264] RSRP Reference Signal Received Power

[0265] RSRQ Reference Signal Received Quality

[0266] RSSI Received Signal Strength Indicator

[0267] RV-QOE RAN Visible QoE

[0268] S1 The interface between the RAN and the CN in LTE. S1AP S1 Application Protocol

[0269] SCell Secondary Cell

[0270] SCG Secondary Cell Group

[0271] SDT Small Data Transmission

[0272] SINR Signal to Interference and Noise Ratio

[0273] SMF Session Management Function

[0274] SMO Service Management and Orchestration

[0275] SN Secondary Node

[0276] SNR Signal to Noise RatioSRB Signaling Radio Bearer

[0277] SSB Synchronization Signal Block

[0278] TA Terminal Adaptor

[0279] TCE Trace Collector Entity

[0280] TE Terminal Equipment

[0281] TS Technical Specification

[0282] UCI Uplink Control Information

[0283] UDM User Data Management

[0284] UE User Equipment

[0285] UP User Plane

[0286] URI Uniform Resource Identifier

[0287] URL Uniform Resource Locator

[0288] URLLC Ultra-Reliable Low-Latency Communication VR Virtual Reality

[0289] XML Extensible Markup Language

Claims

CLAIMS1. A method, in a wireless device operating in a wireless network, for monitoring performance of artificial-intelligence / machine-learning, AI / ML, predictions performed by the wireless device, the method comprising:calculating (620) at least one performance metric characterizing prediction accuracy for a plurality of measurement and / or event predictions made by one or more AI / ML models running in the wireless device; andsending (630), to a network node in the wireless network, an indication of prediction accuracy for the measurement and / or event predictions, based on the at least one performance metric.

2. The method of claim 1 , wherein the indication of prediction accuracy comprises an indication that the performance metric or other measure of prediction accuracy for the measurement and / or event predictions has fallen below a threshold value.

3. The method of claim 1 or 2, wherein the method further comprises receiving (640), from the network node, an instruction to deactivate an AI / ML inference configuration previously activated for the wireless device, in response to sending the indication of prediction accuracy.

4. The method of claim 2, wherein the method further comprises autonomously deactivating (640) an AI / ML inference configuration previously provided to the wireless device, based on the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions having fallen below the threshold value.

5. The method of claim 1 , wherein the indication of prediction accuracy comprises an indication that the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions has risen above a threshold value.

6. The method of claim 1 or 5, wherein the method further comprises receiving (640), from the network node, an instruction to activate an AI / ML inference configuration previously provided to the wireless device, in response to sending the indication of prediction accuracy.

7. The method of claim 5, wherein the method further comprises autonomously activating (640) an AI / ML inference configuration previously provided to the wireless device, based on the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions having risen above the threshold value.

8. The method of claim 7, wherein the AI / ML inference configuration previously provided to the wireless device was deactivated, prior to said sending of the indication of prediction accuracy for the measurement and / or event predictions, in response to a previous report of prediction accuracy for the AI / ML inference configuration.

9. The method of any of claims 1-8, wherein the method further comprises receiving (610), prior to said calculating, an inference configuration comprising one or more performance monitoring parameters for monitoring performance of the one or more AI / ML models.

10. The method of claim 9, wherein the inference configuration comprises a threshold parameter indicative of a threshold performance for predictive accuracy of the one or more AI / ML models.

11. The method of claim 9 or 10, wherein the inference configuration comprises an indication of whether the wireless device is permitted to autonomously deactivate and / or active the inference configuration.

12. The method of any one of claims 1-11 , wherein the at least one performance metric comprises an F1 score or a mean-squared error, MSE, metric.

13. A wireless device adapted to carryout a method according to anyone of claims 1-12.

14. A wireless device (900) comprising radio circuitry (912) and processing circuitry (902) operatively coupled to the radio circuitry (912), the processing circuitry (902) being configured, e.g., with program instructions stored in memory (910) operatively coupled to the processing circuitry (902), to:calculate at least one performance metric characterizing prediction accuracy for a plurality of measurement and / or event predictions made by one or more artificialintelligence / machine-learning, AI / ML, models running in the wireless device; andsend, to a network node in the wireless network, an indication of prediction accuracy for the measurement and / or event predictions, based on the at least one performance metric.

15. The wireless device (900) of claim 14, wherein the indication of prediction accuracy comprises an indication that the performance metric or other measure of prediction accuracy for the measurement and / or event predictions has fallen below a threshold value.

16. The wireless device (900) of claim 14 or 15, wherein the processing circuitry (902) is further configured to receive, from the network node, an instruction to deactivate an AI / ML inference configuration previously activated for the wireless device, in response to sending the indication of prediction accuracy.

17. The wireless device (900) of claim 15, wherein the processing circuitry (902) is further configured to autonomously deactivate an AI / ML inference configuration previously provided to the wireless device, based on the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions having fallen below the threshold value.

18. The wireless device (900) of claim 14, wherein the indication of prediction accuracy comprises an indication that the performance metric or other measure of prediction accuracy prediction accuracy for the measurement and / or event predictions has risen above a threshold value.

19. The wireless device (900) of claim 14 or 18, wherein the processing circuitry (902) is further configured to receive, from the network node, an instruction to activate an AI / ML inference configuration previously provided to the wireless device, in response to sending the indication of prediction accuracy.

20. The wireless device (900) of claim 19, wherein the processing circuitry (902) is further configured to autonomously activate an AI / ML inference configuration previously provided to the wireless device, based on the performance metric or other measure of prediction accuracyprediction accuracy for the measurement and / or event predictions having risen above the threshold value.

21. The wireless device (900) of claim 20, wherein the AI / ML inference configuration previously provided to the wireless device was deactivated, prior to said sending of the indication of prediction accuracy for the measurement and / or event predictions, in response to a previous report of prediction accuracy for the AI / ML inference configuration.

22. The wireless device (900) of any of claims 14-21 , wherein the processing circuitry (902) is further configured to receive, prior to said calculating, an inference configuration comprising one or more performance monitoring parameters for monitoring performance of the one or more AI / ML models.

23. The wireless device (900) of claim 22, wherein the inference configuration comprises a threshold parameter indicative of a threshold performance for predictive accuracy of the one or more AI / ML models.

24. The wireless device (900) of claim 22 or 23, wherein the inference configuration comprises an indication of whether the wireless device is permitted to autonomously deactivate and / or active the inference configuration.

25. The wireless device (900) of anyone of claims 14-24, wherein the at least one performance metric comprises an F1 score or a mean-squared error, MSE, metric.

26. A method, in a network node in a wireless network, for monitoring performance of artificial-intelligence / machine-learning, AI / ML, predictions performed by a wireless device operating in the wireless network, the method comprising:receiving (720), from the wireless device, an indication of prediction accuracy for measurement and / or event predictions made by one or more AI / ML models running in the wireless device; anddetermining (730) whether to activate or deactivate an inference configuration in the wireless device, based on the indication.

27. The method of claim 26, wherein the indication of prediction accuracy comprises an indication that a performance metric or other measure of prediction accuracy for the measurement and / or event predictions has fallen below a threshold value, and wherein the method comprises sending, to the wireless device, an instruction to deactivate the inference configuration, in response to receiving the indication.

28. The method of claim 26, wherein the indication of prediction accuracy comprises an indication that a performance metric or other measure of prediction accuracy for the measurement and / or event predictions has risen above a threshold value, and wherein the method comprises sending, to the wireless device, an instruction to activate the inference configuration, in response to receiving the indication.

29. The method of any of claims 26-28, wherein the method further comprises sending (740) to the wireless device, prior to said receiving, an inference configuration comprising one or more performance monitoring parameters for monitoring performance of the one or more AI / ML models.

30. The method of claim 29, wherein the inference configuration comprises a threshold parameter indicative of a threshold performance for predictive accuracy of the one or more AI / ML models.

31. The method of claim 29 or 30, wherein the inference configuration comprises an indication of whether the wireless device is permitted to autonomously deactivate and / or active the inference configuration.

32. The method of claim 31 , wherein the inference configuration comprises an indication of a second inference configuration for the wireless device to use in the event that the wireless device autonomously deactivates the inference configuration.

33. A network node (1000) adapted to carryout a method according to anyone of claims 26-32.

34. A network node (1000) comprising radio circuitry (1018) and processing circuitry (1002) operatively coupled to the radio circuitry (1018), the processing circuitry (1002) beingconfigured, e.g., with program instructions stored in memory (1004) operatively coupled to the processing circuitry (1002), to:receive, from the wireless device, an indication of prediction accuracy for measurement and / or event predictions made by one or more AI / ML models running in the wireless device; anddetermine whether to activate or deactivate an inference configuration in the wireless device, based on the indication.

35. The network node (1000) of claim 34, wherein the indication of prediction accuracy comprises an indication that a performance metric or other measure of prediction accuracy for the measurement and / or event predictions has fallen below a threshold value, and wherein the processing circuitry (1002) is configured to send, to the wireless device, an instruction to deactivate the inference configuration, in response to receiving the indication.

36. The network node (1000) of claim 34, wherein the indication of prediction accuracy comprises an indication that a performance metric or other measure of prediction accuracy for the measurement and / or event predictions has risen above a threshold value, and wherein the processing circuitry (1002) is further configured to send, to the wireless device, an instruction to activate the inference configuration, in response to receiving the indication.

37. The network node (1000) of any of claims 34-36, wherein the processing circuitry (1002) is further configured to send to the wireless device, prior to said receiving, an inference configuration comprising one or more performance monitoring parameters for monitoring performance of the one or more AI / ML models.

38. The network node (1000) of claim 37, wherein the inference configuration comprises a threshold parameter indicative of a threshold performance for predictive accuracy of the one or more AI / ML models.

39. The network node (1000) of claim 37 or 38, wherein the inference configuration comprises an indication of whether the wireless device is permitted to autonomously deactivate and / or active the inference configuration.

40. The method of claim 39, wherein the inference configuration comprises an indication of a second inference configuration for the wireless device to use in the event that the wireless device autonomously deactivates the inference configuration.

41. A computer program product comprising program instructions for execution by a network node, the program instructions being configured so as to cause the network node to carry out a method according to any of claims 26-32.

42. A computer program product comprising program instructions for execution by a wireless device, the program instructions being configured so as to cause the wireless device to carry out a method according to any of claims 1-12.

43. A computer-readable medium comprising, stored thereupon, a computer program product accordingto claim 41 or 42.