Ran-based minimization of drive testing (MDT) techniques for collecting data for ai / ml models

Abstract

Embodiments include methods performed by radio access network (RAN) node for collecting data to be used with artificial intelligence / machine learning (AI / ML) models. Such methods may include receiving, from a network node or function (NNF) outside of the RAN, a first request to activate a trace session for measurements by one or more user equipment (UEs) in the RAN. The first request includes information indicating that the measurements are associated with one or more AI / ML models. Such methods may include, based on the information received with the first request, selecting one or more UEs served by the RAN node to perform the requested measurements. Such methods may include sending, to the selected UEs, respective second requests to perform the measurements for the trace session and receiving, from the selected UEs, respective measurement reports comprising results of the measurements performed in accordance with the respective second requests.

Description

[0001] RAN-BASED MINIMIZATION OF DRIVE TESTING (MDT) TECHNIQUES FOR COLLECTING DATA FOR AI / ML MODELS

[0002] TECHNICAL FIELD

[0003] The present disclosure relates generally to wireless networks, and more specifically to techniques for collecting data in a radio access network (RAN) for training and / or monitoring of artificial intel ligence / machi ne learning (AI / ML) models used for intelligent RAN operation.

[0004] BACKGROUND

[0005] Currently the fifth generation (5G) of cellular systems is being stand ardized within the Third-Generation Partnership Project (3GPP). NR is developed for maximum flexibility to support multiple and substantially different use cases. These include enhanced mobile broadband (eMBB), machine type communications (MTC), ultra-reliable low latency communications (URLLC), side-link device-to-device (D2D), and several other use cases.

[0006] Figure 1 illustrates a high-level view of an exemplary 5G network architecture, consisting of a Next Generation Radio Access Network (NG-RAN, 199) and a 5G Core (5GC, 198). The NG-RAN can include one or more gNodeB's (gNBs) connected to the 5GC via one or more NG interfaces, such as gNBs (100, 150) connected via respective interfaces (102, 152). More specifically, the gNBs can be connected to one or more Access and Mobility Management Functions (AMFs) in the 5GC via respective NG-C interfaces and to one or more User Plane Functions (UPFs) in 5GC via respective NG-U interfaces. The 5GC can include various other network functions (NFs), such as Session Management Function(s) (SMF).

[0007] Although not shown, in some deployments the 5GC can be replaced by an Evolved Packet Core (EPC, 198), which conventionally has been used together with a Long-Term Evolution (LTE) Evolved UMTS RAN (E-UTRAN). In such deployments, gNBs (e.g., 100, 150) can connect to one or more Mobility Management Entities (MMEs) in the EPC via respective S1-C interfaces and to one or more Serving Gateways (SGWs) in EPC via respective NG-U interfaces.

[0008] In addition, the gNBs can be connected to each other via one or more Xn interfaces, such as Xn interface (140) between gNBs (100, 150). The radio technology for the NG-RAN is often referred to as "New Radio” (NR). With respect to the NR interface to UEs, each of the gNBs can support frequency division duplexing (FDD), time division duplexing (TDD), or a combination thereof. Each of the gNBs can serve a geographic coverage area including one or more cells and, in some cases, can also use various directional beams to provide coverage in the respective cells. In general, a downlink (DL) "beam” is a coverage area of a network-transmitted reference signal (RS) that may be measured or monitored by a UE.

[0009] The NG-RAN is layered into a Radio Network Layer (RNL) and a Transport Network Layer (TNL). The NG- RAN logical nodes and interfaces between them, is defined as part of the RNL. For each NG-RAN interface (NG, Xn, F1) the related TNL protocol and the functionality are specified. The TNL provides services for user plane transport and signaling transport.

[0010] NG RAN logical nodes (e.g., gNB 100) include a Central Unit (CU or gNB-CU, e.g., 110) and one or more Distributed Units (DU or gNB-DU, e.g., 120, 130). CUs are logical nodes that host higher-layer protocols and perform various gNB functions such controlling the operation of DUs. DUs are decentralized logical nodes that host lower layer protocols and can include, depending on the functional split option, various subsets of the gNB functions. Each CU and DU can include various circuitry needed to perform their respective functions, including processing circuitry, communication interface circuitry {e.g., transceivers), and power supply circuitry.

[0011] A gNB-CU connects to one or more gNB-DUs over respective F1 logical interfaces (e.g., 122 and 132 shown in Figure 1). However, a gNB-DU can be connected to only a single gNB-CU. The gNB-CU and its connected gNB- DU(s) are only visible to other gNBs and the 5GC as a gNB. In other words, the F1 interface is not visible beyond gNB- CU.

[0012] Centralized control plane protocols {e.g., PDCP-C and RRC) can be hosted in a different CU than centralized user plane protocols {e.g., PDCP-U). For example, a gNB-CU can be divided logically into a CU-CP function (including RRC and PDCP for signaling radio bearers) and CU-UP function (including PDCP for UP). A single CU-CP can be associated with multiple CU-UPs in a gNB. The CU-CP and CU-UP communicate with each other using the E1-AP protocol over the E1 interface. Furthermore, the F1 interface between CU and DU (see Figure 1) is functionally split into F1-C between DU and CU-CP and F1-U between DU and CU-UP. Three deployment scenarios for the split gNB architecture shown in Figure 1 are CU-CP and CU-UP centralized, CU-CP distributed / CU-UP centralized, and CU-CP centralized / CU-UP distributed.

[0013] UEs and RAN nodes (e.g., gNBs) can be configured to perform and report measurements to support minimization of drive testing (MDT), which is intended to reduce and / or minimize the requirements for manual testing of network performance by driving around the geographic coverage of the network. The MDT feature was first studied in Long-Term Evolution (LTE) Rel-9 (e.g., 3GPP TR 36.805 v9.0.0), first standardized for LTE in Rel-10, and is also supported for 5G / NR. MDT can address various network performance improvements such as coverage optimization, capacity optimization, mobility optimization, quality-of-service (QoS) verification, and parameterization for common channels (e.g., PDSCH). In general, MDT measurements can be management-based or signaling-based, and are supported for dual connectivity (DC) and the split node architecture shown in Figure 1.

[0014] Machine learning (ML) is a type of artificial intelligence (Al) that focuses on the use of data and algorithms to imitate the way that humans learn, gradually improving its performance. ML algorithms build models based on sample (or "training”) data, with the models being used subsequently to make predictions or decisions. ML algorithms can be used in a wide variety of applications (e.g., medicine, email filtering, speech recognition, etc.) in which it is difficult or unfeasible to develop conventional algorithms to perform the needed tasks.

[0015] AI / ML can be used to enhance the performance of a RAN, such as the NG-RAN shown in Figure 1 . 3GPP TR 37.817 (v17.0.0) describes a study on enhancement for data collection for NR and EN-DC (EUTRAN-NGRAN dual connectivity). This study describes a functional framework for RAN intelligence enabled by further data collection through use cases, examples etc., and identifies potential standardization impacts on current NG-RAN nodes and interfaces. The study also describes some high-level principles for RAN intelligence enabled by Al .

[0016] SUMMARY

[0017] Existing techniques for MDT activation and data collection are suitable and cost-effective for their original purposes, but are not suitable for data collection for the AI / ML framework described in 3GPP TR 37.817 (v17.0.0). For example, existing signaling-based MDT techniques target specific UEs that are pre-selected by the 5GC without considering up-to-date information available at the RAN node. Likewise, existing management-based MDT techniques are activated on a per-cell granularity, which provides only RAN control based on UEs that are currently present in a cell of interest. As such, existing MDT techniques lack capabilities for RAN-based selection of UEs, which is needed to obtain the desired training and / or inference data needed for proper operation of AI / ML models used for RAN intelligence and / or operations.

[0018] An object of embodiments of the present disclosure is to improve data collection to support AI / ML models used for RAN intelligence and / or operations, such as by providing, enabling, and / or facilitating solutions to exemplary problems summarized above and described in more detail below.

[0019] Embodiments include methods {e.g., procedures) performed by a RAN node for collecting data to be used with AI / ML models.

[0020] These exemplary methods can include receiving, from a network node or function (NNF) outside of the RAN, a first request to activate a trace session for measurements by one or more UEs in the RAN. The first request includes information indicating that the measurements are associated with one or more AI / ML models. These exemplary methods can include, based on the information received with the first request, selecting one or more UEs served by the RAN node to perform the requested measurements. These exemplary methods can include sending, to the selected UEs, respective second requests to perform the measurements for the trace session. These exemplary methods can include receiving, from the selected UEs, respective measurement reports comprising results of the measurements performed in accordance with the respective second requests.

[0021] In some embodiments, the information received with the first request includes one or more of the following:

[0022] • an indication that the requested measurements are for one or more operations to be performed on the one or more AI / ML models;

[0023] • an indication of the one or more operations to be performed;

[0024] • respective identifiers of the AI / ML models; • other information that differentiates the respective AI / ML models;

[0025] • an indication of whether the requested measurements are UE measurements, RAN node measurements, or both;

[0026] • one or more indications or commands for controlling the requested measurements; and

[0027] • one or more output feedback identifiers for which feedback is requested.

[0028] Different variants, features, options, etc. are disclosed for the various above-listed content of the information received with the first request.

[0029] In some embodiments, these exemplary methods can include performing at least one operation on the one or more AI / ML models based on the results of the measurements. In some of these embodiments, one of the following operations is performed for each of the AI / ML model based on the results of the measurements: model training, model re-training, continued model training based on a partially trained model received from the network node, model modification, model inference, model performance evaluation, model performance monitoring, input collection, and feedback collection.

[0030] In some embodiments, these exemplary methods can include receiving, from a core network node, a plurality of service requests associated with a corresponding plurality of UEs. Each service request includes one or more AI / ML- related indicators relevant for selection of the corresponding UE to perform measurements associated with AI / ML models. These exemplary methods can also include storing the one or more Al / M L-rel ated indicators for the respective UEs. In some of these embodiments, the one or more AI / ML-related indicators are stored in or with respective UE contexts.

[0031] In some embodiments, each UE served by the RAN node is associated with one or more AI / ML-related indicators relevant for selection of the UE to perform measurements associated with AI / ML models. In such case, selecting one or more UEs based on the information received with the first request includes selecting, from a plurality of UEs served by the RAN node, UEs with associated AI / ML-related indicators that match or correspond to one or more of the following: at least one of the identifiers of the AI / ML models, at least one scope configuration parameter that differentiates the respective AI / ML models, and at least one output feedback identifier.

[0032] In some of these embodiments, the measurements are MDT measurements, the NNF outside of the RAN is a network management system (e.g., OAM), the core network node is an AMF, and each service request is an initial context setup request or a handover request.

[0033] In some embodiments, these exemplary methods can include sending to a TOE a trace record comprising the results of the measurements performed by the one or more UEs. In some of these embodiments, the trace record also includes an indication that the results of the measurements were collected for one or more of the following: use of AI / ML in the RAN, one or more specific AI / ML use cases, one or more specific AI / ML models, and one or more specific AI / ML model types. In some of these embodiments, the trace record is also sent to one or more of the following: a network management system (e.g., OAM), a core network node (e.g., AMF), and one or more other RAN nodes (e.g., neighbor RAN nodes).

[0034] In some of these embodiments, these exemplary methods can also include the following operations:

[0035] • based on the information received with the first request, selecting one or more other RAN nodes to perform the requested measurements;

[0036] • sending, to the selected RAN nodes, respective third requests to perform the measurements for the trace session; and

[0037] • receiving, from the selected RAN nodes, respective measurement reports comprising results of the measurements performed in accordance with the respective third requests.

[0038] In such embodiments, the trace record sent to the TCE also includes the results of the measurements performed by the one or more RAN nodes.

[0039] In some embodiments, these exemplary methods can include sending, to the NNF outside of the RAN, a further request to activate the trace session for measurements associated with one or more AI / ML models. In such embodiments, the first request is received as a permission or a positive acknowledgement in response to the further request. In some of these embodiments, the further request includes a list of one or more UE identifiers for which user consent is required for the activation of the trace session, and the first request includes an indication of user consent to activate the trace session for the one or more UE identifiers.

[0040] Other embodiments include RAN nodes (e.g., base stations, eNBs, gNBs, ng-eNBs, etc. or unit / function thereof) configured to perform operations corresponding to any of the exemplary methods described herein. Other embodiments include non-transitory, computer-readable media storing program instructions that, when executed by processing circuitry, configure such RAN nodes to perform operations corresponding to any of the exemplary methods described herein.

[0041] These and other embodiments described herein can facilitate the use of conventional MDT / Trace functionality for collection of training data, input data, and feedback information for AI / ML models. In this manner, embodiments can leverage the existing MDT framework for new purposes and / or applications related to AI / ML. Embodiments also facilitate a granular selection of AI / ML related data for collection by OAM and / or RAN. At a high level, embodiments facilitate intelligent RAN implementations based on AI / ML, as well as improved RAN performance.

[0042] These and other objects, features, and advantages of embodiments of the present disclosure will become apparent upon reading the following Detailed Description in view of the Drawings briefly described below.

[0043] BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figures 1-2 illustrate two high-level views of an exemplary 5G network architecture.

[0045] Figure 3 shows a signaling diagram for 5GS management activation.

[0046] Figure 4 shows an NG-RAN trace start procedure for file-based trace reporting.

[0047] Figure 5A-B shows an NG-RAN trace start procedure for streaming trace reporting.

[0048] Figure 6 shows an exemplary signaling-based MDT activation procedure after a UE attaches to 5GC via NG- RAN.

[0049] Figure 7 shows an exemplary procedure for management-based MDT activation in NG-RAN for the case of non-split RAN node architecture.

[0050] Figure 8 is a block diagram of an exemplary framework for RAN intelligence based on AI / ML models.

[0051] Figure 9 shows an exemplary procedure for configuration and activation of RAN-based MDT with 5GC, according to various embodiments of the present disclosure.

[0052] Figure 10 shows a flow diagram of an exemplary method (e.g., procedure) for a RAN node (e.g., base station, eNB, g N B, ng-eNB, etc.), according to various embodiments of the present disclosure.

[0053] Figure 11 shows a communication system according to various embodiments of the present disclosure.

[0054] Figure 12 shows a network node according to various embodiments of the present disclosure.

[0055] Figure 13 is a block diagram of a virtualization environment in which functions implemented by some embodiments of the present disclosure may be virtualized.

[0056] Figure 14 illustrates communication between a host computing system, a network node, and a UE via multiple connections, at least one of which is wireless, according to various embodiments of the present disclosure.

[0057] DETAILED DESCRIPTION

[0058] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Other embodiments, however, are contained within the scope of the subject matter disclosed herein, the disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided as examples to convey the scope of the subject matter to those skilled in the art.

[0059] In general, all terms used herein are to be interpreted according to their ordinary meaning to a person of ordinary skill in the relevant technical field, unless a different meaning is expressly defined and / or implied from the context of use. All references to a / an / the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise or clearly implied from the context of use. The operations of any methods and / or procedures disclosed herein do not have to be performed in the exact order disclosed, unless an operation is explicitly described as following or preceding another operation and / or where it is implicit that an operation must follow or precede another operation. Any feature of any embodiment disclosed herein can apply to any other disclosed embodiment, as appropriate. Likewise, any advantage of any embodiment described herein can apply to any other disclosed embodiment, as appropriate.

[0060] Furthermore, the following terms are used throughout the description given below:

[0061] • Radio Access Node: As used herein, a "radio access node” (or equivalently "radio network node,” "radio access network node,” or "RAN node”) can be any node in a radio access network (RAN) that operates to wirelessly transmit and / or receive signals. Some examples of a radio access node include, but are not limited to, a base station (e.g., gNB in a 3GPP 5G / NR network or an enhanced or eNB in a 3GPP LTE network), base station distributed components (e.g., CU and DU), a high-power or macro base station, a low-power base station (e.g., micro, pico, femto, or home base station, or the like), an integrated access backhaul (IAB) node, a transmission point (TP), a transmission reception point (TRP), a remote radio unit (RRU or RRH), and a relay node.

[0062] • Core Network Node: As used herein, a "core network node” is any type of node in a core network. Some examples of a core network node include, e.g., a Mobility Management Entity (MME), a serving gateway (SGW), a PDN Gateway (P-GW), a Policy and Charging Rules Function (PCRF), an access and mobility management function (AMF), a session management function (SMF), a user plane function (UPF), a Charging Function (CHF), a Policy Control Function (PCF), an Authentication Server Function (AUSF), a location management function (LMF), or the like.

[0063] • Wireless Device: As used herein, a "wireless device” (or "WD” for short) is any type of device that is capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other wireless devices. Communicating wirelessly can involve transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information through air. Unless otherwise noted, the term "wireless device” is used interchangeably herein with the term "user equipment” (or "UE” for short), with both terms having a different meaning than the term "network node”.

[0064] • Radio Node: As used herein, a "radio node” can be either a "radio access node” (or equivalent term) or a "wireless device.”

[0065] • Network Node: As used herein, a "network node” is any node that is either part of the radio access network {e.g., a radio access node or equivalent term) or of the core network {e.g., a core network node discussed above) of a cellular communications network. Functionally, a network node is equipment capable, configured, arranged, and / or operable to communicate directly or indirectly with a wireless device and / or with other network nodes or equipment in the cellular communications network, to enable and / or provide wireless access to the wireless device, and / or to perform other functions {e.g., administration) in the cellular communications network.

[0066] • Node: As used herein, the term "node” (without prefix) can be any type of node that can in or with a wireless network (including RAN and / or core network), including a radio access node (or equivalent term), core network node, or wireless device. However, the term "node” may be limited to a particular type (e.g., radio access node, IAB node) based on its specific characteristics in any given context.

[0067] The above definitions are not meant to be exclusive. In other words, various ones of the above terms may be explained and / or described elsewhere in the present disclosure using the same or similar terminology. Nevertheless, to the extent that such other explanations and / or descriptions conflict with the above definitions, the above definitions should control.

[0068] Note that the description given herein focuses on a 3GPP cellular communications system and, as such, 3GPP terminology or similar terminology is sometimes used. However, the concepts disclosed herein are not limited to a 3GPP system and can be applied to any communication system that may benefit from them. Furthermore, although the term "cell” is used herein, it should be understood that (particularly with respect to 5G NR) beams may be used instead of cells and, as such, concepts described herein apply equally to both cells and beams.

[0069] Figure 2 shows another high-level view of an exemplary 5G network architecture, including an NG-RAN (299) and a 5GC (298). As shown in the figure, the NG-RAN can include gNBs {e.g., 210a, b) and ng-eNBs {e.g., 220a, b) that are interconnected with each other via respective Xn interfaces. The gNBs and ng-eNBs are also connected via the NG interfaces to the 5GC, more specifically to Access and Mobility Management Functions (AMFs, e.g., 230a, b) via respective NG-C interfaces and to User Plane Functions (UPFs, e.g., 240a, b) via respective NG-U interfaces. Moreover, the AMFs can communicate with Policy Control Functions (PCFs, e.g., 250a, b) and Network Exposure Functions (NEFs, e.g., 260a, b).

[0070] Each of the gNBs can support the NR radio interface including frequency division duplexing (FDD), time division duplexing (TDD), or a combination thereof. Each of ng-eNBs can support the fourth generation (4G) Long- Term Evolution (LTE) radio interface. Unlike conventional LTE eNBs, however, ng-eNBs connect to the 5GC via the NG interface. Each of the gNBs and ng-eNBs can serve a geographic coverage area including one or more cells (e.g., 211 a-b, 221 a-b). Depending on the cell in which it is located, a UE (205) can communicate with the gNB or ng-eNB serving that cell via the NR or LTE radio interface, respectively. Although Figure 2 shows gNBs and ng-eNBs separately, it is also possible that a single NG-RAN node provides both types of functionality.

[0071] In addition to providing coverage via cells as in LTE, NR networks also provide coverage via "beams.” In general, a downlink (DL, i.e., network to UE) "beam” is a coverage area of a network-transmitted reference signal (RS) that may be measured or monitored by a UE. In NR, for example, RS can include any of the following: synchronization signal / PBCH block (SSB), channel state information RS (CSI -RS), tertiary reference signals (or any other sync signal), positioning RS (PRS), demodulation RS (DMRS), phase-tracking reference signals (PTRS), etc. In general, SSB is available to all UEs regardless of the state of their connection with the network, while other RS {e.g., CSI-RS, DM-RS, PTRS) are associated with specific UEs that have a network connection.

[0072] The radio resource control (RRC) protocol controls communications between UE and gNB at the radio interface as well as mobility of a UE between cells in the NG-RAN. RRC also broadcasts system information (SI) and performs establishment, configuration, maintenance, and release of data radio bearers (DRBs) and signaling radio bearers (SRBs) used by UEs. Additionally, RRC controls addition, modification, and release of carrier aggregation (CA) and dual-connectivity (DC) configurations for UEs. RRC also performs various security functions such as key management.

[0073] After a UE is powered ON it will be in the RRCJDLE state until an RRC connection is established with the network, at which time the UE will transition to RRCJDONNECTED state {e.g., where data transfer can occur). The UE returns to RRCJDLE after the connection with the network is released, in RRCJDLE state, the UE’s radio is active on a discontinuous reception (DRX) schedule configured by upper layers. During DRX active periods (also referred to as “DRX On durations”), an RRCJDLE UE receives SI broadcast in the ceil where the UE is camping, performs measurements of neighbor cells to support cell reselection, and monitors a paging channel on PDCCH for pages from 5GC via gNB. An NR UE in RRCJDLE state is not known to the gNB serving the cell where the UE is camping. However, NR RRC includes an RRCJNACTIVE state in which a UE is known (e.g., via UE context) by the serving gNB. RRCJNACTIVE has some properties similar to a "suspended” condition used in LTE.

[0074] As briefly mentioned above, UEs and RAN nodes (e.g., gNBs) can be configured to perform and report measurements to support minimization of drive testing (MDT), which is intended to reduce and / or minimize the requirements for manual testing of network performance by driving around the geographic coverage of the network. MDT was previously specified for LTE and is being specified for NR in Rel-16. In general, there are two types of MDT measurements: logged MDT measurements for UEs in RRCJDLE / RRCJNACTIVE states and immediate MDT measurements for UEs in RRC_CONNECTED state.

[0075] For example, a UE in RRCJDLE state can be configured (e.g., via a LoggedMeasurement-Configuration RRC message from the network) to perform logged MDT measurements. An MDT configuration can include logginginterval and loggingduration. The UE starts a timer (T330) set to loggingduration (e.g., 10-120 min) upon receiving the configuration, and perform periodic MDT logging every logginginterval (1.28-61.44 s) within the loggingduration while the UE is in RRCJDLE state. In particular, the UE collects DL reference signal received strength and quality (i.e., RSRP, RSRQ) based on existing measurements required for cell reselection purposes. The UE reports the collected / logged information to the network when the UE returns to RRC_CONNECTED state.

[0076] A UE can also be configured to perform and report immediate MDT measurements while in RRC_CONNECTED state. Like logged MDT, immediate MDT measurements are based on existing UE and / or network measurements performed while a UE is in RRC_CONNECTED state, and can include any of the following measurement quantities:

[0077] • M1 : DL signal quantities measurement results for the serving cell and for intra-frequency / lnter- frequency / inter-RAT neighbor cells, including cell / beam level measurement for NR cells only.

[0078] • M2: Power Headroom measurement by UE.

[0079] • M3: Void for NR.

[0080] • M4: PDCP SDU Data Volume measurement separately for DL and UL, per DRB per UE.

[0081] • M5: Average UE throughput measurement separately for DL and UL, per DRB per UE and per UE for the DL, per DRB per UE and per UE for the UL, by gNB.

[0082] • M6: Packet Delay measurement separately for DL and UL, per DRB per UE.

[0083] • M7: Packet loss rate measurement separately for DL and UL, per DRB per UE.

[0084] • M8: RSSI measurement by UE (for WLAN / Bluetooth measurement).

[0085] • M9: RTT Measurement by UE (for WLAN measurement).

[0086] For example, the reporting of M1 measurements can be event-triggered according to existing RRM configuration for any of events A1 -A6 or B1-B2. In addition, M1 reporting can be periodic, A2 event-triggered, or A2 event-triggered periodic according to an MDT-specific measurement configuration. As another example, the reporting of M2 measurements can be based on reception of Power Headroom Report (PHR), while reporting for M3-M9 can be triggered by the expiration of a measurement collection period.

[0087] The MDT framework provides two ways of activating MDT and collecting data from the UEs. In managementbased MDT, data is collected from UEs in a specified area defined as a list of cells or as a list of tracking / routing / location areas. Management-based MDT is an enhancement of management-based trace functionality and can be used with either logged or immediate MDT. In signaling-based MDT, data is collected from a specific UE as specified by an IMEI(SV) or an IMSI. Signaling-based MDT is an enhancement of signaling-based subscriber and equipment trace functionality and can be used with either logged or immediate MDT.

[0088] In the 5G system (5GS), management-based trace activation uses a Services Based Management Architecture (SBMA). A Network Element (NE, specified according to 3GPP TS 32.422 (v17.7.1) section 5.4) is configured with Trace Control and Configuration parameters via interaction between a Provisioning management service (MnS) consumer and a Provisioning MnS producer. Figure 3 shows a signaling diagram for 5GS management activation where the role of Provisioning MnS producer is played by the NE and the role of Provisioning MnS consumer is played by the Management System. The configured NE shall not propagate the received Trace Control and Configuration parameters to any other NE's - regardless of whether the NE is involved in the actual data collection for the call.

[0089] The NG-RAN Cell Traffic trace functionality can be used for Management Based Trace Activation. In this case, Trace Session Activation is done to one or more (e.g., a list of) cells provided by an NG-RAN node. The following trace control and configuration parameters of the Trace Session are received by NG-RAN node in the trace session activation message from the Management System:

[0090] • Trace Reference (TR).

[0091] • Trace Depth.

[0092] • Trace Target: NG-RAN cells list.

[0093] • List of Interfaces for NG-RAN node.

[0094] • Trace Collection Entity (TCE) IP Address for the file-based trace reporting or Trace Reporting Consumer URI for the streaming trace reporting.

[0095] • Trace Reporting Format.

[0096] When the NG-RAN node receives the Trace Session Activation message from the management system, the NG-RAN node starts a Trace Session for the given cell or a list of cells.

[0097] After the Cell Traffic Trace has been activated in the monitored cell(s), the NG-RAN node starts Trace Recording Sessions for new and existing calls / sessions, although events for existing calls / sessions are not required to be recorded prior to the activation of the Cell Traffic Trace. When the NG-RAN node starts a Trace Recording Session it allocates a Trace Recording Session Reference (TRSR) for that session. For file-based trace reporting, the NG-RAN node sends the allocated TRSR, the TR, and the TCE IP Address in a CELL TRAFFIC TRACE message to the AMF via the NG connection. Figure 4 shows an NG-RAN trace start procedure for file-based trace reporting.

[0098] For streaming trace reporting, the NG-RAN node sends the allocated TRSR, the TR, and the URI of the Trace Reporting MnS consumer in a CELL TRAFFIC TRACE message to the AMF via the NG connection. Figure 5A-B shows an NG-RAN trace start procedure for streaming trace reporting.

[0099] In either case, when AMF receives the CELL TRAFFIC TRACE message, it obtains the SUPI / IMEI(SV) of the given call from its database and sends the obtained SUPI / IMEI (SV) numbers together with the TSRS and TR to the TCE (for file-based trace reporting) or to the Trace Reporting MnS consumer (for streaming trace reporting). When the NG-RAN supports several public land mobile networks (PLMNs), the NG-RAN node only selects UEs that provide a selectedPLMN-ldentity (e.g., in an RRCConnectionSetup message) that matches a pLMNTarget.

[0100] For management-based MDT activation, an NG-RAN node starts a trace recording session in a given cell for either immediate or logged MDT measurements for each selected UE that satisfies the MDT UE selection criteria or capability condition (e.g., logged MDT capable UE), provided that the management system previously activated a relevant cell trace session for that cell. The NG-RAN node configures each of the selected UEs with corresponding MDT RRC measurements. When the NG-RAN supports several PLMNs, the NG-RAN node only selects UEs that provide a selectedPLMN-ldentity (e.g., in an RRCConnectionSetup message) that matches a pLMNTarget.

[0101] For signaling-based MDT activation, activation of UE measurements extends the 5GC trace activation procedure. When a Trace Session is activated, configuration parameters of MDT are added into the message. For UE selection based on IMSI / I MEI (SV) / I MEI-TAC / SUPI, or in combination with geographical area information, an activation request for UE performance measurements is propagated to a selected UE. This mechanism works for any of following input parameters:

[0102] • IMSI only,

[0103] • IMSI and area information,

[0104] • IMEI(SV) only,

[0105] • IMEI(SV) and area information,

[0106] • I MEI-TAO only,

[0107] • I MEI-TAC and area information,

[0108] • SUPI only, or

[0109] • SUPI and area information.

[0110] After the user associated with the IMSI, IMEI(SV), IMEI-TAC, or SUPI attaches to the network via a RAN node, the AMF forwards the MDT configurations to the RAN node. If area information is specified, the AMF only forwards the MDT configuration to the RAN node when the UE satisfies the area information.

[0111] For immediate MDT, after receiving the MDT configuration, the RAN node can determine whether the selected IMSI / IMEI(SV) / SUPI satisfies criteria for initiating MDT data collection. If the area information is not satisfied, the RAN node retains the MDT configuration and propagates it during handover of the selected IMSI / IMEI (SV) / SUPI, as further specified in 3GPP TS 32.422 (v17.7.1 ) section 4.4.

[0112] For immediate MDT, there is no need to do MDT criteria checking by the UE. For logged MDT, after receiving the MDT configuration, the RAN node will forward it to the selected IMSI / I MEI (SV) / SUPI in an RRCReconfiguration message. After the UE receives the MDT configuration from the RAN node, it detects whether it is within the specified area information. If so, the UE will perform the measurement job specified by the MDT configuration. Otherwise, the UE will refrain from performing the measurement job and occasionally re-check whether it is within the specified area information.

[0113] For immediate MDT, the UE's trace session context is preserved in the network when the UE enters RRCJDLE or RRCJNACTIVE. For logged MDT, the trace session context is preserved in the UE until the trace session duration expires, even during state transitions between RRCJDLE / RRCJNACTIVE and RRC_CONNECTED and vice versa. The UE's logged MDT trace session context is stored in the network while the trace session is active, including when the UE is in RRC_CONNECTED. For signaling-based immediate MDT, a RAN node stores / retains an MDT configuration for a selected UE that is in RRCJNACTIVE when the RAN node receives the MDT configuration. When the UE moves to a cell served by a second RAN node, the RAN node forwards this information to the second RAN node during UE context retrieval.

[0114] The MDT activation procedure after UE attachment is the same whether the UE attaches to 5GC or EPC. Figure 6 shows an exemplary MDT activation procedure after a UE attaches to 5GC via NG-RAN. After receiving the Trace Session Activation command with the MDT configuration, the unified data management function (UDM) stores the relevant information and sends a message to the AMF to activate the trace for the MDT job. This message includes the following configuration parameters:

[0115] • Area Scope (TA, Cell).

[0116] • Trace Reference (TR).

[0117] • Trace Recording Session Reference (TRSR).

[0118] • List of Measurements.

[0119] • Reporting Trigger.

[0120] • Report Amount.

[0121] • Report Interval.

[0122] • Event Threshold.

[0123] • Logging Interval.

[0124] • Logging Duration.

[0125] • Trace Collection Entity IP Address.

[0126] • Positioning Method.

[0127] • Collection Period for NR RRM Measurements (present only if any M4 / M5 measurements are requested).

[0128] • Collection Period M6 in NR (present only if any M6 UL / DL measurements are requested).

[0129] • Collection Period M7 in NR (present only if any M7 UL / DL measurements are requested).

[0130] • MDT PLMN List.

[0131] • Report Type for Logged MDT (periodical logged or event-triggered measurement).

[0132] • Events List for Event-Triggered Measurement (logged MDT only), e.g., Event Threshold, Hysteresis, Time to Trigger (present only if L1 event is configured for logged MDT).

[0133] • Area Configuration for Neighbor Cells (logged MDT only).

[0134] • Sensor Information.

[0135] In case of logged MDT for a UE that is currently in RRCJDLE or RRCJNACTIVE, the AMF is not required to initiate paging of the UE to send the configuration.

[0136] The Management System initiating MDT activation shall validate that PLMNs specified in the MDT PLMN List are supported by all the cells specified in the area scope. A RAN node ignores any received requests for which none of the PLMNs in the MDT PLMN List match any PLMN that the RAN node is associated with.

[0137] The RAN node sends the AMF a TRACE FAILURE INDICATION message when the RAN node could not configure the UE during a handover to a target cell. Subsequently, the AMF shall try to reactivate MDT in the target cell if the target cell scope meets the MDT criteria.

[0138] When the UE re-enters a PLMN specified in the MDT PLMN List, the AMF shall be responsible for restarting the immediate MDT activation. If the re-entry is a result of an Xn handover, then one option is for AMF to use the path switch request as trigger. However, this re-starting "requirement” is only for best effort, since there can be cases where AMF may not be able to restart the MDT when the UE re-enters the PLMN specified in the MDT PLMN List. For example, the AMF may not be able to restart MDT when the UE performs intra-RAN node handover where path switch request is not necessarily sent.

[0139] Figure 7 shows an exemplary procedure for management-based MDT activation in NG-RAN for the case of non-split RAN node architecture. For management-based MDT data collection with no I MSI / IMEI (SV) / SUPI criteria for non-split architecture, UE selection can be done in the RAN node (e.g., gNB) based on input information received from management system and the user consent information stored in the RAN node. This mechanism works for Area Information GAM input parameter. A description of this procedure is given in 3GPP TS 32.422 (v17.7.1) section 4.1.1.9.2, with the most relevant parts repeated below. Whenever the RAN node receives the Management based MDT PLMM List in an Initial Context Setup Request or in a Handover Request message, it stores it for possible later usage. This is shown as operation 0, occurring at two possible places in the flow.

[0140] In operation 1 , the management system sends a Trace Session activation request to the RAN node. This request includes one or more of the following parameters for configuring UE measurements (as described further in 3GPP TS 32.422 (v17.7.1) section 5):

[0141] • Job Type.

[0142] • Area Scope where the UE measurements should be collected, e.g., list of NG-RAN cells. Tracking Area should be converted to NG-RAN cell.

[0143] • List of Measurements.

[0144] • Reporting Trigger.

[0145] • Report Interval.

[0146] • Report Amount.

[0147] • Event Threshold.

[0148] • Logging Interval.

[0149] • Logging Duration.

[0150] • Trace Reference.

[0151] • TCE IP Address.

[0152] • Anonymization of MDT Data.

[0153] • Collection Period for NR RRM Measurements (present only if any M4 or M5 measurements are requested).

[0154] • Collection Period M6 in NR (present only if any M6 UL / DL measurements are requested).

[0155] • Collection Period M7 in NR (present only if any M7 UL / DL measurements are requested).

[0156] • MDT PLMN List.

[0157] • Report Type for Logged MDT (periodical logged or event-triggered measurement).

[0158] • Events List for Event-Triggered Measurement (logged MDT only), e.g., Event Threshold, Hysteresis, Time to Trigger (present only if L1 event is configured for logged MDT).

[0159] • Area Configuration for Neighbor Cells (logged MDT only).

[0160] • Sensor Information.

[0161] In operation 2, when the RAN node receives the Trace Session activation request from its management system, it starts a Trace Session and saves the parameters associated with the Trace Session. In operation 3, the RAN node selects suitable UEs for MDT data collection. The selection is based on the area received from the management system, the areas where UEs are located, and user consent information received from the 5GC in the Management based MDT PLMN List. If a user is not in the specified area or if the Management based MDT PLMN List IE is not present in the UE context, the RAN node does not select the UE for MDT data collection. During UE selection, the RAN node also considers UE MDT capability when it selects UE for logged MDT configuration and does not select a UE that does not support logged MDT. If M4 or M5 measurements are requested in the MDT configuration, the RAN node starts measurement according to the received configuration.

[0162] In operation 4, the RAN node activates the MDT functionality to the selected UEs. When the RAN node selects a UE, it considers availability of Management based MDT PLMN List in the associated user context and the area scope parameter received in MDT configuration (Trace Session activation). Detailed description about user consent handling and how it is provided to the RAN node is described in 3GPP TS 32.422 (v17.7.1 ) section 4.9.2. If there is no Management based MDT PLMN List in the user context or the user is outside the area scope defined in the MDT configuration, the RAN node does not select the UE for MDT data collection. The RAN node assigns a Trace Recording Session Reference (TRSR) to each selected UE.

[0163] For logged MDT, the RAN node sends at least the following configuration information to the UE:

[0164] • Trace Reference (TR);

[0165] • TRSR; • TCE Id (e.g., value signaled as IP address of TCE from the EM is mapped to a TCE Id, using a configured mapping in the RAN);

[0166] • Logging Interval;

[0167] • Logging Duration;

[0168] • Absolute time reference;

[0169] • Area Scope where the UE measurements should be collected (e.g., cell list or tracking area); and

[0170] • MDT PLMN List.

[0171] Note that the RAN node cannot send the logged MDT configuration to UEs currently camping in the cell in RRCJDLE / RRCJNACTIVE. These UEs may be configured when they next initiate some activity (e.g., Service Request or Tracking Area Update) that causes them to transition to RRC_CONNECTED.

[0172] For immediate MDT, the RAN node sends one or more of the following parameters to the UE: List of Measurements, Reporting Trigger, Report Interval, Report Amount, and Event Threshold.

[0173] The RAN node performs necessary actions specified in TS 38.305

[0052] according to the value of Positioning Method (e.g., activating GNSS module of the UE) received in the Trace configuration. The RAN node captures location information and / or positioning measurements in the MDT trace record. If the Reporting Trigger parameter indicates that all configured RRM measurement triggers should be reported in MDT, the RAN node requests the UE to provide "best effort" location information together with the measurement reporting by setting the includeLocationlnfo IE in all RRC measurement reporting configurations.

[0174] When UE receives the MDT activation, it starts MDT functionality based on the received configuration parameters. In operation 5, the RAN node can retrieve the MDT report from the UE, via RRC message. However, the RAN node shall not retrieve MDT report from the UE if UE's registered PLMN (rPLMN) does not match the PLMN where TCE used to collect MDT data resides (e.g., RAN node's primary PLMN).

[0175] For immediate MDT, when the RAN node receives the MDT report from the UE in the RRC message the RAN node stores the UE's serving cell CGI together with the MDT report from the UE in the trace record. A UE configured to perform logged MDT measurements in RRCJDLE or RRCJNACTIVE indicates the availability of MDT measurements by means of a one-bit indicator, in RRCConnectionSetupComplete message during connection establishment as specified in 3GPP TS 32.422 (v17.7.1). The RAN node can decide to retrieve the logged measurements based on this indication by sending the UElnformationRequest message to the UE. The UE can answer with the collected MDT logs in UElnformationResponse message.

[0176] When the RAN node receives the MDT report from UE, the RAN node obtains TRSR, TR, and TCE Id from the report and compare the Trace PLMN (i.e., PLMN portion of TR) with the PLMN where TCE used to collect MDT data resides (e.g., its primary PLMN). The RAN node discards the MDT report in case of a mismatch but otherwise saves the UE measurements from the report to MDT records in operation 6.

[0177] In operation 7, the RAN node checks whether MDT anonymization requires the IMEI-TAC in the MDT record. If so, in operation 8 the RAN node sends TRSR, TR, serving cell CGI, and TCE IP Address in the CELL TRAFFIC TRACE message to the AMF via the NG connection. When the AMF receives this NG signaling message from the RAN node, it checks the Privacy Indicator (set to Logged MDT or Immediate MDT depending on configured Job Type). For logged MDT, the AMF sends the IMEI-TAC together with TRSR and TR to TCE as shown in Figure 7.

[0178] For immediate MDT, the AMF looks up the subscriber identities (IMEI(SV)) of the given call from its database and includes that information in the message sent in operation 9. Note that for management-based immediate MDT, TRSR may be duplicated among different RAN nodes when multiple cells are selected as the area scope for the same MDT job. In this case, the combination of TRSR and the UE's serving cell CGI in the MDT report can uniquely identify one trace recording session.

[0179] In operation 10, the RAN node forwards the trace records to the TCE. In case of logged MDT, the TCE Id indicated in the MDT report is translated to the actual IP address of the TCE by the RAN node before it forwards the measurement records, using configured mapping in the RAN node. In case of immediate MDT, the IP address of the TCE is indicated for the RAN node in the trace configuration.

[0180] An Immediate MDT measurement configuration is deleted in the UE together with the RRC context when entering idle or inactive mode. A Logged MDT trace session is preserved in the UE until the duration time of the trace session expires, including also multiple idle or inactive periods interrupted by various state transitions between RRCJDLE and RRC_CONNECTED.

[0181] In operation 11 , the TCE combines the MDT record received in operation 10 with TAG received in operation 9 (for logged MDT) based on TR and TRSR included in both. The Management system validates that mobile country code (MCC) and mobile network code (MNC) specified TR is the same as the PLMN supported by all the cells specified in the area scope. If the RAN node receives a request with a PLMN in TR that does not match any PLMN in its list, it shall ignore the request.

[0182] As briefly mentioned above, AI / ML can be used to enhance the performance of a RAN, such as NG-RAN. 3GPP TR 37.817 (v17.0.0) describes a study on enhancement for data collection for NR and EN -DC. This study describes a functional framework for RAN intelligence enabled by further data collection through use cases, examples, etc., and identifies potential standardization impacts on current NG-RAN nodes and interfaces. The study also describes some high-level principles to be applied for RAN intelligence enabled by Al.

[0183] 3GPP TR 37.817 (v17.0.0) also includes Figure 8, which is a block diagram of an exemplary framework for RAN intelligence based on AI / ML models. In general, an AI / ML model is a data driven algorithm that applies AI / ML techniques to generate a set of outputs based on a set of inputs. 3GPP TR 37.817 (v17.0.0) describes the following high-level principles in the context of Figure 8:

[0184] • Detailed AI / ML algorithms and models for use cases are implementation specific and out of RAN3 scope.

[0185] • Focus on AI / ML functionality and corresponding types of inputs / outputs.

[0186] • Input / output and the location of the Model Training and Model Inference function should be studied case by case.

[0187] • Focus on the analysis of data needed at the Model T raining function from Data Collection, while the aspects of how the Model Training function uses inputs to train a model are out of RAN3 scope.

[0188] • Focus on the analysis of data needed at the Model Inference function from Data Collection, while the aspects of how the Model Inference function uses inputs to derive outputs are out of RAN3 scope.

[0189] • Where AI / ML functionality resides within the current RAN architecture depends on deployment and on the specific use cases.

[0190] • Model Training and Model Inference functions should be able to request, if needed, specific information to be used to train or execute the AI / ML algorithm and to avoid reception of unnecessary information. The nature of such information depends on the use case and on the AI / ML algorithm .

[0191] • Model Inference function should signal the outputs of the model only to nodes that have explicitly requested them (e.g., via subscription), or nodes that take actions based on the output from Model Inference.

[0192] • An AI / ML model used in a Model Inference function must be initially trained, validated and tested before deployment.

[0193] The Data Collection block in Figure 8 is a function that provides input data to Model Training and Model Inference functions (described below). In general, "training data” is used by the Model Training function while "inference data” is used by the Model Inference function. AI / ML algorithm-specific data preparation (e.g., data preprocessing and cleaning, formatting, and transformation) is not carried out in the Data Collection function. Examples of input data include measurements from UEs or different network entities, feedback from the Actor block (described below), and output from an AI / ML model.

[0194] The Model Training block in Figure 8 is a function that performs the AI / ML model training, validation, and testing. In general, training involves learning the input / output relationship for an AI / ML model in a data-driven manner, so the trained AI / ML Model can be used for inference. The testing may generate model performance metrics. The Model Training function is also responsible for data preparation (e.g., data pre-processing and cleaning, formatting, and transformation) based on Training Data provided by the Data Collection function, if required.

[0195] The Model Training block includes a Model Deployment / Update procedure that is used to initially deploy a trained, validated, and tested AI / ML model to the Model Inference function , as well as to provide model updates to the Model Inference function. Details of the Model Deployment / Update procedure and the use case-specific AI / ML models transferred via this procedure are out of scope of the Rel-17 study item (SI). The feasibility of single-vendor or multi-vendor environment has not been studied in the Rel-17 SI.

[0196] The Model Inference block in Figure 8 is a function that provides AI / ML model outputs such as predictions or decisions. In general, inference involves using a trained AI / ML model to produce a set of outputs based on a set of inputs. The Model Inference function may provide model performance feedback to the Model Training function, when applicable. The Model Inference function is also responsible for data preparation (e.g. , data pre-processing and cleaning, formatting, and transformation) based on the Inference Data provided by the Data Collection function, if required. Details of the inference outputs and the model performance feedback are out of scope of the Rel-17 SI.

[0197] The Actor block in Figure 8 is a function that receives the output from the Model inference function and triggers or performs corresponding actions. The Actor may trigger actions directed to itself or to other entities. The Actor can also provide Feedback Information that may be needed to derive the Training Data, the Inference Data, or to monitor the performance of the AI / ML Model and its impact to the network through updating of key performance indicators (KPIs) and performance counters.

[0198] 3GPP TR 37.817 (v17.0.0) also identifies three use cases for RAN Intelligence: network energy savings, load balancing, and mobility optimization. 3GPP TR 37.817 (v17.0.0) describes that the following solutions can be used to support these three use cases:

[0199] • AI / ML Model Training function is deployed in GAM, while the Model Inference function resides within the RAN node (e.g., gNB).

[0200] • Both the AI / ML Model Training function and the AI / ML Model Inference function reside within the RAN node (e.g., gNB).

[0201] • gNB can continue Model Training based on AI / ML model trained in the 0AM.

[0202] In case of CU-DU split architecture, the following solutions are possible:

[0203] • AI / ML-based network energy saving: 1) AI / ML Model Training is in the 0AM and AI / ML Model Inference is in the gNB-CU; or 2) AI / ML Model Training and Model Inference are both located in the gNB-CU.

[0204] • AI / ML-based Mobility Optimization: AI / ML Model Training is in CU-CP or 0AM, and AI / ML Model Inference function is in CU-CP

[0205] • AI / ML-based Load Balancing: 1) AI / ML Model Training is in the CAM and AI / ML Model Inference is in the gNB-CU; or 2) AI / ML Model Training and Model Inference are both located in the gNB-CU.

[0206] Existing techniques for MDT activation and data collection (described above) are suitable and cost- effective for their original purposes, but are not suitable for data collection for the AI / ML framework described in 3GPP TR 37.817 (v17.0.0). For example, existing signaling-based MDT techniques target specific UEs that are preselected by the 5GC without considering up-to-date information available at the RAN node. Likewise, existing management-based MDT techniques are activated on a per-cell granularity, which provides only RAN control based on UEs that are currently present in a cell of interest. As such, existing MDT techniques lack capabilities for RAN- based selection of UEs, which is needed to obtain the desired training and / or inference data needed for proper operation of AI / ML models used for RAN intelligence and / or operations.

[0207] For example, the Model Training function is located at the 0AM and the Model Inference function is located at the RAN. In this case, the 0AM needs data such as UE performance feedback to train / retrain the AI / ML model and / or to monitor AI / ML model performance, which may then trigger re-training and / or modification of the AI / ML model by the 0AM. Since the Model Inference function is located at the RAN, the 0AM cannot know which UEs are involved in or affected by AI / ML-based optimization / control (including conventional optimization / control enhanced by AI / ML predictions). 0AM is thus at risk of selecting incorrect UE(s) as data source(s), limiting its abilities to improve the AI / ML model or even monitor the AI / ML model performance.

[0208] Accordingly, embodiments of the present disclosure provide flexible and efficient techniques for setting up data collection for AI / ML performance monitoring, AI / ML model training, and similar purposes using the trace / MDT framework, which is enhanced to enable RAN-based MDT configurations in addition to conventional management- and signaling-based MDT configurations. In these techniques, a RAN node can select relevant UEs for performing the requested measurements and reporting the requested information based on up-to-date RAN information and / or conditions, potentially combined with further relevant information received from CN (e.g., 5GC), management system, or other RAN nodes. In some embodiments, the RAN node can base this selection on additional information such as current UE location, measured UE traffic, predicted UE traffic, etc. In some embodiments, a RAN node can configure and / or activate MDT functionality in selected UE(s) independently, e.g., without receiving (or receiving only partial) MDT activation and / or configuration from CN or management system.

[0209] At a high level, the RAN node can perform one or more of the following operations in various embodiments:

[0210] • Identifying the relevant AI / ML model and / or AI / ML use case based on the Trace activation message received from the management system or independently (e.g., without receiving such a message fro m a management system node)

[0211] • Storing AI / ML model and / or AI / ML use case relevant input as received from one or more of: a core network node, another RAN node, and a management system node, to be used as AI / ML UE selection assistance information.

[0212] • Using stored AI / ML-related indicators to select individual UEs or group of UEs to collect data (e.g., for AI / ML model training) using the MDT data collection framework.

[0213] • Receiving and using conditions for collecting MDT data, e.g., to be used for training / retraining / monitoring / inference of AI / ML model(s) (with / without any information to identify a specific AI / ML model).

[0214] • Receiving and using indications to start / stop / pause / resume collection of MDT data and measurements, e.g., for training / retraining an AI / ML model (with / without any information to identify a specific AI / ML model).

[0215] • Receiving and using indications to start / stop / pause / resume reporting of MDT data and measurements, e.g., for training / retraining an AI / ML model (with / without any information to identify a specific AI / ML model). Various embodiments may also facilitate OAM collection of data as input to an AI / ML process to enable

[0216] AI / ML inference. In such embodiments, the OAM may host the Model Inference function and the OAM collects input data via MDT from the RAN and from UEs served by the RAN.

[0217] Various embodiments may also facilitate OAM collection of feedback data to evaluate performance of an AI / ML model and make decisions about updating, replacing, removing, etc. the AI / ML model. In such embodiments, the OAM may host the Model Inference function or the OAM is in responsible for monitoring performance of an AI / ML model and instructing other parts of the system on actions that depend on the impact of the AI / ML model on the overall system performance.

[0218] Various embodiments may also be used when the OAM hosts the Model T raining function and a RAN node hosts the Model Inference function, or when a RAN node hosts both Model Training and Model Inference functions. For the latter case, RAN-based MDT enables the RAN node to collect data for training / retraining an AI / ML model without receiving an MDT activation message from a management system. This makes it possible for the RAN node to reuse MDT to collect data (e.g., UE performance feedback) that is or will be available via MDT.

[0219] Embodiments can provide various benefits, advantages, and / or solutions to problems described herein. For example, embodiments facilitate the use of conventional MDT / Trace framework for the collection of training data, input data, and feedback information for AI / ML models by enhancing and / or addressing problems with the existing framework. At the same time, embodiments can leverage the MDT framework - which was designed to convey data from RAN to OAM in an efficient way - for new purposes and / or applications related to AI / ML. Embodiments also facilitate a granular selection of AI / ML related data for collection by OAM and / or RAN, such as required input data from correct UE(s) for AI / ML model inference at the RAN, and new training data (including feedback data) for AI / ML model training at the RAN. At a high level, embodiments facilitate intelligent RAN implementations based on AI / ML.

[0220] In the following description of embodiments, the following groups of terms and / or abbreviations have the same or substantially similar meanings and, as such, are used interchangeably and / or synonymously unless specifically noted or unless a different meaning is clear from a specific context of use:

[0221] • "training”, "optimizing”, "optimization”, and "updating” of models;

[0222] • "changing” and "modifying” of models, which are used to indicate that a type, structure, parameters, connectivity, etc. of a model is different than it was before the change or modification;

[0223] • "model”, "policy”, "algorithm”, "AI / ML model”, "AI / ML policy”, "AI / ML algorithm”. In general, embodiments disclosed herein are applicable to any type of AI / ML algorithm, model, and / or technique used in a RAN. Non-limiting examples include supervised learning, deep learning, reinforcement learning, contextual multi-armed bandit algorithms, autoregression algorithms, etc. or combinations thereof . Such algorithms may exploit functional approximation models, also referred to as AI / ML models, including feedforward neural networks, deep neural networks, recurrent neural networks, convolutional neural networks, etc.

[0224] Specific examples of reinforcement learning algorithms include deep reinforcement learning algorithms such as deep Q-network (DQN), proximal policy optimization (PPO), double Q -learning, policy gradient algorithms, off-policy learning algorithms, actor-critic algorithms, and advantage actor-critic algorithms (e.g., A2C, A3C, actorcritic with experience replay, etc.).

[0225] The embodiments summarized above will now be described in more detail . In general, for MDT-based collection of training data, input data, and feedback data for an AI / ML model, UE selection can be performed by a RAN node (e.g., a gNB) based on AI / ML-related indicators received from a management system node, a CN node, another RAN node, or a UE. The RAN can also use these AI / ML-related indicators to identify which RAN-related measurements are relevant for AI / ML purposes.

[0226] In some embodiments, the AI / ML-related indicators can be included in a message that activates the MDT configuration at the RAN (e.g., trace activation message over the NG interface). The receiving RAN node extracts the AI / ML-related indicators from the received message, stores them, and uses them to determine which data are to be collected from UEs and / or from the RAN and how to execute the data collection process.

[0227] In various embodiments, the AI / ML-related indicators may include one or more of the following:

[0228] • AI / ML Model ID, or another identification that allows the RAN to deduce the data needed by the OAM for AI / ML purposes;

[0229] • one or more AI / ML use cases, such as Network Energy Saving, Power Saving, Load Balancing Optimization, Mobility Optimization, Link Adaptation, QoS Optimization, QoE Optimization, Coverage and Capacity Optimization, MIMO Layer Optimization, CSI Prediction, Beam management, Positioning, Channel Coding, Reference Signal Enhancements, Interference Optimization;

[0230] • data collection areas, e.g., cells, PLMNs, tracking areas, RAN nodes, geographical areas, geolocations, public network identifiers, non-public networks identifiers, etc.;

[0231] • list of data to be collected for AI / ML processes; and

[0232] • information related to the data to be collected.

[0233] In some embodiments, the list of data to be collected for AI / ML processes can include one or more of the following needed from the RAN node serving the UEs involved in the measurements:

[0234] • Current and / or predicted resource status of the RAN node, one or more cells provided by RAN node, and / or one or more beams (e.g., SSB) provided by the RAN node;

[0235] • Predicted path or trajectory for one or more UEs;

[0236] • Current and / or predicted UE traffic for one or more UEs;

[0237] • Current and / or predicted resource status for neighbor cells, beams, and / or RAN nodes;

[0238] • Current and / or predicted energy efficiency of the RAN node, one or more cells provided by the RAN node, and / or one or more beams provided by the RAN node; and

[0239] • Current and / or predicted energy consumption of the RAN node, one or more cells provided by the RAN node, and / or one or more beams provided by the RAN node.

[0240] In some embodiments, the list of data to be collected for AI / ML processes can include one or more of the following needed from the UEs involved in the measurements:

[0241] • UE location information (e.g., coordinates, serving cell ID, moving velocity) interpreted by RAN node implementation when available;

[0242] • UE Mobility History Information;

[0243] • UE measurement report (e.g., RSRP, RSRQ, SI NR, etc.) for cell and beam measurements;

[0244] • Current and / or predicted UE energy efficiency; and

[0245] • Current and / or predicted UE energy consumption. In some embodiments, the list of data to be collected for AI / ML processes can include one or more of the following needed from neighbor RAN nodes:

[0246] • Current and / or predicted resource status of the neighbor RAN node, one or more cells provided by neighbor RAN node, and / or one or more beams (e.g., SSB) provided by the neighbor RAN node;

[0247] • UE performance measurement after the UE has been offloaded to a neighbor cell;

[0248] • Current and / or predicted energy efficiency of the neighbor RAN node, one or more cells provided by neighbor RAN node, and / or one or more beams (e.g., SSB) provided by the neighbor RAN node;

[0249] • Current and / or predicted energy consumption of the neighbor RAN node, one or more cells provided by neighbor RAN node, and / or one or more beams (e.g., SSB) provided by the neighbor RAN node;

[0250] • UE's history information from neighbor RAN nodes and cells.

[0251] In some embodiments, the information related to the data to be collected can include one or more of the following:

[0252] • number of data samples (e.g., fixed amount, continuously until a stop signal, etc.). For example, a fixed number of samples can be specified by a number of UEs, a number of events (e.g., RLF, successful HO), a number of actions (e.g., HO, cell deactivation), or any combination thereof.

[0253] • time period where to collect the data, e.g., next 30 minutes, during 8:00AM-9:00AM, on Mondays, continuously, etc.

[0254] • type of data traffic, e.g., web browsing, video call, video streaming.

[0255] • required or requested variation of the data samples, such as one or more of the following: o minimum time interval between samples o minimum distance between sample locations (e.g., due to the involved UEs, location of events, etc.), which can be specified geographically or based on transmission channel characteristics (e.g., SI NR) o differences in hardware / software characteristics of involved nodes (e.g., RAN nodes, UEs). For example, all the samples should come from UEs with chipset vendor X, at least 50% of samples should come from non-5G-capable UEs, samples should only include 5G-capable UEs handed-over to RAN nodes of vendor Y, etc. o differences in load / resource utilization at RAN node. For example, only sample when there is 100% PRB utilization, samples should be nearly uniformly distributed over a range of 30%-70% of PRB utilization, etc.

[0256] Figure 9 shows an exemplary procedure for configuration and activation of RAN -based MDT with 5GC, according to various embodiments of the present disclosure . The RAN can be NG-RAN, E-UTRAN (for LTE), UTRAN (for UMTS), etc. Although certain operations in Figure 9 are given numerical labels, this is done to facilitate explanation rather than to require or imply any sequential order, unless expressly stated otherwise.

[0257] Operation 0 can be considered a pre-requisite or pre-condition. Whenever a RAN node (940) receives AI / ML- related indicators that can be used for selecting a UE (950) or group of UEs for collection of data for AI / ML purposes, the RAN node stores the AI / ML-related indicators with the respective UE contexts for possible later usage. For example, the RAN node can receive such information for each UE in an Initial Context Setup Request message, a Handover Request message, or in any other relevant message from the CN (e.g., from AMF 920). The UE selection process can also involve user consent indicators, UE capabilities, etc., from the CN as well as UE presence, UE traffic predictions, and other more dynamic RAN-based information (discussed in more detail below). The UE selection process can also include an initial default group.

[0258] In operation 1 , the management system (910) sends a trace session activation request (or similar message) to the RAN node. In addition to any of the parameters described above in relation to Figure 7 operation 1 , this request can also include one or more of the following parameters:

[0259] • Indication that measurements are collected for an operation concerning an AI / ML model (the indication only concerning UE measurements, or only concerning RAN measurements, or concerning both UE and RAN measurements). • Indication that measurements of a specific measurement type are collected for an operation concerning an AI / ML model (the indication only concerning UE measurements, or only concerning RAN measurements, or concerning both UE and RAN measurements).

[0260] • Type of operation related to the AI / ML model (e.g., training, re-training, modification, inference, performance evaluation, performance monitoring, input collection, feedback collection).

[0261] • AI / ML Model ID, for the AI / ML model for which the measurements should be collected.

[0262] • Other AI / ML model differentiators, allowing the identification of measurements to be collected for specific AI / ML processes.

[0263] • Indications to start / stop / pause / resume measurements (RAN, UE, or both) for collecting data to be used in relation to the AI / ML model (e.g., for training).

[0264] AI / ML model differentiators can include model version, model vendor, and scope configuration parameters targeted by the AI / ML model. Some example scope configuration parameters include:

[0265] • AI / ML use case(s), e.g., Network Energy Saving, Mobility Optimization, Load Balancing, Link Adaptation, QoS Optimization, QoE Optimization, Coverage and Capacity Optimization, Ml MO Layer Optimization, CSI estimation, beam management, positioning accuracy enhancements, Channel Coding, Reference Signal Enhancements, Interference Optimization, etc.;

[0266] • AI / ML model type;

[0267] • whether requested data is for AI / ML inputs, outputs, or feedback information;

[0268] • data requested from UEs;

[0269] • priorities or relative weights between data requested from UEs;

[0270] • data requested from RAN nodes;

[0271] • priorities or relative weights between data requested from RAN nodes;

[0272] • any of the information related to the data to be collected, discussed above; and

[0273] • RAN node reporting format.

[0274] The criteria for which parameters are present can depend on one or more of the AI / ML model differentiators and other indication concerning the AI / ML model, such as the exact use case(s) to which the AI / ML model is applicable, the type of operation related to the AI / ML model, etc.

[0275] Based on the message received in operation 1 , the RAN may collect AI / ML information only for the specified AI / ML model and / or only for AI / ML models related to the specified AI / ML use cases. Likewise, the RAN may select UEs that are affected by the specified AI / ML model and / or by AI / ML models related to the specified AI / ML use cases. Similarly, the RAN node may collect only the information requested in the message and may select UEs based on the requested information.

[0276] In some embodiments, the TR parameter can also be used to communicate some of the above-listed information that can be included in the message of operation 1 , such as the AI / ML model, relevant AI / ML use case, etc. For example, a subset of available TR values can be used to identify AI / ML specific measurement collection processes.

[0277] In general, the RAN may perform various actions (e.g., optimizations) based on outputs of an AI / ML model. To improve the AI / ML model (e.g., via training, re-training, modification, etc.), information about these actions can be collected and provided to the AI / ML Model Training function. The collected information can include feedback associated with the actions and / or feedback associated with outputs of the AI / ML model (i.e., th at triggered the actions). Embodiments described herein can be utilized for collecting such information.

[0278] In some embodiments, the message in operation 1 can include one or more output feedback identifiers, each of which uniquely identifies an output of an AI / ML model or an action taken by a RAN node based on an output of an AI / ML model. Each output feedback identifier may include and / or identify one or more of the following:

[0279] • the relevant output or action taken, e.g., an action index into a predefined set of actions;

[0280] • AI / ML model(s) relevant to the output or action taken, e.g., specified by AI / ML Model ID and optionally version;

[0281] • AI / ML use case(s) relevant to the output or action taken;

[0282] • time relevant to the output or action taken, e.g., at which hour, day, month, and / or year, at which hour of the day, day of the week, etc.; • area scope for the output or action taken, e.g., in which cell, RAN node, tracking area, RAN-based notification area, PLMN, etc.;

[0283] • RAN node(s) affected by the output or action taken; and

[0284] • network slice affected by the output or action taken.

[0285] In some variants, the RAN node does not need to know which of the above information that an output feedback identifier includes. Alternately, if one or more output feedback identifiers are included in the trace session activation request message in operation 1 , the RAN node can / should consider the included output feedback identifiers (along with other information in the message) when selecting UEs for collecting data (e.g., UE feedback) for AI / ML purposes in operation 3.

[0286] In operation 2, after RAN receives the trace session activation request from the management system, it starts a Trace Session and saves relevant information associated with the Trace Session. In operation 3, the RAN node selects suitable UEs for MDT data collection for the specific AI / ML model and / or use case parameters. UE selection is based on the relevant information received from the management system, the area where UE is located, and information from the core network and / or other RAN nodes that are relevant for AI / ML model and / or use cases. When RAN node selects a UE, it considers requirements on mandatory parameters for the relevant set of AI / ML model differentiators (e.g., relevant AI / ML use case) as well as UE capability (e.g., MDT capability). UEs that do not satisfy required attributes and / or conditions (e.g., area) are not selected for MDT data collection.

[0287] For example, the RAN node should select all UEs involved in and / or affected by AI / ML-based actions associated with or identified by the one or more output feedback identifiers received. If the identified action was performed by a second RAN node, the RAN node should receive the output feedback identifier from the second RAN node, e.g., as part of a handover preparation procedure or another procedure between the two RAN nodes.

[0288] In some embodiments, a RAN node can activate the MDT functionality in selected UE(s) without receiving a fully specified trace session activation request from the management system (as stated above). In some variants, the RAN node receives an MDT activation indication having a minimum set of information and conditions for MDT configurations, enabling the RAN node to configure UEs for MDT data collection as needed, e.g., for AI / ML purposes.

[0289] In other variants, the RAN node receives a partially specified trace session activation request from the management system, allowing the RAN node to decide on the remaining parts of MDT configuration for the selected UEs. For example, certain MDT configuration parameters (or lEs) in a trace session activation request can be marked as "configurable by RAN node” or in a similar way.

[0290] As a specific example, the management system sends the RAN node a trace session activation request that includes the TCE IP address but specifies only a minimum set of feedback information needed to evaluate and monitor the AI / ML model performance. This allows the RAN node to configure UE(s) to perform additional MDT measurements and / or report additional MDT data, if required for RAN-based AI / ML model inference and / or retraining.

[0291] In operation 4, the RAN node activates the MDT functionality to the selected UE or identified group of UEs. The RAN node assigns a TRSR corresponding to the selected UE, with the same TRSR being assigned to all UEs of a UE group for the relevant MDT data collection. In various embodiments, the MDT activation sent in operation 4 can include any of the same information described above in relation to Figure 7 operation 4 for logged and immediate MDT cases. As mentioned above, the RAN node cannot send the logged MDT configuration to UEs currently camping in the cell in RRCJDLE / RRCJNACTIVE. These UEs may be configured when they next initiate some activity (e.g., Service Request or Tracking Area Update) that causes them to transition to RRC_CONNECTED.

[0292] In some embodiments, the TRSR sent in operation 4 can include or relate to an output feedback identifier, such as discussed above. If the action was performed (or output produced) by a RAN node other than the RAN node configuring UEs for MDT measurements, the configuring RAN node receives such an identifier from the performing RAN node, e.g., as part of handover signaling when the output / action leads to a handover of the one or more UEs. In some variants, the output feedback identifier defines a group of UEs that are assigned the same TRSR.

[0293] In operation 5, when the UE receives the MDT activation it starts the MDT functionality based on the received configuration parameters and can start MDT reporting using methods relevant to the use case. This can be done in a similar way as Figure 7 operation 5, described above. However, the RAN node shall not retrieve MDT report from the UE if UE's registered PLMN (rPLMN) does not match the PLMN where TCE (960) used to collect MDT data resides (e.g., RAN node's primary PLMN). When the RAN node receives the MDT report from UE, the RAN node obtains TRSR, TR, and TCE Id from the report and compare the Trace PLMN (i.e., PLMN portion of TR) with the PLMN where TCE used to collect MDT data resides (e.g., its primary PLMN). The RAN node discards the MDT report in case of a mismatch.

[0294] In operation 5a, in certain AI / ML use cases, the RAN node may receive MDT reporting from other RAN nodes (930) via Xn interface (or other interface used between nodes in a different RAN). In operation 6, after receiving the MDT reports, the RAN node can use the information therein to update its records about UE indexed information such as actual UE location, actual UE traffic, predicted UE location, predicted UE traffic, etc.

[0295] In operation 6a (which can be optional), the RAN node performs AI / ML model training (using Model Training function). If training criteria are not satisfied, the RAN node repeats operations 3-6 until the training criteria are satisfied, after which the RAN node and other network entities shown perform operations 7-11 described above in relation to Figure 7.

[0296] In some embodiments, the TCE can be a RAN node, such that MDT reports can also be forwarded to a neighbor RAN node. This is useful for the case where AI / ML based actions lead to mobility / handover of UEs between RAN nodes and the source RAN node (taking the action) needs certain UE information (e.g., feedback) from the target RAN node.

[0297] In some embodiments, a RAN node (e.g., gNB) can activate MDT functionality in association with data collection in one or more selected UEs for a specific AI / ML model without receiving any specified trace session activation request from the management system or CN. In this case, the RAN node determines to activate the MDT functionality independently from a Management System node or a CN node. For example, a RAN node hosting a Model Inference and / or a Model Training function for the specific AI / ML model may make such a determination to activate the MDT functionality.

[0298] In some variants, the RAN node may send a first message to inform a second node (e.g., a Management System node or a CN node) about the independent activation of the MDT data collection functionality for the specific AI / ML model. The RAN node may also send the second node a configuration (or related information) used for the independently activated MDT data collection functionality. Examples of information provided to the second node can include one or more of the following:

[0299] • an indication that MDT data collection functionality has been independently activated by the RAN node;

[0300] • an identifier associated with the AI / ML model (optionally including a version thereof) for which the MDT functionality was activated;

[0301] • a list of one or more UE identities for which user consent is required for the activation of the MDT functionality; and

[0302] • a list of areas in which the MDT functionality was activated, such as a list of cells.

[0303] In some variants, the RAN node may request permission from the second node to activate MDT functionality in association with data collection in one or more selected UEs for a specific AI / ML model, prior to activation. Once permission is given, however, the RAN node performs activation without a trace session activation request from the management system or CN.

[0304] In different variants, the RAN node receives a second message from the second node (e.g., a Management System node or a CN node) including one or more of the following:

[0305] • positive acknowledgment (or permission) for the requested activation of the MDT functionality;

[0306] • negative acknowledgment (or no permission) for the requested activation of the MDT functionality;

[0307] • assistance information for the requested activation of the MDT functionality, such as additional configuration information described above in relation to various embodiments;

[0308] • consent information for one or more UE identities.

[0309] • request and / or configuration for reporting data collected after activation of the MDT functionality (which may be an implicit positive acknowledgement / permission).

[0310] In some variants, the RAN node may transmit a third message to the second node to report data collected by one or more selected UEs for a specific AI / ML model, after activation of the MDT functionality.

[0311] In other embodiments, which may be used independent from or in combination with embodiments described above, a network node that is not part of the RAN (e.g., an OAM node, an SMO node, etc.) sends to one or more RAN nodes an MDT configuration for the purpose of collecting data relevant for AI / ML used in the RAN. For example, an OAM node sends to RAN nodes an MDT configuration comprising one or more or a combination of the following:

[0312] • an indication that the MDT measurements are for collecting data (e.g., training data) related to use of AI / ML in the RAN, e.g., AI / ML use cases involving AI / ML models;

[0313] • identifiers of one or more relevant AI / ML-based use cases;

[0314] • information associated with one or more relevant AI / ML models (e.g., model ID, model vendor, model type, model version, etc.)

[0315] • indication to start / stop / pause / resume collection of MDT measurements

[0316] The MDT configuration in question can be signaling-based, management-based, or any kind of MDT configuration.

[0317] In various embodiments, a RAN node reporting data collected according to any of the techniques described above can send MDT reports that include one or more of the following:

[0318] • an indication that the data was collected for use of AI / ML in the RAN;

[0319] • an indication that the data was collected for specific AI / ML use cases;

[0320] • an indication that the data was collected for a specific AI / ML model (optionally model vendor and / or version); and

[0321] • an indication that the data was for a specific AI / ML Model type.

[0322] Embodiments were described above in the context of MDT measurements, which generally include radiolayer measurements made by UEs (or RAN nodes). Even so, measurements collected for operations on AI / ML models can also include application-layer measurements, such as UE measurements related to quality of experience (QoE). Note that in this context, radio- or application-layer "measurements” may include not only measured values but also predicted values (i.e., predictions).

[0323] Various features of the embodiments described above correspond to various operations illustrated in Figure 10, which shows an exemplary method {e.g., procedures) performed by a RAN node for collecting data to be used with AI / ML models, according to various embodiments of the present disclosure. In other words, various features of the operations described below correspond to various embodiments described above. Although Figure 10 shows specific blocks in a particular order, the operations of the exemplary method can be performed in a different order than shown and can be combined and / or divided into blocks having different functionality than shown. Optional blocks or operations are indicated by dashed lines. The exemplary method can be performed by a RAN node {e.g., base station, eNB, gNB, ng-eNB, etc. or unit / function thereof) such as described elsewhere herein.

[0324] The exemplary method can include the operations of one or more (e.g., some or all) of blocks 1030-1060. In block 1030, the RAN node can receive, from a network node or function (NNF) outside of the RAN, a first request to activate a trace session for measurements by one or more UEs in the RAN. The first request includes information indicating that the measurements are associated with one or more AI / ML models. In block 1040, based on the information received with the first request, the RAN node can select one or more UEs served by the RAN node to perform the requested measurements. In block 1050, the RAN node can send to the selected UEs respective second requests to perform the measurements for the trace session. In block 1060, the RAN node can receive, from the selected UEs, respective measurement reports comprising results of the measurements performed in accordance with the respective second requests.

[0325] In some embodiments, the exemplary method can also include the operations of block 1080, where the RAN node can perform at least one operation on the one or more AI / ML models based on the results of the measurements. In some of these embodiments, one of the following operations is performed for each of the AI / ML model based on the results of the measurements: model training, model re-training, continued model training based on a partially trained model received from the NNF, model modification, model inference, model performance evaluation, model performance monitoring, input collection, and feedback collection.

[0326] In some embodiments, the measurements include radio-layer measurements and / or application-layer measurements.

[0327] In some embodiments, the exemplary method can also include the operations of blocks 1010-1015. In block 1010, the RAN node can receive, from a core network node, a plurality of service requests associated with a corresponding plurality of UEs. Each service request includes one or more AI / ML-related indicators relevant for selection of the corresponding UE to perform measurements associated with AI / ML models. In block 1015, the RAN node can store the one or more AI / ML-related indicators for the respective UEs. In some of these embodiments, the one or more AI / ML-related indicators are stored in or with respective UE contexts.

[0328] In some of these embodiments, the measurements are MDT measurements, the NNF outside of the RAN is a network management system (e.g., OAM), the core network node is an AMF, and each service request is an initial context setup request or a handover request.

[0329] In some variants, the MDT measurements are logged MDT measurements, and the first request and each second request include or indicate the same one or more of the following information:

[0330] • a trace reference;

[0331] • an area scope where UE measurements should be collected;

[0332] • a logging interval;

[0333] • a logging duration;

[0334] • an address (e.g., IP) or identifier of a TCE; and

[0335] • a list of PLMNs for which measurements are requested.

[0336] In other variants, the MDT measurements are immediate MDT measurements, and the first request and each second request include or indicate the same one or more of the following information: a list of measurements, a reporting trigger, a reporting interval, a reporting amount, and an event threshold.

[0337] In some embodiments, the information received with the first request includes one or more of the following:

[0338] • an indication that the requested measurements are for one or more operations to be performed on the one or more AI / ML models;

[0339] • an indication of the one or more operations to be performed;

[0340] • respective identifiers of the AI / ML models;

[0341] • other information that differentiates the respective AI / ML models;

[0342] • an indication of whether the requested measurements are UE measurements, RAN node measurements, or both;

[0343] • one or more indications or commands for controlling the requested measurements; and

[0344] • one or more output feedback identifiers for which feedback is requested.

[0345] In some of these embodiments, each output feedback identifier identifies one or more of the following:

[0346] • an output or action taken;

[0347] • one or more AI / ML models relevant to the output or action taken;

[0348] • one or more AI / ML use cases relevant to the output or action taken;

[0349] • time relevant to the output or action taken;

[0350] • area scope for the output or action taken;

[0351] • one or more RAN nodes affected by the output or action taken; and

[0352] • one or more network slices affected by the output or action taken.

[0353] In some of these embodiments, the other information that differentiates the respective AI / ML models includes one or more of the following: model version, model vendor, and one or more scope configuration parameters. In some variants, the one or more scope configuration parameters include one or more of the following:

[0354] • one or more AI / ML use cases;

[0355] • AI / ML model type;

[0356] • whether requested data is for AI / ML model inputs, outputs, or feedback;

[0357] • data requested from UEs;

[0358] • priorities or relative weights between data requested from UEs;

[0359] • data requested from RAN nodes;

[0360] • priorities or relative weights between data requested from RAN nodes;

[0361] • a configuration for data collection and / or reporting. In some further variants, the configuration for data collection and reporting includes one or more of the following:

[0362] • number of data samples to be collected;

[0363] • time period during which the data samples are to be collected;

[0364] • one or more types of user data sessions for which the data samples are to be collected;

[0365] • a minimum time interval between collected data samples

[0366] • a minimum distance between locations associated with collected data sample locations

[0367] • hardware and / or software characteristics of UEs and / or RAN nodes from which the data samples are to be collected; and

[0368] • characteristics of load and / or resource utilization of RAN nodes from which the data samples are to be collected.

[0369] In some further variants, the data requested from UEs includes one or more of the following:

[0370] • UE location information;

[0371] • UE mobility history information;

[0372] • UE cell or beam measurements;

[0373] • current and / or predicted UE energy efficiency; and

[0374] • current and / or predicted UE energy consumption.

[0375] In some further variants, the data requested from RAN nodes includes one or more of the following:

[0376] • current and / or predicted values of one or more of the following for the RAN node and / or for one or more neighbor RAN nodes: resource status, energy efficiency, energy consumption;

[0377] • current and / or predicted values of one or more of the following for one or more cells provided by the RAN node and / or by one or more neighbor RAN nodes: resource status, energy efficiency, energy consumption;

[0378] • current and / or predicted values of one or more of the following for one or more beams (e.g., SSB) provided by the RAN node and / or by one or more neighbor RAN nodes: resource status, energy efficiency, energy consumption;

[0379] • predicted path or trajectory for one or more UEs served by the RAN node; and

[0380] • current and / or predicted traffic for one or more UEs served by the RAN node.

[0381] In some further variants, the AI / ML use cases include one or more of the following: network energy savings, mobility optimization, load balancing, link adaptation, quality-of-service (QoS) optimization, quality-of-experience (QoE) optimization, coverage and capacity optimization (CCO), multi-input multi-output (MIMO) layer optimization, channel state information (CSI) estimation, beam management, positioning accuracy enhancement, channel coding, reference signal enhancements, and interference optimization.

[0382] In some of these embodiments, each UE served by the RAN node is associated with one or more AI / ML- related indicators relevant for selection of the UE to perform measurements associated with AI / ML models. In such case, selecting one or more UEs based on the information received with the first request in block 1040 includes the operations of sub-block 1041, where the RAN node select, from a plurality of UEs served by the RAN node, UEs with associated AI / ML-related indicators that match or correspond to one or more of the following: at least one of the identifiers of the AI / ML models, at least one scope configuration parameter that differentiates the respective AI / ML models, and at least one output feedback identifier.

[0383] In some of these embodiments, for each AI / ML model, the indicated operation to be performed is one of the following: training, re-training, modification, inference, performance evaluation, performance monitoring, input collection, and feedback collection.

[0384] In some embodiments, selecting the one or more UEs in block 1040 is further based on one or more of the following: UE MDT capability, UE location, user consent indicators, and predicted UE traffic.

[0385] In some embodiments, the exemplary method can also include the operations of block 1090, where the RAN node can send to a TCE a trace record comprising the results of the measurements performed by the one or more UEs. In some of these embodiments, the trace record also includes an indication that the results of the measurements were collected for one or more of the following: use of AI / ML in the RAN, one or more specific AI / ML use cases, one or more specific AI / ML models, and one or more specific AI / ML model types. In some of these embodiments, the trace record is also sent to one or more of the following: a network management system (e.g., OAM), a core network node (e.g., AMF), and one or more other RAN nodes (e.g., neighbor RAN nodes).

[0386] In some of these embodiments, the exemplary method can also include the following operations, labelled with corresponding block numbers in parentheses:

[0387] • (1065) based on the information received with the first request, selecting one or more other RAN nodes to perform the requested measurements;

[0388] • (1070) sending, to the selected RAN nodes, respective third requests to perform the measurements for the trace session; and

[0389] • (1075) receiving, from the selected RAN nodes, respective measurement reports comprising results of the measurements performed in accordance with the respective third requests.

[0390] In such case, the trace record sent to the TCE in block 1090 also includes the results of the measurements performed by the one or more RAN nodes.

[0391] In some embodiments, the exemplary method can also include the operations of block 1020, where the RAN node can send, to the NNF outside of the RAN, a further request to activate the trace session for measurements associated with one or more AI / ML models. In such case, the first request is received (e.g., in block 1030) as a permission or a positive acknowledgement in response to the further request. In some of these embodiments, the further request includes a list of one or more UE identifiers for which user consent is required for the activation of the trace session, and the first request includes an indication of user consent to activate the trace session for the one or more UE identifiers.

[0392] Although various embodiments are described above in terms of methods, techniques, and / or procedures, the person of ordinary skill will readily comprehend that such methods, techniques, and / or procedures can be embodied by various combinations of hardware and software in various systems, communication devices, computing devices, control devices, apparatuses, non-transitory computer-readable media, computer program products, etc.

[0393] Figure 11 shows an example of a communication system 1100 in accordance with some embodiments. In this example, communication system 1100 includes a telecommunication network 1102 that includes an access network 1104 (e.g., RAN) and a core network 1106, which includes one or more core network nodes 1108. In some embodiments, telecommunication network 1102 can also include one or more Network Management (NM) nodes 1018, which can be part of an operation support system (OSS), a business support system (BSS), or an operations / administration / maintenance (OAM) system. NM nodes 1018 can monitor operations of other nodes in access network 1004 and core network 1006.

[0394] Access network 1104 includes one or more access network nodes, such as network nodes 1110a-b (one or more of which may be generally referred to as network nodes 1110), or any other similar 3GPP access node or non- 3GPP access point. Network nodes 1110 facilitate direct or indirect connection of UEs, such as by connecting UEs 1112a-d (one or more of which may be generally referred to as UEs 1112) to core network 1106 over one or more wireless connections.

[0395] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, communication system 1100 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. Communication system 1100 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0396] UEs 1112 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with network nodes 1110 and other communication devices. Similarly, network nodes 1110 are arranged, capable, configured, and / or operable to communicate directly or indirectly with UEs 1112 and / or with other network nodes or equipment in telecommunication network 1102 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in telecommunication network 1102.

[0397] In the depicted example, core network 1106 connects network nodes 1110 to one or more hosts, such as host 1116. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. Core network 1106 includes one or more core network nodes (e.g., 1108) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 1108. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0398] Host 1116 may be under the ownership or control of a service provider other than an operator or provider of access network 1104 and / or telecommunication network 1102, and may be operated by the service provider or on behalf of the service provider. Host 1116 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0399] As a whole, communication system 1100 of Figure 11 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE); New Radio (NR); and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) Zig Bee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0400] In some examples, telecommunication network 1102 is a cellular network that implements 3GPP standardized features. Accordingly, telecommunication network 1102 may support network slicing to provide different logical networks to different devices that are connected to telecommunication network 1102. For example, telecommunication network 1102 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC)ZMassive loT services to yet further UEs.

[0401] In some examples, UEs 1112 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to access network 1104 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from access network 1104. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e., being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

[0402] In the example, hub 1114 communicates with access network 1104 to facilitate indirect communication between one or more UEs (e.g., 1112c and / or 1112d) and network nodes (e.g., 1110b). In some examples, hub 1114 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, hub 1114 may be a broadband router enabling access to core network 1106 for the UEs. As another example, hub 1114 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 1110, or by executable code, script, process, or other instructions in hub 1114. As another example, hub 1114 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, hub 1114 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, hub 1114 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which hub 1114 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, hub 1114 acts as a proxy server or orchestrator for the UEs, in particular if one or more of the UEs are low energy loT devices.

[0403] Hub 1114 may have a constant / persistent or intermittent connection to the network node 1110b. Hub 1114 may also allow for a different communication scheme and / or schedule between hub 1114 and UEs (e.g., 1112c and / or 1112d), and between hub 1114 and core network 1106. In other examples, hub 1114 is connected to core network 1106 and / or one or more UEs via a wired connection. Moreover, hub 1114 may be configured to connect to an M2M service provider over access network 1104 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with network nodes 1110 while still connected via hub 1114 via a wired or wireless connection. In some embodiments, hub 1114 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 1110b. In other embodiments, hub 1114 may be a non-dedicated hub - that is, a device which can route communications between the UEs and network node 1110b, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0404] Figure 12 shows a network node 1200 in accordance with some embodiments. Examples of network nodes include, but are not limited to, access points (e.g., radio access points) and base stations (e.g., radio base stations, Node Bs, eNBs, gNBs).

[0405] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0406] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multistandard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi- cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).

[0407] Network node 1200 includes processing circuitry 1202, memory 1204, communication interface 1206, and power source 1208. Network node 1200 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which network node 1200 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, network node 1200 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1204 for different RATs) and some components may be reused (e.g., a same antenna 1210 may be shared by different RATs). Network node 1200 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1200, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1200.

[0408] Processing circuitry 1202 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 1200 components, such as memory 1204, to provide network node 1200 functionality.

[0409] In some embodiments, processing circuitry 1202 includes a system on a chip (SOC). In some embodiments, processing circuitry 1202 includes radio frequency (RF) transceiver circuitry 1212 and / or baseband processing circuitry 1214. In some embodiments, RF transceiver circuitry 1212 and / or baseband processing circuitry 1214 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 1212 and / or baseband processing circuitry 1214 may be on the same chip or set of chips, boards, or units.

[0410] Memory 1204 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by processing circuitry 1202. Memory 1204 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions (collectively denoted computer program product 1204a) capable of being executed by processing circuitry 1202 and utilized by network node 1200. Memory 1204 may be used to store any calculations made by processing circuitry 1202 and / or any data received via communication interface 1206. In some embodiments, processing circuitry 1202 and memory 1204 is integrated.

[0411] Communication interface 1206 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, communication interface 1206 comprises port(s) / terminal(s) 1216 to send and receive data, for example to and from a network over a wired connection. Communication interface 1206 also includes radio front-end circuitry 1218 that may be coupled to, or in certain embodiments a part of, antenna 1210. Radio front-end circuitry 1218 comprises filters 1220 and amplifiers 1222. Radio front-end circuitry 1218 may be connected to an antenna 1210 and processing circuitry 1202. The radio front-end circuitry may be configured to condition signals communicated between antenna 1210 and processing circuitry 1202. radio front-end circuitry 1218 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection, radio front-end circuitry 1218 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 1220 and / or amplifiers 1222. The radio signal may then be transmitted via antenna 1210. Similarly, when receiving data, antenna 1210 may collect radio signals which are then converted into digital data by radio front-end circuitry 1218. The digital data may be passed to processing circuitry 1202. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0412] In certain alternative embodiments, network node 1200 does not include separate radio front-end circuitry 1218, instead, processing circuitry 1202 includes radio front-end circuitry and is connected to antenna 1210. Similarly, in some embodiments, all or some of RF transceiver circuitry 1212 is part of communication interface 1206. In still other embodiments, communication interface 1206 includes one or more ports or terminals 1216, radio front-end circuitry 1218, and RF transceiver circuitry 1212, as part of a radio unit (not shown), and communication interface 1206 communicates with baseband processing circuitry 1214, which is part of a digital unit (not shown).

[0413] Antenna 1210 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. Antenna 1210 may be coupled to radio front-end circuitry 1218 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, antenna 1210 is separate from network node 1200 and connectable to network node 1200 through an interface or port.

[0414] Antenna 1210, communication interface 1206, and / or processing circuitry 1202 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a UE, another network node and / or any other network equipment. Similarly, antenna 1210, communication interface 1206, and / or processing circuitry 1202 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.

[0415] Power source 1208 provides power to the various components of network node 1200 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). Power source 1208 may further comprise, or be coupled to, power management circuitry to supply the components of network node 1200 with power for performing the functionality described herein. For example, network node 1200 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of power source 1208. As a further example, power source 1208 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0416] Embodiments of network node 1200 may include additional components beyond those shown in Figure 12 for providing certain aspects of the network node's functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, network node 1200 may include user interface equipment to allow input of information into network node 1200 and to allow output of information from network node 1200. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for network node 1200.

[0417] Figure 13 is a block diagram illustrating a virtualization environment 1300 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1300 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized.

[0418] Applications 1302 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in virtualization environment 1300 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0419] Hardware 1304 includes processing circuitry, memory that stores software and / or instructions (collectively denoted computer program product 1304a) executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1306 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1308a and 1308b (one or more of which may be generally referred to as VMs 1308), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 1306 may present a virtual operating platform that appears like networking hardware to VMs 1308.

[0420] VMs 1308 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1306. Different embodiments of the instance of a virtual appliance 1302 may be implemented on one or more of VMs 1308, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[0421] In the context of NFV, each VM 1308 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each VM 1308, and that part of hardware 1304 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1308 on top of hardware 1304 and corresponds to application 1302.

[0422] Hardware 1304 may be implemented in a standalone network node with generic or specific components. Hardware 1304 may implement some functions via virtualization. Alternatively, hardware 1304 may be part of a larger cluster of hardware (e.g., such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1310, which, among others, oversees lifecycle management of applications 1302. In some embodiments, hardware 1304 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 1312 which may alternatively be used for communication between hardware nodes and radio units.

[0423] Figure 14 shows a communication diagram of a host 1402 communicating via a network node 1404 with a UE 1406 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 1112a of Figure 11), network node (such as network node 1110a of Figure 11), and host (such as host 1116 of Figure 11) discussed in the preceding paragraphs will now be described with reference to Figure 14.

[0424] Embodiments of host 1402 include hardware, such as a communication interface, processing circuitry, and memory. Host 1402 also includes software, which is stored in or accessible by host 1402 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as UE 1406 connecting via an over-the-top (OTT) connection 1450 extending between UE 1406 and host 1402. In providing the service to the remote user, a host application may provide user data which is transmitted using OTT connection 1450.

[0425] Network node 1404 includes hardware enabling it to communicate with host 1402 and UE 1406. Connection 1460 may be direct or pass through a core network (like core network 1106 of Figure 11) and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.

[0426] UE 1406 includes hardware and software, which is stored in or accessible by UE 1406 and executable by the UE's processing circuitry. The software includes a client application, such as a web browser or operator-specific "app” that may be operable to provide a service to a human or non-human user via UE 1406 with the support of host 1402. In host 1402, an executing host application may communicate with the executing client application via OTT connection 1450 terminating at UE 1406 and host 1402. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. OTT connection 1450 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through OTT connection 1450.

[0427] OTT connection 1450 may extend via a connection 1460 between host 1402 and network node 1404 and via wireless connection 1470 between network node 1404 and UE 1406 to provide the connection between host 1402 and UE 1406. Connection 1460 and wireless connection 1470, over which OTT connection 1450 may be provided, have been drawn abstractly to illustrate the communication between host 1402 and UE 1406 via network node 1404, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

[0428] As an example of transmitting data via OTT connection 1450, in step 1408, host 1402 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with UE 1406. In other embodiments, the user data is associated with a UE 1406 that shares data with host 1402 without explicit human interaction. In step 1410, host 1402 initiates a transmission carrying the user data towards UE 1406. Host 1402 may initiate the transmission responsive to a request transmitted by UE 1406. The request may be caused by human interaction with UE 1406 or by operation of the client application executing on UE 1406. The transmission may pass via network node 1404, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1412, network node 1404 transmits to UE 1406 the user data that was carried in the transmission that host 1402 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1414, UE 1406 receives the user data carried in the transmission, which may be performed by a client application executed on UE 1406 associated with the host application executed by host 1402.

[0429] In some examples, UE 1406 executes a client application which provides user data to host 1402. The user data may be provided in reaction or response to the data received from host 1402. Accordingly, in step 1416, UE 1406 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input / output interface of UE 1406. Regardless of the specific manner in which the user data was provided, UE 1406 initiates, in step 1418, transmission of the user data towards host 1402 via network node 1404. In step 1420, in accordance with the teachings of the embodiments described throughout this disclosure, network node 1404 receives user data from UE 1406 and initiates transmission of the received user data towards host 1402. In step 1422, host 1402 receives the user data carried in the transmission initiated by UE 1406.

[0430] One or more of the various embodiments improve the performance of OTT services provided to UE 1406 using OTT connection 1450, in which wireless connection 1470 forms the last segment. More precisely, embodiments can facilitate use of conventional MDT / Trace functionality for collection of training data, input data, and feedback information for AI / ML models. At a high level, embodiments facilitate intell igent RAN implementations based on AI / ML, thereby improving RAN performance. As such, OTT services delivered via RANs improved in this manner become more valuable to both end users and service providers.

[0431] In an example scenario, factory status information may be collected and analyzed by host 1402. As another example, host 1402 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, host 1402 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, host 1402 may store surveillance video uploaded by a UE. As another example, host 1402 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, host 1402 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and / or transmitting data.

[0432] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency, and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring OTT connection 1450 between host 1402 and UE 1406, in response to variations in the measurement results. The measurement procedure and / or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of host 1402 and / or UE 1406. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which OTT connection 1450 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of OTT connection 1450 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of network node 1404. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by host 1402. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or 'dummy' messages, using OTT connection 1450 while monitoring propagation times, errors, etc.

[0433] The foregoing merely illustrates the principles of the disclosure. Various modifications and alterations to the described embodiments will be apparent to those skilled in the art in view of the teachings herein. It will thus be appreciated that those skilled in the art will be able to devise numerous systems, arrangements, and procedures that, although not explicitly shown or described herein, embody the principles of the disclosure and can be thus within the spirit and scope of the disclosure. Various embodiments can be used together with one another, as well as interchangeably therewith, as should be understood by those having ordinary skill in the art.

[0434] The term unit, as used herein, can have conventional meaning in the field of electronics, electrical devices and / or electronic devices and can include, for example, electrical and / or electronic circuitry, devices, modules, processors, memories, logic solid state and / or discrete devices, computer programs or instructions for carrying out respective tasks, procedures, computations, outputs, and / or displaying functions, and so on, as such as those that are described herein.

[0435] Any appropriate steps, methods, features, functions, or benefits disclosed herein may be performed through one or more functional units or modules of one or more virtual apparatuses. Each virtual apparatus may comprise a number of these functional units. These functional units may be implemented via processing circuitry, which may include one or more microprocessor or microcontrollers, as well as other digital hardware, which may include Digital Signal Processor (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as Read Only Memory (ROM), Random Access Memory (RAM), cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory includes program instructions for executing one or more telecommunications and / or data communications protocols as well as instructions for carrying out one or more of the techniques described herein. In some implementations, the processing circuitry may be used to cause the respective functional unit to perform corresponding functions according to one or more embodiments of the present disclosure.

[0436] As described herein, device and / or apparatus can be represented by a semiconductor chip, a chipset, or a (hardware) module comprising such chip or chipset; this, however, does not exclude the possibility that a functionality of a device or apparatus, instead of being hardware implemented, be implemented as a software module such as a computer program or a computer program product comprising executable software code portions for execution or being run on a processor. Furthermore, functionality of a device or apparatus can be implemented by any combination of hardware and software. A device or apparatus can also be regarded as an assembly of multiple devices and / or apparatuses, whether functionally in cooperation with or independently of each other. Moreover, devices and apparatuses can be implemented in a distributed fashion throughout a system, so long as the functionality of the device or apparatus is preserved. Such and similar principles are considered as known to a skilled person.

[0437] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs . It will be further understood that terms used herein should be interpreted as having a meaning that i s consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0438] In addition, certain terms used in the present disclosure, including the specification and drawings, can be used synonymously in certain instances (e.g., "data” and "information”). It should be understood, that although these terms (and / or other terms that can be synonymous to one another) can be used synonymously herein, there can be instances when such words can be intended to not be used synonymously.

[0439] Embodiments of the techniques and apparatus described herein also include, but are not limited to, the following enumerated examples:

[0440] A1 . A method performed by radio access network (RAN) node for collecting data to be used with artificial intel ligence / machine learning (AI / ML) models, the method comprising one or more of the following: receiving (1030), from a network node or function (NNF) outside of the RAN, a first request to activate a trace session for measurements by one or more user equipment (UEs) in the RAN, wherein the first request includes information indicating that the measurements are associated with one or more AI / ML models; based on the information received with the first request, selecting (1040) one or more UEs served by the RAN node to perform the requested measurements; sending (1050), to the selected UEs, respective second requests to perform the measurements for the trace session; receiving (1060), from the selected UEs, respective measurement reports comprising results of the measurements performed in accordance with the respective second requests.

[0441] A1 a. The method of embodiment A1 , further comprising performing (1080) at least one operation on the one or more AI / ML models based on the results of the measurements.

[0442] A1 b. The method of embodiment A1 a, wherein one of the following operations is performed for each of the AI / ML model based on the results of the measurements: model training, model re-training, continued model training based on a partially trained model received from the network node, model performance evaluation, model performance monitoring, model inference, model modification, input collection, and feedback collection.

[0443] A1c. The method of any of embodiments A1-A1 b, wherein the measurements include one or more of the following: radio-layer measurements, and application-layer measurements.

[0444] A2. The method of any of embodiments A1-A1 c, further comprising : receiving (1010), from a core network node, a plurality of service requests associated with a corresponding plurality of UEs, wherein each service request includes one or more AI / ML-related indicators relevant for selection of the corresponding UE to perform measurements associated with AI / ML models; and storing (1015) a plurality of UE contexts that include the one or more AI / ML-related indicators for the respective UEs.

[0445] A2a. The method of embodiment A2, wherein: the measurements are minimization of drive testing (MDT) measurements; the NNF outside of the RAN is a network management system; the core network node is an access and mobility management function (AMF); and each service request is one of the following: an initial context setup request, or a handover request.

[0446] A2b. The method of embodiment A2a, wherein the MDT measurements are logged MDT measurements, and the first request and each second request includes or indicates the same one or more of the following information: a trace reference; an area scope where UE measurements should be collected; a logging interval; a logging duration; an address or identifier of a trace collection entity (TCE); and a list of public land mobile networks (PLMNs) for which measurements are requested.

[0447] A2c. The method of embodiment A2a, wherein the MDT measurements are immediate MDT measurements, and the first request and each second request includes or indicates the same one or more of the following information: a list of measurements, a reporting trigger, a reporting interval, a reporting amount, and an event threshold.

[0448] A3. The method of any of embodiments A2-A2c, wherein the one or more UEs are selected from the plurality of UEs based on the one or more AI / ML-related indicators included in the respective UE contexts.

[0449] A4. The method of any of embodiments A1-A3, wherein the information received with the first request includes one or more of the following: an indication that the requested measurements are for one or more operations to be performed on the one or more AI / ML models; an indication of the one or more operations to be performed; respective identifiers of the AI / ML models; other information that differentiates the respective AI / ML models; an indication of whether the requested measurements are UE measurements, RAN node measurements, or both; one or more indications or commands for controlling the requested measurements; and one or more output feedback identifiers, each of which uniquely identifies an AI / ML-based output or action taken therefrom, for which feedback is requested.

[0450] A4a. The method of embodiment A4, wherein the other information that differentiates the respective AI / ML models includes one or more of the following: model version, model vendor, and one or more scope configuration parameters.

[0451] A4b. The method of embodiment A4a, wherein the one or more scope configuration parameters include one or more of the following: one or more AI / ML use cases;

[0452] AI / ML model type; intended use of the requested measurements are information with respect to the AI / ML model; data requested from UEs; priorities or relative weights between data requested from UEs; data requested from RAN nodes; priorities or relative weights between data requested from RAN nodes; and configuration for data collection and / or reporting.

[0453] A4c. The method of embodiment A4b, wherein the configuration for data collection and reporting includes one or more of the following: number of data samples to be collected; time period during which the data samples are to be collected; one or more types of user data sessions for which the data samples are to be collected; a minimum time interval between collected data samples; a minimum distance between locations associated with collected data sample locations; hardware and / or software characteristics of UEs and / or RAN nodes from which the data samples are to be collected; and characteristics of load and / or resource utilization of RAN nodes from which the data samples are to be collected.

[0454] A4d. The method of any of embodiments A4b-A4c, wherein the requested UE data for the AI / ML model include one or more of the following:

[0455] UE location information;

[0456] UE mobility history information;

[0457] UE cell or beam measurements; current and / or predicted UE energy efficiency; and current and / or predicted UE energy consumption.

[0458] A4e. The method of any of embodiments A4b-A4d, wherein the requested RAN node data for the AI / ML model include one or more of the following: current and / or predicted values of one or more of the following for the RAN node and / or for one or more neighbor RAN nodes: resource status, energy efficiency, energy consumption; current and / or predicted values of one or more of the following for one or more cells provided by the RAN node and / or by one or more neighbor RAN nodes: resource status, energy efficiency, energy consumption; current and / or predicted values of one or more of the following for one or more beams provided by the RAN node and / or by one or more neighbor RAN nodes: resource status, energy efficiency, energy consumption; predicted path or trajectory for one or more UEs served by the RAN node; and current and / or predicted traffic for one or more UEs served by the RAN node.

[0459] A4f. The method of any of embodiments A4b-A4e, wherein the one or more AI / ML use cases include one or more of the following: network energy savings, mobility optimization, load balancing, link adaptation, quality-of- service (QoS) optimization, quality-of-experience (QoE) optimization, coverage and capacity optimization (CCO), multi-input multi-output (Ml MO) layer optimization, channel state information (CSI) estimation, beam management, positioning accuracy enhancement, channel coding, reference signal enhancements, and interference optimization.

[0460] A4g. The method of any of embodiments A4-A4f, wherein each output feedback identifier also identifies one or more of the following associated with the output or action taken therefrom: one or more AI / ML models; one or more AI / ML use cases; a time; an area scope; one or more RAN nodes; and one or more network slices.

[0461] A4h. The method of any of embodiments A4-A4g, wherein for each AI / ML model, the indicated operation to be performed is one of the following: training, re-training, modification, inference, performance evaluation, performance monitoring, input collection, and feedback collection.

[0462] A5. The method of any of embodiments A1-A4h, further comprising sending (1090) to a trace collection entity (TCE) a trace record comprising the results of the measurements performed by the one or more UEs.

[0463] A6. The method of embodiment A5, wherein the trace record also includes an indication that the results of the measurements were collected for one or more of the following: use of AI / ML in the RAN, one or more specific AI / ML use cases, one or more specific AI / ML models, and one or more specific AI / ML model types.

[0464] A6a. The method of any of embodiments A5-A6, wherein the trace record is also sent to one or more of the following: a network management system, a core network node, and one or more other RAN nodes.

[0465] A7. The method of any of embodiments A6-A6a, further comprising: based on the information received with the first request, selecting (1065) one or more other RAN nodes to perform the requested measurements; sending (1070), to the selected RAN nodes, respective third requests to perform the measurements for the trace session; receiving (1075), from the selected RAN nodes, respective measurement reports comprising results of the measurements performed in accordance with the respective third requests, wherein the trace record sent to the TCE also includes the results of the measurements performed by the one or more RAN nodes.

[0466] A8. The method of any of embodiments A1-A7, further comprising sending (1020), to the NNF outside of the RAN, a further request to activate the trace session for measurements associated with one or more AI / ML models, wherein the first request is received as a permission or a positive acknowledgement in response to the further request.

[0467] A9. The method of embodiment A8, wherein the further request includes a list of one or more UE identifiers for which user consent is required for the activation of the trace session, and the first request includes an indication of user consent to activate the trace session for the one or more UE identifiers.

[0468] B1 . A radio access network (RAN) node (100, 150, 210, 220, 940, 1110, 1200, 1302 1404) configured to collect data to be used with artificial intelligence / machine learning (AI / ML) models, the RAN node comprising: communication interface circuitry(1206, 1304) configured to communicate with UEs, with other RAN nodes, and with one or more network nodes or functions (NNFs) outside of the RAN; and processing circuitry (1202, 1304) operatively coupled to the communication interface circuitry, whereby the processing circuitry and the communication interface circuitry are configured to perform operations corresponding to any of the methods of embodiments A1-A9.

[0469] B2. A radio access network (RAN) node (100, 150, 210, 220, 940, 1110, 1200, 1302 1404) configured to collect data to be used with artificial intelligence / machine learning (AI / ML) models, the RAN node being configured to perform operations corresponding to any of the methods of embodiments A1-A9. B3. A non-transitory, computer-readable medium (1204, 1304) storing computer-executable instructions that, when executed by processing circuitry (1202, 1304) associated with a radio access network (RAN) node configured to collect data to be used with artificial intelligence / machine learning (AI / ML) models, configure the RAN node to perform operations corresponding to any of the methods of embodiments A1-A9.

[0470] B4. A computer program product (1204a, 1304a) comprising computer-executable instructions that, when executed by processing circuitry (1202, 1304) associated with a radio access network (RAN) node configured to collect data to be used with artificial intelligence / machine learning (AI / ML) models, configure the RAN node to perform operations corresponding to any of the methods of embodiments A1-A9.

Claims

CLAIMS1 . A method performed by radio access network, RAN, node for collecting data to be used with artificial intelligence / machine learning, AI / ML, models, the method comprising: receiving (1030), from a network node or function, NNF, outside of the RAN, a first request to activate a trace session for measurements by one or more user equipment, UEs, in the RAN, wherein the first request includes information indicating that the measurements are associated with one or more AI / ML models; based on the information received with the first request, selecting (1040) one or more UEs served by the RAN node to perform the requested measurements; sending (1050), to the selected UEs, respective second requests to perform the measurements for the trace session; and receiving (1060), from the selected UEs, respective measurement reports comprising results of the measurements performed in accordance with the respective second requests.

2. The method of claim 1, further comprising performing (1080) at least one operation on the one or more AI / ML models based on the results of the measurements.

3. The method of claim 2, wherein one of the following operations is performed for each of the AI / ML models based on the results of the measurements: model training, model re-training, continued model training based on a partially trained model received from the NNF, model performance evaluation, model performance monitoring, model inference, model modification, input collection, and feedback collection.

4. The method of any of claims 1 -3, wherein the measurements include one or more of the following: radiolayer measurements, and application-layer measurements.

5. The method of any of claims 1 -4, further comprising: receiving (1010), from a core network node, a plurality of service requests associated with a corresponding plurality of UEs, wherein each service request includes one or more AI / ML-related indicators relevant for selection of the corresponding UE to perform measurements associated with AI / ML models; and storing (1015) the one or more AI / ML-related indicators for the respective UEs.

6. The method of claim 5, wherein the one or more AI / ML-related indicators are stored in or with respective UE contexts.

7. The method of any of claims 5-6, wherein: the measurements are minimization of drive testing, MDT, measurements; the NNF outside of the RAN is a network management system; the core network node is an access and mobility management function, AMF; and each service request is one of the following: an initial context setup request, or a handover request.

8. The method of claim 7, wherein the MDT measurements are logged MDT measurements, and the first request and each second request include or indicate the same one or more of the following information: a trace reference; an area scope where UE measurements should be collected; a logging interval; a logging duration; an address or identifier of a trace collection entity, TCE; and a list of public land mobile networks, PLMNs, for which measurements are requested.

9. The method of claim 7, wherein the MDT measurements are immediate MDT measurements, and the first request and each second request include or indicate the same one or more of the following information: a list of measurements, a reporting trigger, a reporting interval, a reporting amount, and an event threshold.

10. The method of any of claims 1-9, wherein the information received with the first request includes one or more of the following: an indication that the requested measurements are for one or more operations to be performed on the one or more AI / ML models; an indication of the one or more operations to be performed; respective identifiers of the AI / ML models; other information that differentiates the respective AI / ML models; an indication of whether the requested measurements are UE measurements, RAN node measurements, or both; one or more indications or commands for controlling the requested measurements; and one or more output feedback identifiers for which feedback is requested.11 . The method of claim 10, wherein each output feedback identifier identifies one or more of the following: an output or action taken; one or more AI / ML models relevant to the output or action taken; one or more AI / ML use cases relevant to the output or action taken; time relevant to the output or action taken; area scope for the output or action taken; one or more RAN nodes affected by the output or action taken; and one or more network slices affected by the output or action taken.

12. The method of any of claims 10-11, wherein the other information that differentiates the respective AI / ML models includes one or more of the following: model version, model vendor, and one or more scope configuration parameters.

13. The method of claim 12, wherein the one or more scope configuration parameters include one or more of the following: one or more AI / ML use cases;AI / ML model type; data requested from UEs; priorities or relative weights between data requested from UEs; data requested from RAN nodes; priorities or relative weights between data requested from RAN nodes; whether the requested data is for AI / ML model inputs, outputs, or feedback; and configuration for data collection and / or reporting.

14. The method of claims 13, wherein the configuration for data collection and reporting includes one or more of the following: number of data samples to be collected; time period during which the data samples are to be collected; one or more types of user data sessions for which the data samples are to be collected; a minimum time interval between collected data samples; a minimum distance between locations associated with collected data sample locations; hardware and / or software characteristics of UEs and / or RAN nodes from which the data samples are to be collected; andcharacteristics of load and / or resource utilization of RAN nodes from which the data samples are to be collected.

15. The method of any of claims 13-14, wherein the data requested from UEs includes one or more of the following:UE location information;UE mobility history information;UE cell or beam measurements; current and / or predicted UE energy efficiency; and current and / or predicted UE energy consumption.

16. The method of any of claims 13-15, wherein the data requested from RAN nodes includes one or more of the following: current and / or predicted values of one or more of the following for the RAN node and / or for one or more neighbor RAN nodes: resource status, energy efficiency, energy consumption; current and / or predicted values of one or more of the following for one or more cells provided by the RAN node and / or by one or more neighbor RAN nodes: resource status, energy efficiency, energy consumption; current and / or predicted values of one or more of the following for one or more beams provided by the RAN node and / or by one or more neighbor RAN nodes: resource status, energy efficiency, energy consumption; predicted path or trajectory for one or more UEs served by the RAN node; and current and / or predicted traffic for one or more UEs served by the RAN node.

17. The method of any of claims 13-16, wherein the one or more AI / ML use cases include one or more of the following: network energy savings; mobility optimization; load balancing; link adaptation; quality-of-service, QoS, optimization; quality-of-experience, QoE, optimization; coverage and capacity optimization, CCO; multi-input multioutput, MIMO, layer optimization; channel state information, CSI, estimation; beam management; positioning accuracy enhancement; channel coding; reference signal enhancements; and interference optimization.

18. The method of any of claims 12-17, wherein: each UE served by the RAN node is associated with one or more AI / ML-related indicators relevant for selection of the UE to perform measurements associated with AI / ML models; and selecting one or more UEs based on the information received with the first request comprises selecting, from a plurality of UEs served by the RAN node, UEs with associated AI / ML-related indicators that match or correspond to one or more of the following: at least one of the identifiers of the AI / ML models, at least one scope configuration parameter that differentiates the respective AI / ML models, and at least one output feedback identifier.

19. The method of any of claims 10-18, wherein for each AI / ML model, the indicated operation to be performed is one of the following: training, re-training, modification, inference, performance evaluation, performance monitoring, input collection, and feedback collection.

20. The method of any of claims 1-19, wherein selecting (1040) the one or more UEs is further based on one or more of the following: UE MDT capability, UE location, user consent indicators, and predicted UE traffic.

21. The method of any of claims 1-20, further comprising sending (1090) to a trace collection entity, TCE, a trace record comprising the results of the measurements performed by the one or more UEs.

22. The method of claim 21 , wherein the trace record also includes an indication that the results of the measurements were collected for one or more of the following: use of AI / ML in the RAN, one or more specific AI / ML use cases, one or more specific AI / ML models, and one or more specific AI / ML model types.

23. The method of any of claims 21-22, wherein the trace record is also sent to one or more of the following: a network management system, a core network node, and one or more other RAN nodes.

24. The method of any of claims 21-23, further comprising: based on the information received with the first request, selecting (1065) one or more other RAN nodes to perform the requested measurements; sending (1070), to the selected RAN nodes, respective third requests to perform the measurements for the trace session; receiving (1075), from the selected RAN nodes, respective measurement reports comprising results of the measurements performed in accordance with the respective third requests, wherein the trace record sent to the TCE also includes the results of the measurements performed by the one or more RAN nodes.

25. The method of any of claims 1-24, further comprising sending (1020), to the NNF outside of the RAN, a further request to activate the trace session for measurements associated with one or more AI / ML models, wherein the first request is received as a permission or a positive acknowledgement in response to the further request.

26. The method of claim 25, wherein the further request includes a list of one or more UE identifiers for which user consent is required for the activation of the trace session, and the first request includes an indication of user consent to activate the trace session for the one or more UE identifiers.

27. A radio access network, RAN, node (100, 210, 940, 1110, 1200, 1302, 1404) configured to collect data to be used with artificial intelligence / machine learning, AI / ML, models, the RAN node comprising: communication interface circuitry (1206, 1304) configured to communicate with user equipment, UEs (205, 950, 1112, 1406), with other RAN nodes (150, 220, 930), and with one or more network nodes or functions, NNFs (910, 1108, 1118) outside of the RAN; and processing circuitry (1202, 1304) operatively coupled to the communication interface circuitry, whereby the processing circuitry and the communication interface circuitry are configured to: receive, from an NNF outside of the RAN, a first request to activate a trace session for measurements by one or more UEs in the RAN, wherein the first request includes information indicating that the measurements are associated with one or more AI / ML models; based on the information received with the first request, select one or more UEs served by the RAN node to perform the requested measurements; send, to the selected UEs, respective second requests to perform the measurements for the trace session; and receive, from the selected UEs, respective measurement reports comprising results of the measurements performed in accordance with the respective second requests.

28. The RAN node of claim 27, wherein the processing circuitry and the communication interface circuitry are further configured to perform operations corresponding to any of the methods of claims 2-26.

29. A radio access network, RAN, node (100, 210, 940, 1110, 1200, 1302, 1404) configured to collect data to be used with artificial intelligence / machine learning, AI / ML, models, the RAN node being further configured to: receive, from a network node or function, NNF (910, 1108, 1118) outside of the RAN, a first request to activate a trace session for measurements by one or more user equipment, UEs (205, 950, 1112,1406) in the RAN, wherein the first request includes information indicating that the measurements are associated with one or more AI / ML models; based on the information received with the first request, select one or more UEs served by the RAN node to perform the requested measurements; send, to the selected UEs, respective second requests to perform the measurements for the trace session; and receive, from the selected UEs, respective measurement reports comprising results of the measurements performed in accordance with the respective second requests.

30. The RAN node of claim 29, being further configured to perform operations corresponding to any of the methods of claims 2-26.

31. A non-transitory, computer-readable medium (1204, 1304) storing computer-executable instructions that, when executed by processing circuitry (1202, 1304) associated with a radio access network, RAN, node (100, 210,940. 1110. 1200. 1302. 1404) configured to collect data to be used with artificial intelligence / machine learning, AI / ML, models, configure the RAN node to perform operations corresponding to any of the methods of claims 1-26.

32. A computer program product (1204a, 1304a) comprising computer-executable instructions that, when executed by processing circuitry (1202, 1304) associated with a radio access network, RAN, node (100, 210, 940,1110. 1200. 1302. 1404) configured to collect data to be used with artificial intelligence / machine learning, AI / ML, models, configure the RAN node to perform operations corresponding to any of the methods of claims 1-26.

Images