Online machine learning in a wireless communication system

Abstract

Various aspects of the present disclosure relate to an apparatus for wireless communication. The apparatus may be configured to, capable of, or operable to: receive a request for a trained machine learning model; select at least one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus; request feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication; receive feedback information that includes at least one predicted label of at least one feature; and update at least one trained label of at least one feature of the machine learning model based on the received feedback information.

Description

ONLINE MACHINE LEARNING IN A WIRELESS COMMUNICATIONSYSTEMTECHNICAL FIELD

[0001] The present disclosure relates generally to wireless communication, including online machine learning.BACKGROUND

[0002] A wireless communications system may include one or multiple network communication devices, which may be otherwise knowns as network equipment (NE) supporting wireless communications for one or multiple user communication devices, which may be otherwise known as user equipment (UE), or other suitable terminology. The wireless communications system may support wireless communications with one or multiple user communication devices by utilizing resources of the wireless communication system (e.g., time resources (e.g., symbols, slots, subframes, frames, or the like) or frequency resources (e.g., subcarriers, carriers, or the like). Additionally, the wireless communications system may support wireless communications across various radio access technologies including third generation (3G) radio access technology, fourth generation (4G) radio access technology, fifth generation (5G) radio access technology, among other suitable radio access technologies beyond 5G (e.g., sixth generation (6G)).SUMMARY

[0003] An article “a” before an element is unrestricted and understood to refer to “at least one” of those elements or “one or more” of those elements. The terms “a,” “at least one,” “one or more,” and “at least one of one or more” may be interchangeable. As used herein, including in the claims, “or” as used in a list of items (e.g., a list of items prefaced by a phrase such as “at least one of’ or “one or more of’ or “one or both of’) indicates an inclusive list such that, for example, a list of at least one of A, B, or C means A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Also, as used herein, the phrase “based on” shall not be construed as a reference to a closed set of conditions. For example, an example step that is described as “based on condition A” may be based on both a condition A and a condition BDocket No. SMM920240279-GR-NPwithout departing from the scope of the present disclosure. In other words, as used herein, the phrase “based on” shall be construed in the same manner as the phrase “based at least in part on. Further, as used herein, including in the claims, a “set” may include one or more elements.

[0004] An apparatus for wireless communication is described. The apparatus may be configured to, capable of, or operable to perform one or more operations as described herein. For example, the apparatus may include at least one memory; and at least one processor coupled with the at least one memory and configured to cause the apparatus to: receive a request for a trained machine learning model; select at least one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus; request feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication; receive feedback information that includes at least one predicted label of at least one feature; update at least one trained label of at least one feature of the machine learning model based on the received feedback information.

[0005] A method performed by an apparatus is described. The method may comprise: receiving a request for a trained machine learning model; selecting at least one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus; requesting feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication; receiving feedback information that includes at least one predicted label of at least one feature; and updating at least one trained label of at least one feature of the machine learning model based on the received feedback information.

[0006] A processor for wireless communication is described. The processor may be configured to, capable of, or operable to perform one or more operations as described herein. For example, the processor may comprise at least one controller coupled with at least one memory and configured to cause the processor to: receive a request for a trained machine learning model; select at least one apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at leastDocket No. SMM920240279-GR-NPone apparatus; request feedback for the at least one feature of the machine learning model from the selected at least one apparatus, wherein the request includes a data indication; receive feedback information that includes at least one predicted label of at least one feature; and update at least one trained label of at least one feature of the machine learning model based on the received feedback information.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 illustrates an example of a wireless communications system in accordance with aspects of the present disclosure.

[0008] Figure 2 illustrates an example of how analytics may be derived and provided to analytics consumer Network Functions (NFs).

[0009] Figure 3 illustrates an example Network Data Analytics Function, NWDAF, architecture for analytics generation based on trained machine learning (ML) models.

[0010] Figure 4 illustrates an example of offline ML model training.

[0011] Figure 5 illustrates a schematic overview of horizontal federated learning.

[0012] Figure 6 illustrates a schematic overview of vertical federated learning.

[0013] Figure 7 illustrates models of vertical federated learning.

[0014] Figure 8 illustrates examples of non-split vertical federated learning.

[0015] Figure 9 illustrates an example of split vertical federated learning.

[0016] Figure 10 illustrates an example of online ML model training.

[0017] Figure 11 illustrates an example architecture for supporting ML model training in accordance with aspects of the present disclosure.

[0018] Figure 12 illustrates an example of a process flow that implements ML model training in accordance with aspects of the present disclosure.

[0019] Figure 13 illustrates an example of a processor 1300 in accordance with aspects of the present disclosure.Docket No. SMM920240279-GR-NP

[0020] Figure 14 illustrates an example of a network equipment (NE) 1400 in accordance with aspects of the present disclosure.

[0021] Figure 15 illustrates a flowchart of a method 1500 performed by a NE in accordance with aspects of the present disclosure.DETAILED DESCRIPTION

[0022] A wireless communication system, including one or more apparatuses which may be in the form of NEs, may co-operate to train and / or update an ML model. Such co-operation may take place in the context of what is known as online learning. During online learning, a participating NE seeking to train or update an ML model may seek feedback on aspects of the model from other NEs. Such feedback might take the form of one or more labels and / or features of the model. It is desirable to implement online learning using radio access architecture (such as 5G, 6G, etc.), and to implement online learning in a robust, accurate, transparent, and repeatable manner so that a training or updating phase may be examined, assessed, and / or recreated where required.

[0023] The present disclosure provides apparatuses, methods, and processors for taking part in improved online learning.

[0024] Aspects of the present disclosure are described in the context of a wireless communications system.

[0025] Figure 1 illustrates an example of a wireless communications system 100 in accordance with aspects of the present disclosure. The wireless communications system 100 may include one or more NE 102, one or more UE 104, and a core network (CN) 106. The wireless communications system 100 may support various radio access technologies. In some implementations, the wireless communications system 100 may be a 4G network, such as an LTE network or an LIE -Advanced (LTE-A) network. In some other implementations, the wireless communications system 100 may be a NR network, such as a 5G network, a 5G- Advanced (5G-A) network, or a 5G ultrawideband (5G-UWB) network. In other implementations, the wireless communications system 100 may be a combination of a 4G network and a 5G network, or other suitable radio access technology including Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEEDocket No. SMM920240279-GR-NP802.20. The wireless communications system 100 may support radio access technologies beyond 5G, for example, 6G. Additionally, the wireless communications system 100 may support technologies, such as time division multiple access (TDMA), frequency division multiple access (FDMA), or code division multiple access (CDMA), etc.

[0026] The one or more NE 102 may be dispersed throughout a geographic region to form the wireless communications system 100. One or more of the NE 102 described herein may be or include or may be referred to as a network node, a base station, a network element, a network function, a network entity, a radio access network (RAN), a NodeB, an eNodeB (eNB), a next-generation NodeB (gNB), or other suitable terminology. An NE 102 and a UE 104 may communicate via a communication link, which may be a wireless or wired connection. For example, an NE 102 and a UE 104 may perform wireless communication (e.g., receive signalling, transmit signalling) over a Uu interface.

[0027] An NE 102 may provide a geographic coverage area for which the NE 102 may support services for one or more UEs 104 within the geographic coverage area. For example, an NE 102 and a UE 104 may support wireless communication of signals related to services (e.g., voice, video, packet data, messaging, broadcast, etc.) according to one or multiple radio access technologies. In some implementations, an NE 102 may be moveable, for example, a satellite associated with a non-terrestrial network (NTN). In some implementations, different geographic coverage areas associated with the same or different radio access technologies may overlap, but the different geographic coverage areas may be associated with different NE 102.

[0028] The one or more UE 104 may be dispersed throughout a geographic region of the wireless communications system 100. A UE 104 may include or may be referred to as a remote unit, a mobile device, a wireless device, a remote device, a subscriber device, a transmitter device, a receiver device, or some other suitable terminology. In some implementations, the UE 104 may be referred to as a unit, a station, a terminal, or a client, among other examples. Additionally, or alternatively, the UE 104 may be referred to as an Internet-of-Things (loT) device, an Internet-of-Everything (loE) device, or machine-type communication (MTC) device, among other examples.Docket No. SMM920240279-GR-NP

[0029] A UE 104 may be able to support wireless communication directly with other UEs 104 over a communication link. For example, a UE 104 may support wireless communication directly with another UE 104 over a device-to-device (D2D) communication link. In some implementations, such as vehicle-to-vehicle (V2V) deployments, vehicle-to-everything (V2X) deployments, or cellular-V2X deployments, the communication link may be referred to as a sidelink. For example, a UE 104 may support wireless communication directly with another UE 104 over a PC5 interface.

[0030] An NE 102 may support communications with the CN 106, or with another NE 102, or both. For example, an NE 102 may interface with other NE 102 or the CN 106 through one or more backhaul links (e.g., SI, N2, N2, or network interface). In some implementations, the NE 102 may communicate with each other directly. In some other implementations, the NE 102 may communicate with each other or indirectly (e.g., via the CN 106. In some implementations, one or more NE 102 may include subcomponents, such as an access network entity, which may be an example of an access node controller (ANC). An ANC may communicate with the one or more UEs 104 through one or more other access network transmission entities, which may be referred to as a radio heads, smart radio heads, or transmission-reception points (TRPs).

[0031] The CN 106 may support user authentication, access authorization, tracking, connectivity, and other access, routing, or mobility functions. The CN 106 may be an evolved packet core (EPC), or a 5G core (5GC), which may include a control plane entity that manages access and mobility (e.g., a mobility management entity (MME), an access and mobility management functions (AMF)) and a user plane entity that routes packets or interconnects to external networks (e.g., a serving gateway (S-GW), a Packet Data Network (PDN) gateway (P-GW), or a user plane function (UPF)). In some implementations, the control plane entity may manage non-access stratum (NAS) functions, such as mobility, authentication, and bearer management (e.g., data bearers, signal bearers, etc.) for the one or more UEs 104 served by the one or more NE 102 associated with the CN 106.

[0032] The CN 106 may communicate with a packet data network over one or more backhaul links (e.g., via an SI, N2, N2, or another network interface). The packet data network may include an application server. In some implementations, one or more UEs 104Docket No. SMM920240279-GR-NPmay communicate with the application server. A UE 104 may establish a session (e.g., a protocol data unit (PDU) session, or the like) with the CN 106 via an NE 102. The CN 106 may route traffic (e.g., control information, data, and the like) between the UE 104 and the application server using the established session (e.g., the established PDU session). The PDU session may be an example of a logical connection between the UE 104 and the CN 106 (e.g., one or more network functions of the CN 106).

[0033] In the wireless communications system 100, the NEs 102 and the UEs 104 may use resources of the wireless communications system 100 (e.g., time resources (e.g., symbols, slots, subframes, frames, or the like) or frequency resources (e.g., subcarriers, carriers)) to perform various operations (e.g., wireless communications). In some implementations, the NEs 102 and the UEs 104 may support different resource structures. For example, the NEs 102 and the UEs 104 may support different frame structures. In some implementations, such as in 4G, the NEs 102 and the UEs 104 may support a single frame structure. In some other implementations, such as in 5G and among other suitable radio access technologies, the NEs 102 and the UEs 104 may support various frame structures (i.e., multiple frame structures). The NEs 102 and the UEs 104 may support various frame structures based on one or more numerologies.

[0034] One or more numerologies may be supported in the wireless communications system 100, and a numerology may include a subcarrier spacing and a cyclic prefix. A first numerology (e.g., / r=0) may be associated with a first subcarrier spacing (e.g., 15 kHz) and a normal cyclic prefix. In some implementations, the first numerology (e.g., / r=0) associated with the first subcarrier spacing (e.g., 15 kHz) may utilize one slot per subframe. A second numerology (e.g., / r=l) may be associated with a second subcarrier spacing (e.g., 30 kHz) and a normal cyclic prefix. A third numerology (e.g., / r=2) may be associated with a third subcarrier spacing (e.g., 60 kHz) and a normal cyclic prefix or an extended cyclic prefix. A fourth numerology (e.g., / r=3) may be associated with a fourth subcarrier spacing (e.g., 120 kHz) and a normal cyclic prefix. A fifth numerology (e.g., / r=4) may be associated with a fifth subcarrier spacing (e.g., 240 kHz) and a normal cyclic prefix.

[0035] A time interval of a resource (e.g., a communication resource) may be organized according to frames (also referred to as radio frames). Each frame may have a duration, forDocket No. SMM920240279-GR-NPexample, a 10 millisecond (ms) duration. In some implementations, each frame may include multiple subframes. For example, each frame may include 10 subframes, and each subframe may have a duration, for example, a 1 ms duration. In some implementations, each frame may have the same duration. In some implementations, each subframe of a frame may have the same duration.

[0036] Additionally, or alternatively, a time interval of a resource (e.g., a communication resource) may be organized according to slots. For example, a subframe may include a number (e.g., quantity) of slots. The number of slots in each subframe may also depend on the one or more numerologies supported in the wireless communications system 100. For instance, the first, second, third, fourth, and fifth numerologies (i.e., / r=0, jU=l, / r=2, jU=3, / r=4) associated with respective subcarrier spacings of 15 kHz, 30 kHz, 60 kHz, 120 kHz, and 240 kHz may utilize a single slot per subframe, two slots per subframe, four slots per subframe, eight slots per subframe, and 16 slots per subframe, respectively. Each slot may include a number (e.g., quantity) of symbols (e.g., OFDM symbols). In some implementations, the number (e.g., quantity) of slots for a subframe may depend on a numerology. For a normal cyclic prefix, a slot may include 14 symbols. For an extended cyclic prefix (e.g., applicable for 60 kHz subcarrier spacing), a slot may include 12 symbols. The relationship between the number of symbols per slot, the number of slots per subframe, and the number of slots per frame for a normal cyclic prefix and an extended cyclic prefix may depend on a numerology. It should be understood that reference to a first numerology (e.g., / i =0) associated with a first subcarrier spacing (e.g., 15 kHz) may be used interchangeably between subframes and slots.

[0037] In the wireless communications system 100, an electromagnetic (EM) spectrum may be split, based on frequency or wavelength, into various classes, frequency bands, frequency channels, etc. By way of example, the wireless communications system 100 may support one or multiple operating frequency bands, such as frequency range designations FR1 (410 MHz - 7.125 GHz), FR2 (24.25 GHz - 52.6 GHz), FR3 (7.125 GHz - 24.25 GHz), FR4 (52.6 GHz - 114.25 GHz), FR4a or FR4-1 (52.6 GHz - 71 GHz), and FR5 (114.25 GHz - 300 GHz). In some implementations, the NEs 102 and the UEs 104 may perform wireless communications over one or more of the operating frequency bands. In someDocket No. SMM920240279-GR-NPimplementations, FR1 may be used by the NEs 102 and the UEs 104, among other equipment or devices for cellular communications traffic (e.g., control information, data). In some implementations, FR2 may be used by the NEs 102 and the UEs 104, among other equipment or devices for short-range, high data rate capabilities.

[0038] FR1 may be associated with one or multiple numerologies (e.g., at least three numerologies). For example, FR1 may be associated with a first numerology (e.g., / r=0), which includes 15 kHz subcarrier spacing; a second numerology (e.g., / r=l), which includes 30 kHz subcarrier spacing; and a third numerology (e.g., / r=2), which includes 60 kHz subcarrier spacing. FR2 may be associated with one or multiple numerologies (e.g., at least 2 numerologies). For example, FR2 may be associated with a third numerology (e.g., / r=2), which includes 60 kHz subcarrier spacing; and a fourth numerology (e.g., / r=3), which includes 120 kHz subcarrier spacing.

[0039] Referring to Figure 2, in 3GPP architecture up to Release 18 a Network Data Analytics Function (NWDAF) provides analytic output to one or more analytics consumer Network Functions (NFs) based on data collected from one or more data producer NFs.

[0040] Referring to Figure 3, in Release 17, an NWDAF is split into an Analytics Logical Function (AnLF) and a Model Training Function (MTLF). An MTLF trains a Machine Learning (ML) model related to an Analytic ID. An ML model is configured by a ML model designer and the MTLF trains it by collecting data from data producer NFs (e.g. location information from an Application Management Function (AMF)) or by receiving historical data from a Data Collection Coordination Function (DCCF) or from an Analytics Data Repository Function (ADRF). The AnLF receives the trained ML models for a specific Analytic ID from the MTLF and uses the model, plus new data from data producer NF(s) / DCCF, to derive requested analytics. The AnLF may subscribe to the MTLF to receive trained ML models for an Analytic ID using a specific Application Programming Interface (API). The AnLF subscribes based on an analytics request from a consumer NF. An NWDAF may support both analytics inference and ML model training functions.Docket No. SMM920240279-GR-NP

[0041] The NWDAF (MTLF) may support one or several ML model training techniques for training an ML model required by an analytics consumer. These are described below with reference to Figures 4-9.

[0042] Referring to Figure 4, offline learning is one method for training an ML model and a schematic representation of an example is shown. In the training phase, the ML model is trained using a fixed dataset (data that has been already collected). The data does not change during the training phase. Offline learning requires extracting the features from the data as well as labelling the data (i.e. indicating the expected outcome for a feature). Once the model is trained, the trained ML model is used for inference (i.e. for predictions), i.e. to make predictions on newly collected data. The ML model cannot be updated once it has been trained. If the ML model needs to be updated, then new data must be collected, and the ML model training is performed. If the ML model needs to be re-trained, then a new training procedure needs to be re-started with updated data collected. Offline learning is commonly used in situations where the dataset has sufficient data available in order to train an ML model to cover the inference needs.

[0043] Federated learning has been researched with the main goal being to resolve privacy and signalling load efficiency. Federated learning allows local model training functions to exchange model parameters and aggregate trained model and / or intermediate training results instead of centrally training models by collecting raw data. It is a distributed machine learning framework that allows a model to be trained collectively from data that is distributed across different data owners. The advantage of federated learning is that artificial intelligence (AI) / ML models can be trained closer to the data source(s), rather than sending raw data to a centralised training node. In federated learning, only parameters / weights or intermediate results of an AI / ML model need to be sent back to the centralized node to assist generic model training.

[0044] Referring to Figures 5 and 6, there are two types of federated learning: horizontal federated learning (HFL); and vertical federated learning (VFL). In HFL, or sample-based federated learning, data sets share the same feature space but have different samples. This is illustrated in Figure 5. VFL or feature-based federated learning is applicable to the cases that two data sets share the same sample space but differ in feature space. This is illustrated inDocket No. SMM920240279-GR-NPFigure 6. In both HFL and VFL, model parameters from each local model training function are sent to a model aggregator to calculate an aggregate model. The model aggregator provides updated model parameters to each MTLF that each MTLF uses to retrain its own model thus allowing every local model training function to have a trained model using data from multiple sources.

[0045] In Federated Machine Learning: Concept and Applications, ACM Transactions on Intelligent Systems and Technology (TIST) Volume 10 Issue 2, Article No. 12, January 2019 and 5GPPP, Al and ML - Enablers for Beyond 5G Networks, examples are provided on how a model aggregator generates aggregated models for each type of federated learning. In Vertical Federated Learning: Taxonomies, Threats, and Prospects, Qun Li et.al., various models for supporting VFL are described and are shown in Figure 7.

[0046] Referring to Figure 8, there are different approaches on how to train a model with VFL depending on whether non-split or split VFL is used which are described in Vertical Federated Learning. In all VFL scenarios, one party is the "label owner" or "active participant", that is, knows how to classify input data and the other parties "passive participants" or "workers" (can be multiple) participate in the VFL training process. In nonsplit VFL, passive participants send intermediate results based on data collected locally using their own model to active participants and the active participants computes gradients / losses using as basis the labels and its own ML model. Some scenarios also involve a coordinator that ensures exchange of messages between VFL parties are encrypted.

[0047] Referring to Figure 9, in split VFL, the model is split between several parties. One party own the top model (label owner / VLF server) and other parties own one or more bottom models (passive participants). The label owner may also be the active participant. In contrast to non-split VFL, the VFL server is aware of the labels and can compute gradient / losses that are shared to passive participants.

[0048] With reference to Figure 10, online learning continuously trains an ML model when new data or real time data is collected. Online learning is useful in situations where there is a vast amount of data collected which makes offline learning difficult to be supported due to the high processing involved or in scenarios where the data changes dynamically over time or in scenarios where data is not available centrally but needs to be collected from localDocket No. SMM920240279-GR-NPdata sources. In 3GPP networks, there may be a case where, for example, a load of a network function changes during peak times (e.g. when user commute to work). One option for online learning is to constantly / periodically monitor the labels and revise by observing the predicted labels and comparing with previously predicted labels.

[0049] Figure 11 illustrates an example architecture 1100 for supporting online learning in accordance with the present disclosure. The architecture 1100 includes a consumer 1102, NWDAFs 1104, 1106 and 1110, and data producers (or sources) 1108. One or more NWDAFs, such as NWDAF 1104, performs online training based on receiving information, such as information from NWDAF 1106, on predicted labels and compares the predicted labels against the ML model's current labels. The NWDAF 1104 may generate updated labels by processing new data, such as data from data producers 1108.

[0050] In embodiments, the NWDAF 1104 either supports or interacts with another node, such as NWDAF 1102, that supports a label creation service for an ML model used for an analytics service (which may be identified by an analytic ID) or an ML model for an AI / ML service (e.g. ML model for AI / ML based positioning), predict the new labels (using e.g., AI / ML logic) based on expected data by inspecting the current new data and / or adjusts current labels considering the variation of old and new in data statistics, i.e., range, density, min-max, standard variation, etc., (using , e.g., moving average).

[0051] In embodiments, NWDAF 1104 supports model training functionality using online learning (Online Training NWDAF - OTNWDAF) and evaluates the labels of an ML model by comparing predicted (or new) labels based on new available data against the existing labels of an ML model (the existing labels may have been developed using older data) and / or against ground truth data. The NWDAF 1104 may thus be referred to as an OTNWDAF.

[0052] Such OTNWDAF requests may be transmitted by the NWDAF 1104 to NWDAF(s), such as NWDAF 1106, that are provisioned with an ML model to provide feedback for a provisioned ML model. NWDAF 1104 may then further train the ML model taking into account one or more of the predicted or new labels, the existing labels of the trained model, and / or the data used for predicting the labels. NWDAF 1104 may be an NWDAF MTLF, an NWDAF AnLF, and / or may support both AnLF and MTLF.Docket No. SMM920240279-GR-NP

[0053] In embodiments, the NWD AF 1104 may monitor predicted or new labels of each feature of an ML model against the ML model existing labels for the feature. The NWDAF 1104 may request other NWDAF(s), such as NWDAF 1106, to provide feedback for predicted labels on a per-ML model feature basis.

[0054] In embodiments, the NWDAF 1104 is aware of what data were used to train the current ML model. To further train the ML model, NWDAF 1104 requests feedback based on "unlabelled data" (i.e. data that have not been used yet for training the ML model). The NWDAF 1104 further trains the ML model by comparing predicted labels from unlabelled data against the trained labels of the trained model by using "labelled data" (i.e., data that have been used to train the ML model). NWDAF 1104 may also take into account ground truth information when further training the ML model.

[0055] In embodiments, the NWDAF 1104 keeps track of labelled and unlabelled data based on data set tag identifier(s). Each identifier may be used to classify the available data between labelled and unlabelled data. Data may also be grouped based on time of day, area of interest, slice, DNN, DNAI, and / or data source identifier (e.g. NF-ID, NF-type) where data were collected. NWDAF 1104 may assign a data set tag identifier to labelled / unlabelled data and update the contents of the data when data have been used for online training. NWDAF 1104 may check what the available data are by interacting with data producers, such as data producers 1108, and / or other NFs, AF in the core network, or other NWDAFZDCCF(s) and classify the data accordingly.

[0056] In embodiments, the NWDAF 1104 online trains the ML model and updates its labels by inspecting the current new data and / or adjusts current labels considering the variation of old and new in data statistics, e.g., range, density, min-max, standard variation, etc., (using, e.g., a moving average).

[0057] In embodiments, at training / inference, the NWDAF 1104 may also select from among candidate NWDAF(s), such as NWDAF 1106, to provide feedback by identifying data locally available at each of the plurality of candidate NWDAFs. When NWDAF 1104 checks the available data, it may obtain information on data set tag identifiers of the available data and corresponding information of the contents of the available data (e.g., the samples available, UE identifiers, feature information, areas of interest, times of day, sliceDocket No. SMM920240279-GR-NPinformation, DNN, DNAI, and / or data source identifiers (e.g. NF ID, NF-set ID). After the available data are identified, the NWDAF 1104 may indicate to each NWDAF 1106 what data to use for providing feedback (e.g., by including a data set tag identifier) and / or may indicate to each NWDAF 1106 the order of the data to provide predictions. In some embodiments, the data to use are unlabelled data. The NWDAF 1104 may request the NWDAF 1106 to transmit to it the data used to generate the feedback.

[0058] In embodiments, the NWDAF 1104 transmits the new derived labels to an ML model owner, such as NWDAF 1110, (in case the ML model is trained is distributed via multiple entities).

[0059] In embodiments, the NWDAF 1104 is collocated with one more NWDAFs, such as NWDAF 1106, that provide the feedback.

[0060] In embodiments, an NWDAF, such as NWDAF 1106, that is provisioned with an ML model trained using online learning provides feedback based on a request from a second NWDAF, such as NWDAF 1104. Feedback may include a prediction of a label of an ML model. An NWDAF, such as NWDAF 1106, that is provisioned with an ML model trained using online learning may provide feedback by applying inference on the provisioned ML model.

[0061] In embodiments, the NWDAF 1106 is configured (e.g., by NWDAF 1104) to provide feedback for one or more features of an ML model. The NWDAF 1106 may provide feedback by predicting one or more labels for one or more features based on data available to the NWDAF 1106.

[0062] In embodiments, the NWDAF 1106, when providing feedback to NWDAF 1104, includes information on the data used for inference and / or feedback. NWDAF 1106 may be an NWDAF ANLF, an NWDAF MTLF, and / or may support both functionalities. The data to use may be on a per-ML model feature basis.

[0063] In embodiments, available data at the NWDAF 1106 is identified based on a data set tag identifier. The NWDAF 1106 may group data and assign a data set tag identifier. Data may be grouped based on times of day, areas of interest, slices, DNN, S-NSSAI, data sourceDocket No. SMM920240279-GR-NPidentifiers (e.g., NF-ID, NF -type) where data were collected. NWDAF 1106 may be an NWDAF that is provisioned with an ML model trained using online learning.

[0064] In embodiments, the NWDAF 1106 receives information on, or an indication of, what data to use to provide feedback. Such information may be provided by NWDAF 1104 when it requests feedback from NWDAF 1106. Such information may additionally include one or more areas of interest, times of date, slice information, DNN or DNAI information, and / or data source identifiers where data were collected. The NWDAF 1104 may include a data set tag identifier corresponding to the data to use to provide feedback. NWDAF 1104 may be an NWDAF ANLF, an NWDAF MTLF and / or support both functionalities.

[0065] In embodiments the NWDAF 1104, before requesting feedback for an ML model trained using online learning, identifies which of a plurality of candidate NWDAF(s) use such ML model and should provide feedback. In other words, The NWDAF 1104 is to select one or more NWDAFs from the candidate NWDAFs to provide feedback. In embodiments, to effect the selection, the NWDAF 1104 constructs a selection list of NWDAFs that provide support for feedback. The NWDAF 1104 may selects one or more NWDAFs based on one or more of the following factors: 1) the NWDAF(s) that use the ML model trained using online learning for inference - the NWDAF(s) that have requested an ML model and NWDAF 1104 determines that the ML model is to be trained using online learning; 2) NWDAF(s) that support the capability for model training using online learning or using online learning based on labels; 3) NWDAF(s) that support the capability for providing feedback for ML models using online learning; 4) the data available at each NWDAF - the data available may be identified by a data set tag identifier, where each data set tag identifier provides information on the data contents (e.g., the samples available, UE identifiers, feature information, areas of interest, times of day, slice information, DNN, and / or DNAI); 5) the feature(s) supported at each NWDAF that can be used to train the ML model; 6) a time of day that data has been collected by each NWDAF; 7) an area of interest (e.g., geographical area) supported by each NWDAF; and / or 8) slice information supported by each NWDAF.

[0066] In embodiments, the NWDAF 1104 interfaces with a Network Repository Function, NRF (not shown), to obtain a list of NWDAFs that support one or more of the above factors or may use as a list the NWDAFs that have requested an ML model or use anDocket No. SMM920240279-GR-NPML model that was trained using online learning. The NWDAF 1104 may also obtain an initial list of candidate NWDAFs from the NRF and then check with each of the candidate NWDAFs to identify data locally available to each of the candidate NWDAFs in order to identify the final list of NWDAF to use for online learning. The NWDAF 1106 may be a selected NWDAF selected from a list of candidate NWDAFs. There may be a plurality of selected NWDAFs 1106.

[0067] Figure 12 illustrates an example of a process flow 1200 in accordance with aspects of the present disclosure. The process flow 1200 may implement or be implemented by aspects of the wireless communication system 100. For example, the process flow 1200 may include one or more consumers 1202, an NWDAF in the form of OTNWDAF 1204, one or more NWDAFs 1206 which may be selected from a list of candidate NWDAFs, and one or more data sources 1208, which may be one or more examples of devices described herein with reference to Figure 1. For instance, an apparatus of the present disclosure may be or comprise OTNWDAF 1204.

[0068] The process flow 1200 may be referred to as a procedure, including one or more operations performed by one or more of the consumers 1202, OTNWDAF 1204, NWDAFs 1206, and / or data sources 1208.

[0069] In Figure 12, the process flow 1200 includes step 1210 wherein a consumer 1202 sends a request for a ML model to an OTNWDAF 1204. The consumer may be an NWDAF and may support inference. The request may be for an ML model for a specific analytics ID or a service from an NWDAF supporting model training.

[0070] The OTNWDAF 1204 determines, at step 1212, that the ML model is to be trained using online learning. The determination may be based on a number of factors, including: an amount of data and / or a type of data available to train a model for the requested analytics ID or service; a variation of the data, i.e., whether or an extent to which the data required to train the model changes dynamically which requires constant training of the model; and / or implementation / deployment scenarios.

[0071] The OTNWDAF 1204 determines what data is available to one or more NWDAFs. The OTNWDAF 1204 may send, at step 1214, a request to one or more dataDocket No. SMM920240279-GR-NPsources 1208 for data needed to train the ML model. The data sources may be data producers. The data may be identified based on one or more Event IDs and / or based on one or more Data Set tag identifiers. The data requested may be chosen or targeted to train one or more features of the ML model. The data producers may include NFs, AFs, NWDAFs, DCCFs, OAMs and / or UEs.

[0072] At step 1216, the OTNWDAF 1204 receives the requested data from the data sources 1208.

[0073] At step 1218, the OTNWDAF 1204 selects one or more NWDAFs 1206 to be involved in the online learning process as described above.

[0074] At step 1220, the OTNWDAF 1204 starts an online learning process by sending to the one or more selected NWDAFs 1206 a request for feedback for the ML model. The request includes a data indication. The data indication may comprise a request for an indication of what data was used by the selected one or more NWDAFs to generate the feedback. The data indication may comprise an indication of data to be used to provide the feedback. The data indication may comprise one or more of the following: an ML model identifier; an indication to provide feedback on predictions or predictions for each feature of the ML model (a feature may be identified by a feature ID; current labels of each feature; an indication of how many samples to use for each feature to provide predictions; an indication that the data for which predictions are to be provided are collected from a specific area of interest; an indication that the data for which predictions are to be provided are collected from a certain time of day range; a data set tag identifier indicating data / samples to use for providing feedback (the data set tag ID may be on per feature basis); and / or default labels for the ML model.

[0075] At step 1222, the one or more selected NWDAFs determine feedback as requested. Determining feedback may include determining labels corresponding to features.

[0076] At step 1224, the OTNWDAF 1204 receives the requested feedback from the one or more selected NWDAFs 1206. The feedback may include an indication of an accuracy of a respective label or may provide one or more predicted labels (on a per feature basis). TheDocket No. SMM920240279-GR-NPfeedback may also include information on the data used to provide feedback. Such data may be identified based on a data set tag identifier.

[0077] At step 1226, the OTNWDAF 1204 trains the ML model using the feedback received at step 1224 and / or the data received in step 1216. The OTNWDAF 1204 may update the labels of each feature of the ML model. The OTNWDAF 1204 may mark the data used by each NWDAF 1206 for providing inference / feedback (e.g., by marking the data as "labelled data").

[0078] At step 1228, the OTNWDAF 1204 provides an updated ML model to one or more NWDAFs. In addition, the OTNWDAF 1204 may send a request for further feedback to provide (for example, with a focus on providing feedback on specific features and / or additional data to use to provide further feedback).

[0079] Alternatively, the OTNWDAF 1204 may send a request for label prediction feedback to one or more NWDAFs 1206 using an ML model that is trained using online learning by using a Service-Based Interface (SBI) service specific for inference (e.g., Nnwdaf_MLModelInference_Request). This alternative involves the following: at step 1220, the OTNWDAF 1204 includes an indication that feedback is requested for an ML model, and at step 1224, the NWDAFs provide feedback via an ML model provisioning service. The OTNWDAF then sends an SBI request for inference (e.g. a Nnwdaf MLModel inference request) including in the request information a feedback indication indicating what type of feedback to respond with. The feedback indication may include a request for label prediction or request on label prediction on per feature basis. The NWDAF provides the requested feedback in the response.

[0080] In the description of the process flow 1200 herein, the operations or signalling performed between one or more of the consumers 1202, OTNWDAF 1204, selected NWDAFs 1206, and / or data sources 1208 may be performed or signalled (e.g., transmitted, received) in a different order than the example order shown, or the operations or signalling performed by one or more of the consumers 1202, OTNWDAF 1204, selected NWDAFs 1206, and / or data sources 1208 may be performed or signalled (e.g., transmitted, received) in different orders or at different times. Some operations or signalling may also be omitted from the process flow 1200. Additionally, although some operations or signalling may beDocket No. SMM920240279-GR-NPshown to occur at different times, these operations or signalling may occur at the same time or in overlapping time periods.

[0081] Figure 13 illustrates an example of a processor 1300 in accordance with aspects of the present disclosure. The processor 1300 may be an example of a processor configured to perform various operations in accordance with examples as described herein. The processor 1300 may include a controller 1302 configured to perform various operations in accordance with examples as described herein. The processor 1300 may optionally include at least one memory 1304, which may be, for example, an L1 / L2 / L3 cache. Additionally, or alternatively, the processor 1300 may optionally include one or more arithmetic-logic units (ALUs) 1306. One or more of these components may be in electronic communication or otherwise coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces (e.g., buses).

[0082] The processor 1300 may be a processor chipset and include a protocol stack (e.g., a software stack) executed by the processor chipset to perform various operations (e.g., receiving, obtaining, retrieving, transmitting, outputting, forwarding, storing, determining, identifying, accessing, writing, reading) in accordance with examples as described herein. The processor chipset may include one or more cores, one or more caches (e.g., memory local to or included in the processor chipset (e.g., the processor 1300) or other memory (e.g., random access memory (RAM), read-only memory (ROM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), static RAM (SRAM), ferroelectric RAM (FeRAM), magnetic RAM (MRAM), resistive RAM (RRAM), flash memory, phase change memory (PCM), and others).

[0083] The controller 1302 may be configured to manage and coordinate various operations (e.g., signalling, receiving, obtaining, retrieving, transmitting, outputting, forwarding, storing, determining, identifying, accessing, writing, reading) of the processor 1300 to cause the processor 1300 to support various operations in accordance with examples as described herein. For example, the controller 1302 may operate as a control unit of the processor 1300, generating control signals that manage the operation of various components of the processor 1300. These control signals include enabling or disabling functional units, selecting data paths, initiating memory access, and coordinating timing of operations.Docket No. SMM920240279-GR-NP

[0084] The controller 1302 may be configured to fetch (e.g., obtain, retrieve, receive) instructions from the memory 1304 and determine subsequent instruct! on(s) to be executed to cause the processor 1300 to support various operations in accordance with examples as described herein. The controller 1302 may be configured to track memory address of instructions associated with the memory 1304. The controller 1302 may be configured to decode instructions to determine the operation to be performed and the operands involved. For example, the controller 1302 may be configured to interpret the instruction and determine control signals to be output to other components of the processor 1300 to cause the processor 1300 to support various operations in accordance with examples as described herein. Additionally, or alternatively, the controller 1302 may be configured to manage flow of data within the processor 1300. The controller 1302 may be configured to control transfer of data between registers, arithmetic logic units (ALUs), and other functional units of the processor 1300.

[0085] The memory 1304 may include one or more caches (e.g., memory local to or included in the processor 1300 or other memory, such RAM, ROM, DRAM, SDRAM, SRAM, MRAM, flash memory, etc. In some implementations, the memory 1304 may reside within or on a processor chipset (e.g., local to the processor 1300). In some other implementations, the memory 1304 may reside external to the processor chipset (e.g., remote to the processor 1300).

[0086] The memory 1304 may store computer-readable, computer-executable code including instructions that, when executed by the processor 1300, cause the processor 1300 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium such as system memory or another type of memory. The controller 1302 and / or the processor 1300 may be configured to execute computer-readable instructions stored in the memory 1304 to cause the processor 1300 to perform various functions. For example, the processor 1300 and / or the controller 1302 may be coupled with or to the memory 1304, the processor 1300, the controller 1302, and the memory 1304 may be configured to perform various functions described herein. In some examples, the processor 1300 may include multiple processors and the memory 1304 may include multiple memories. One or more of the multiple processors may be coupled with one or more of the multipleDocket No. SMM920240279-GR-NPmemories, which may, individually or collectively, be configured to perform various functions herein.

[0087] The one or more ALUs 1306 may be configured to support various operations in accordance with examples as described herein. In some implementations, the one or more ALUs 1306 may reside within or on a processor chipset (e.g., the processor 1300). In some other implementations, the one or more ALUs 1306 may reside external to the processor chipset (e.g., the processor 1300). One or more ALUs 1306 may perform one or more computations such as addition, subtraction, multiplication, and division on data. For example, one or more ALUs 1306 may receive input operands and an operation code, which determines an operation to be executed. One or more ALUs 1306 be configured with a variety of logical and arithmetic circuits, including adders, subtractors, shifters, and logic gates, to process and manipulate the data according to the operation. Additionally, or alternatively, the one or more ALUs 1306 may support logical operations such as AND, OR, exclusive-OR (XOR), not-OR (NOR), and not-AND (NAND), enabling the one or more ALUs 1306 to handle conditional operations, comparisons, and bitwise operations.

[0088] The processor 1300 may support wireless communication in accordance with examples as disclosed herein. The processor 1300 may be configured to or operable to support a means for receiving a request for a trained machine learning model; selecting at least one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus; requesting feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication; receiving feedback information that includes at least one predicted label of at least one feature; and updating at least one trained label of at least one feature of the machine learning model based on the received feedback information.

[0089] Figure 14 illustrates an example of an apparatus in the form of a NE 1400 in accordance with aspects of the present disclosure. The NE 1400 may include a processor 1402, a memory 1404, a controller 1406, and a transceiver 1408. The processor 1402, the memory 1404, the controller 1406, or the transceiver 1408, or various combinations thereof or various components thereof may be examples of means for performing various aspects ofDocket No. SMM920240279-GR-NPthe present disclosure as described herein. These components may be coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces.

[0090] The processor 1402, the memory 1404, the controller 1406, or the transceiver 1408, or various combinations or components thereof may be implemented in hardware (e.g., circuitry). The hardware may include a processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or other programmable logic device, or any combination thereof configured as or otherwise supporting a means for performing the functions described in the present disclosure.

[0091] The processor 1402 may include an intelligent hardware device (e.g., a general- purpose processor, a DSP, a CPU, an ASIC, an FPGA, or any combination thereof). In some implementations, the processor 1402 may be configured to operate the memory 1404. In some other implementations, the memory 1404 may be integrated into the processor 1402. The processor 1402 may be configured to execute computer-readable instructions stored in the memory 1404 to cause the NE 1400 to perform various functions of the present disclosure.

[0092] The memory 1404 may include volatile or non-volatile memory. The memory 1404 may store computer-readable, computer-executable code including instructions when executed by the processor 1402 cause the NE 1400 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium such the memory 1404 or another type of memory. Computer-readable media includes both non- transitory computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A non-transitory storage medium may be any available medium that may be accessed by a general-purpose or specialpurpose computer.

[0093] In some implementations, the processor 1402 and the memory 1404 coupled with the processor 1402 may be configured to cause the NE 1400 to perform one or more of the functions described herein (e.g., executing, by the processor 1402, instructions stored in the memory 1404). For example, the processor 1402 may support wireless communication at the NE 1400 in accordance with examples as disclosed herein. The NE 1400 may be configured to support a means for receiving a request for a trained machine learning model; selecting atDocket No. SMM920240279-GR-NPleast one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus; requesting feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication; receiving feedback information that includes at least one predicted label of at least one feature; and updating at least one trained label of at least one feature of the machine learning model based on the received feedback information.

[0094] The controller 1406 may manage input and output signals for the NE 1400. The controller 1406 may also manage peripherals not integrated into the NE 1400. In some implementations, the controller 1406 may utilize an operating system such as iOS®, ANDROID®, WINDOWS®, or other operating systems. In some implementations, the controller 1406 may be implemented as part of the processor 1402.

[0095] In some implementations, the NE 1400 may include at least one transceiver 1408. In some other implementations, the NE 1400 may have more than one transceiver 1408. The transceiver 1408 may represent a wireless transceiver. The transceiver 1408 may include one or more receiver chains 1410, one or more transmitter chains 1412, or a combination thereof.

[0096] A receiver chain 1410 may be configured to receive signals (e.g., control information, data, packets) over a wireless medium. For example, the receiver chain 1410 may include one or more antennas for receive the signal over the air or wireless medium. The receiver chain 1410 may include at least one amplifier (e.g., a low-noise amplifier (LN A)) configured to amplify the received signal. The receiver chain 1410 may include at least one demodulator configured to demodulate the receive signal and obtain the transmitted data by reversing the modulation technique applied during transmission of the signal. The receiver chain 1410 may include at least one decoder for decoding the processing the demodulated signal to receive the transmitted data.

[0097] A transmitter chain 1412 may be configured to generate and transmit signals (e.g., control information, data, packets). The transmitter chain 1412 may include at least one modulator for modulating data onto a carrier signal, preparing the signal for transmission over a wireless medium. The at least one modulator may be configured to support one or more techniques such as amplitude modulation (AM), frequency modulation (FM), or digitalDocket No. SMM920240279-GR-NPmodulation schemes like phase-shift keying (PSK) or quadrature amplitude modulation (QAM). The transmitter chain 1412 may also include at least one power amplifier configured to amplify the modulated signal to an appropriate power level suitable for transmission over the wireless medium. The transmitter chain 1412 may also include one or more antennas for transmitting the amplified signal into the air or wireless medium.

[0098] Figure 15 illustrates a flowchart of a method 1500 in accordance with aspects of the present disclosure. The operations of the method 1500 may be implemented by a NE as described herein. In some implementations, the NE may execute a set of instructions to control the function elements of the NE to perform the described functions.

[0099] At 1502, the method 1500 may include receiving a request for a trained machine learning model. The operations of 1502 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1502 may be performed by a NE as described with reference to Figure 14.

[0100] At 1504, the method 1500 may include selecting at least one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus. The operations of 1504 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1504 may be performed by a NE as described with reference to Figure 14.

[0101] At 1506, the method 1500 may include requesting feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication. The operations of 1506 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1506 may be performed a NE as described with reference to Figure 14.

[0102] At 1508, the method 1500 may include receiving feedback information that includes at least one predicted label of at least one feature. The operations of 1508 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1508 may be performed by a NE as described with reference to Figure 14.Docket No. SMM920240279-GR-NP

[0103] At 1510, the method 1500 may include updating at least one trained label of at least one feature of the machine learning model based on the received feedback information. The operations of 1510 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1510 may be performed by a NE as described with reference to Figure 14.

[0104] It should be noted that the method 1500 described herein describes a possible implementation, and that the operations and the steps may be rearranged or otherwise modified and that other implementations are possible.

[0105] There is provided an apparatus for wireless communication. The apparatus comprises: at least one memory; and at least one processor coupled with at least one memory and configured to cause the apparatus to: receive a request for a trained machine learning model; select at least one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus; request feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication; receive feedback information that includes at least one predicted label of at least one feature; and update at least one trained label of at least one feature of the machine learning model based on the received feedback information.

[0106] Such an apparatus tends to facilitate improved online learning at least by communicating information regarding what data are used to generate feedback in relation to the machine learning model.

[0107] The apparatus may be / implement a Network Data Analytics Function, NWDAF. The NWDAF may be an Online Training NWDAF, OTNWDAF. The OTNWDAF may be a NWDAF Model Training Logical Function, MTLF, an NWDAF Analytics Logical Function, AnLF, and / or may support both AnLF and MTLF.

[0108] The request may be from an NWDAF.

[0109] The at least one processor may be configured to cause the apparatus to determine that the machine learning model is to be trained using online learning. The determination may be based on amount of data and / or a type of data available to train the ML model. TheDocket No. SMM920240279-GR-NPdetermination may be based on whether and / or an extent to which the data required to train the model changes dynamically thereby requiring continual training of the model. The determination may be based on an implementation and / or a deployment scenario of the model.

[0110] The processor may be configured to cause the apparatus to acquire, from one or more data sources, training data for training the machine learning model. The training data may be identified by an event identifier and / or a data set tag identifier. One or more data sources may comprise a network function, an application function, a further NWDAF, a Data Collection Configuration Function, DCCF, an Operations and Management, 0AM, function, and / or a User Equipment, UE.

[0111] The data available to the at least one further apparatus may not have been used to train the corresponding label of the feature.

[0112] The apparatus may be further configured to transmit the updated machine learning model.

[0113] The data indication may comprise an indication to include in the response the data used when providing feedback.

[0114] The request for an indication of data used to provide the feedback informs the further apparatus of the request that it is to provide information that identifies the data the further apparatus has used to generate the feedback, or to provide the data itself, in addition to providing the feedback. This enables the apparatus to determine whether labels can be updated.

[0115] The data indication may comprise an indication of data to be used to provide the feedback.

[0116] The indicated data may include the available data.

[0117] The indication of data to be used to provide the feedback informs the further apparatus which data to use to generate the feedback. This enables the apparatus to have a degree of control over the way the further apparatus influences the training of the model.

[0118] The available data may be identified by at least one data set tag identifier.Docket No. SMM920240279-GR-NP

[0119] At least one data set tag identifier may: classify data as unlabelled; classify data as labelled; and / or provide information on content of the available data.

[0120] Unlabelled data may comprise data that have not been used to predict a label of a feature. Labelled data may comprise data that have been used to predict a label of a feature.

[0121] The data content may include one or more of: a number of samples available; one or more UE identifiers; feature information; one or more areas of interest; one or more times of day, slice information, one or more Data Network Names, DNNs, and / or one or more Data Network Access Identifiers, DNAIs.

[0122] At least one further apparatus may be or implement a respective further NWDAF.

[0123] The request for feedback may comprise a request for feedback for one or more labels of the machine learning model.

[0124] The request for feedback may comprise a request for data used to generate one or more labels.

[0125] The request for feedback may comprise a request for labels for unlabelled data.

[0126] The one or more labels may be for one or more features of the machine learning model.

[0127] One or more features may be identified with a respective feature identifier.

[0128] The request for feedback may comprise an indication of how many samples to use for each feature.

[0129] A request for feedback may comprise indicating to the at least one further apparatus the identified data locally available to the at least one further apparatus. The indication may comprise a data set tag identifier. The indication may comprise an order of the data for providing one or more predictions. The identified data may be unlabelled data. A request for data may comprise a request for metadata associated with the data. Requested metadata may comprise a time of day, an area of interest, a slice, a Data Network Name, DNN, a Data Network Access Identifier, DNAI, Single Network Slice Selection Assistance Information, S-NSSAI, and / or a data source identifier where data were collected.Docket No. SMM920240279-GR-NP

[0130] A request for feedback may comprise a machine learning model identifier.

[0131] Receiving requested feedback may include receiving one or more indications of label accuracy. Receiving requested feedback may include receiving a data set tag identifier identifying data used to provide the requested feedback.

[0132] Updating the model may be based on both the received requested feedback and on data acquired from one or more data sources prior to the selection of the one or more further apparatuses. Updating the model may comprise adding one or more labels to unlabelled data. Updating the model may comprise replacing or updating one or more labels. Replacing or updating one or more labels may comprise replacing or updating the one or more labels based on a statistical variation. The statistical variation may include a change in one or more of data range, density, min-max, standard deviation, and standard variation.

[0133] The processor may be configured to cause the apparatus to transmit the updated model. The updated model may be transmitted to at least one further NWDAF.

[0134] There is provided a method for wireless communication. The method is performed by an apparatus. The method comprises: receiving a request for a trained machine learning model; selecting at least one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus; requesting feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication; receiving feedback information that includes at least one predicted label of at least one feature; and updating at least one trained label of at least one feature of the machine learning model based on the received feedback information.

[0135] There is provided a processor for wireless communication, comprising: at least one controller coupled with at least one memory and configured to cause the processor to: receive a request for a trained machine learning model; select at least one apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one apparatus; request feedback for the at least one feature of the machine learning model from the selected at least one apparatus, wherein the request includes a data indication; receive feedback information that includes at least oneDocket No. SMM920240279-GR-NPpredicted label of at least one feature; and update at least one trained label of at least one feature of the machine learning model based on the received feedback information.

[0136] The present disclosure provides solutions to support online learning in at least NWDAF architecture of 5G core. The present disclosure addresses at least the following key issues: which Network Functions within the 5G analytics architecture are involved for online learning; during model training using online learning, how feedback is supported regarding the label prediction; how is data processed during online learning; and how to support online learning using labels.

[0137] The present disclosure includes a two-step approach, in which: first, NWDAF(s) that can take part in the online training process are discovered based on, for example, the data available at each NWDAF. Second, each NWDAF is requested to provide feedback, such as by providing information on predicted labels. A central entity that coordinates the online learning may monitor the predicted labels.

[0138] Online learning is an ML model learning technique that has not yet been discussed in 3GPP meetings. However, the online learning method has similarities with the ML model accuracy procedures where a consumer provides feedback on the accuracy of an ML model.

[0139] In existing techniques, feedback provided for ML model accuracy does not contain any information on what data were used to provide feedback, so such techniques cannot be re-used by an NWDAF to determine whether labels can be updated.

[0140] According to the present disclosure, there is provided a method of a first network function that: receives a first request to provide a trained ML model for an analytics or for a service; determines the model is to be trained using online learning; identifies one or more nodes to participate in the model training process using online learning; sends an indication to each participant to provide feedback of the ML model; and, in response to receiving feedback on the labels predicted on per feature, derives updated labels for the ML model and provides an updated ML model to each participant.

[0141] Each participant may be selected based on the locally available data.Docket No. SMM920240279-GR-NP

[0142] Each available data may be identified by a data set tag identifier that indicates the available data.

[0143] Each participant may receive a request to provide feedback on labels of an ML model.

[0144] Each participant may receive a request to provide feedback on the labels of a feature of an ML model.

[0145] A feature may be identified by a feature identifier.

[0146] An indication to provide feedback may include how many samples to use for each feature to provide predictions.

[0147] An indication to provide feedback may include to use data collected from a specific area of interest.

[0148] An indication to provide feedback may include to use data collected from a specific time of day range.

[0149] It should be noted that the method described herein describes a possible implementation, and that the operations and the steps may be rearranged or otherwise modified and that other implementations are possible.

[0150] The description herein is provided to enable a person having ordinary skill in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to a person having ordinary skill in the art, and the generic principles defined herein may be applied to other variations without departing from the scope of the disclosure. Thus, the disclosure is not limited to the examples and designs described herein but is to be accorded the broadest scope consistent with the principles and novel features disclosed herein.

[0151] The following abbreviations are relevant in the field addressed by this document: ADRF - Analytics Data Repository Function; AF - Application Function; AnLF - Analytics Logical Function; DCCF - Data Collection Coordination Function; DNAI - Data Network Access Identifier; DNN - Data Network Name; ML model - Machine Learning Model; MTLF - Model Training Logical Function; NF - Network Function; NWDAF - Network Data Analytics Function; UE - User Equipment; VFL - Vertical Federated Learning.Docket No. SMM920240279-GR-NP

Claims

CLAIMS1. An apparatus for wireless communication, comprising: at least one memory; and at least one processor coupled with at least one memory and configured to cause the apparatus to: receive a request for a trained machine learning model; select at least one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus; request feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication; receive feedback information that includes at least one predicted label of at least one feature; and update at least one trained label of at least one feature of the machine learning model based on the received feedback information.

2. The apparatus of claim 1, wherein the data indication comprises an indication to include in the response the data used when providing feedback.

3. The apparatus of claim 1, wherein the data indication comprises an indication of data to be used to provide the feedback.

4. The apparatus of any preceding claim, wherein the available data are identified by at least one data set tag identifier.

5. The apparatus of claim 4, wherein at least one data set tag identifier: classifies data as unlabelled; classifies data as labelled; and / or provides information on content of the available data.Docket No. SMM920240279-GR-NP6. The apparatus of claim 5, wherein the data content includes one or more of: a number of samples available; one or more UE identifiers; feature information; one or more areas of interest; one or more times of day, slice information, one or more Data Network Names, DNNs, and / or one or more Data Network Access Identifiers, DNAIs.

7. The apparatus of any preceding claim, wherein the request for feedback comprises a request for feedback for one or more labels of the machine learning model.

8. The apparatus of claim 7, wherein the request for feedback comprises a request for data used to generate one or more labels.

9. The apparatus of claim 7 or claim 8, wherein the one or more labels are for one or more features of the machine learning model.

10. The apparatus of claim 9, wherein one or more features are identified with a respective feature identifier.

11. The apparatus of claim 9 or claim 10, wherein the request for feedback comprises an indication of how many samples to use for each feature.

12. A method for wireless communication performed by an apparatus, comprising: receiving a request for a trained machine learning model; selecting at least one further apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one further apparatus; requesting feedback for the at least one feature of the machine learning model from the selected at least one further apparatus, wherein the request includes a data indication; receiving feedback information that includes at least one predicted label of at least one feature; and updating at least one trained label of at least one feature of the machine learning model based on the received feedback information.Docket No. SMM920240279-GR-NP13. The method of claim 12, wherein the data indication comprises an indication to include in the response the data used when providing feedback.

14. The method of claim 12, wherein the data indication comprises an indication of data to be used to provide the feedback.

15. The method of claim 13 or claim 14, wherein the available data are identified by at least one data set tag identifier.

16. The method of claim 15, wherein at least one data set tag identifier: classifies data as unlabelled; classifies data as labelled; and / or provides information on content of the available data.

17. The method of claim 15, wherein the data content includes one or more of: a number of samples available; one or more UE identifiers; feature information; one or more areas of interest; one or more times of day, slice information, one or more Data Network Names, DNNs, and / or one or more Data Network Access Identifiers, DNAIs.

18. The method of any one of claims 12 to 17, wherein the request for feedback comprises a request for feedback for one or more labels of the machine learning model.

19. The method of claim 18, wherein the request for feedback comprises a request for data used to generate one or more labels.

20. A processor for wireless communication, comprising: at least one controller coupled with at least one memory and configured to cause the processor to: receive a request for a trained machine learning model;Docket No. SMM920240279-GR-NPselect at least one apparatus to provide feedback on a machine learning model based on data available for at least one feature of the machine learning model to the at least one apparatus; request feedback for the at least one feature of the machine learning model from the selected at least one apparatus, wherein the request includes a data indication; receive feedback information that includes at least one predicted label of at least one feature; and update at least one trained label of at least one feature of the machine learning model based on the received feedback information.Docket No. SMM920240279-GR-NP

Images