Provision of transfer learning feedback

Abstract

Various aspects of the present disclosure relate to a network entity for wireless communication, comprising at least one memory; and at least one processor coupled with the at least one memory. The at least one processor is configured to cause the network entity to, subsequent to a first analytics function providing transfer learning to a second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics function corresponds to a transfer learning target task: request transfer learning feedback in relation to the second analytics function; and obtain, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task.

Description

PROVISION OF TRANSFER LEARNING FEEDBACKTECHNICAL FIELD

[0001] The present disclosure relates to wireless communications, and more specifically to apparatuses and methods for supporting transfer learning in wireless communication systems.BACKGROUND

[0002] A wireless communications system may include one or multiple network communication devices, such as base stations, which may support wireless communications for one or multiple user communication devices, which may be otherwise known as user equipment (UE), or other suitable terminology. The wireless communications system may support wireless communications with one or multiple user communication devices by utilizing resources of the wireless communication system (e.g., time resources (e.g., symbols, slots, subframes, frames, or the like) or frequency resources (e.g., subcarriers, carriers, or the like). Additionally, the wireless communications system may support wireless communications across various radio access technologies including third generation (3G) radio access technology, fourth generation (4G) radio access technology, fifth generation (5G) radio access technology, among other suitable radio access technologies beyond 5G (e.g., sixth generation (6G)).SUMMARY

[0003] An article “a” before an element is unrestricted and understood to refer to “at least one” of those elements or “one or more” of those elements. The terms “a,” “at least one,” “one or more,” and “at least one of one or more” may be interchangeable. As used herein, including in the claims, “or” as used in a list of items (e.g., a list of items prefaced by a phrase such as “at least one of’ or “one or more of’ or “one or both of’) indicates an inclusive list such that, for example, a list of at least one of A, B, or C means A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Also, as used herein, the phrase “based on” shall not be constmed as a reference to a closed set of conditions. For example, an example step that is described as “based on condition A” may be based on both a condition A and acondition B without departing from the scope of the present disclosure. In other words, as used herein, the phrase “based on” shall be construed in the same manner as the phrase “based at least in part on. Further, as used herein, including in the claims, a “set” may include one or more elements.

[0004] Some implementations of the method and apparatuses described herein may further include a network entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the network entity to, subsequent to a first analytics function providing transfer learning to a second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics function corresponds to a transfer learning target task: request transfer learning feedback in relation to the second analytics function; and obtain, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task.

[0005] The first analytics function may be configured to request the transfer learning feedback by transmitting at least one of: a request for a single transfer learning feedback instance (i.e., on demand); and a subscription request in respect of one or more future transfer learning feedback instances.

[0006] The subscription request may be a request for transfer learning feedback based on at least one of: a transfer learning feedback provision schedule; and responsive to one or more trigger events.

[0007] The performance feedback may comprise a feedback indication indicative of the adopted transfer learning capability.

[0008] The feedback indication may comprise at least one feedback value, wherein the feedback value optionally comprises at least one of a rating parameter, a range, a quantified value, and a score.

[0009] Said feedback indication may be specific to at least one of: one or more of a given usage and a given target task, optionally expressed by the way of an Analytics ID and / or analytics service corresponding to the target task; one or more of a given networkcondition and / or a parameter indicative of a given network environment; one or of a given knowledge transfer capability and / or a given adaptation strategy.

[0010] The performance feedback may comprise at least one of: a performance accuracy indication with respect to the target task; a performance accuracy consistency with respect to time; an indication of model generalization success with respect to the target task; an indication of knowledge transfer adaptability of the model, said indication optionally corresponding to at least one of speed, i.e., the time it takes, and input data samples; and a complexity indicator indicative of required computational and / or communication resources for knowledge transfer exploitation with respect to a given target task.

[0011] The at least one processor may be configured to cause the first analytics function to request the transfer learning feedback by transmitting a transfer learning feedback request comprising at least one of: a time schedule of interest for the transfer learning feedback; a request to provide the performance feedback comprising at least one parameter, which may optionally be a performance parameter, with respect to usage; a request to provide the performance feedback comprising at least one of a knowledge transfer and an adaptation strategy corresponding to the performance feedback; a request to provide the performance feedback with respect to a given event; a request to provide the performance feedback according to a requested reporting style; and a request to provide the performance feedback comprising at least one of an address and a communication and / or storage transaction identity associated with the transfer learning.

[0012] The first analytics function may be configured to authenticate and / or validate the performance feedback provided by the second analytics function.

[0013] The at least one processor may be configured to cause the first network analytics function to provide the transfer learning to the second analytics function.

[0014] The at least one processor may be configured to cause the network entity to determine a subsequent usage of the transfer learning based on the performance feedback.

[0015] The network entity may comprise at least one of the first analytics function and the second analytics function.

[0016] The first analytics function may be a Model Training entity producer (for example a Model Training Function producer) and the second analytics function may be a Model Training entity consumer (for example a Model Training Function consumer).

[0017] The at least one processor may be configured to cause the network entity to determine that the second analytics function supports transfer learning feedback.

[0018] Determining that the second analytics function supports transfer learning feedback may comprise transmitting a feedback learning support inquiry message.

[0019] A model and / or model related information corresponding to the source task may be stored at the first analytics function; and the at least one processor may be configured to cause the network entity to request the transfer learning feedback from the second analytics function.

[0020] A model and / or model related information corresponding to the source task may be stored at a model repository; and the at least one processor may be configured to cause the network entity to: retrieve data indicative of the model from the model repository; and request the transfer learning feedback from the second analytics function.

[0021] The request for transfer learning feedback may be transmitted in one of: a transfer learning model information request message; and a transfer learning feedback request separate from said transfer learning model information request message. The performance feedback may be transmitted in one of: a transfer learning feedback message; and a performance feedback message separate from said transfer learning feedback message.

[0022] In some implementations of the methods and apparatuses described herein, a method, performed by a network entity, comprises, subsequent to a first analytics function providing transfer learning to a second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics functioncorresponds to a transfer learning target task: requesting transfer learning feedback in relation to the second analytics function; and obtaining, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task.

[0023] In some implementations of the methods and apparatuses described herein, a network entity for wireless communication comprises: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the network entity to, subsequent to a first analytics function providing transfer learning to a second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics function corresponds to a transfer learning target task: receive a request for transfer learning feedback in relation to the second analytics function; and provide, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task.BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 illustrates an example of a wireless communications system in accordance with aspects of the present disclosure.

[0025] Figure 2 depicts a NWDAF implementation according to an example.

[0026] Figure 3 depicts an example of transfer learning.

[0027] Figure 4 depicts a process flow according to an example.

[0028] Figure 5 depicts a process flow according to an example.

[0029] Figure 6 illustrates an example of a network equipment (NE) 600 in accordance with aspects of the present disclosure.

[0030] Figure 7 illustrates a flowchart of method performed by a NE in accordance with aspects of the present disclosure.DETAILED DESCRIPTION

[0031] It is being considered to utilize artificial intelligence (AI) / machine learning (ML) techniques in telecommunications systems such as 5G systems. Such models may beused for transfer learning (TL), in which a model trained on one task can be applied to a different task. In such systems, it is desirable to be able to provide a rating or reputation for an AI / ML model and / or AI / ML model parameters or AI / ML model transferable information, for example to quantify its performance for a given task. However, comparative examples which do not implement the present disclosure may not consider how such a rating could be provided, and by which entity, in the context of TL. In addition, since TL can be used for different target tasks, use cases and situations, a single rating may not be sufficient to describe all potential knowledge transfer instances and occasions. The present disclosure provides ways for different ratings to be provided depending on the task and / or with respect to the network environment or network conditions.

[0032] The present disclosure relates to apparatuses and methods for TL feedback. Such feedback can be based on how an AI / ML model performs when adapted and used for a different task and / or in a different network environment. Such a rating may not concentrate on the AI / ML model alone, but can also contain a rating of the knowledge transfer method used or the specific adaptation strategy and provide a rating which reflects how well an AI / ML model was adopted for the target task.

[0033] In AI / ML models and / or AI / ML model information according to comparative examples, feedback and rating is related to accuracy for a specific AI / ML model task, which is the same task for which an AI / ML model is trained. An example of such a method is provided in clause 5C.1 of 3GPP TS 23.288. The feedback for TL, in the present disclosure, can express how an AI / ML model performs when adapted and / or partially employed (in the context of knowledge transfer and / or considering parameter or feature transfer) and used for a different task and / or in a different network environment.

[0034] The present disclosure thus provides improved assessment of model performance in the context of TL, and thus facilitates improved model selection for a particular task and thus overall improved system performance.

[0035] Aspects of the present disclosure are described in the context of a wireless communications system.

[0036] Figure 1 illustrates an example of a wireless communications system 100 in accordance with aspects of the present disclosure. The wireless communications system 100 may include one or more NE 102, one or more UE 104, and a core network (CN) 106. The wireless communications system 100 may support various radio access technologies. In some implementations, the wireless communications system 100 may be a 4G network, such as an LTE network or an LTE- Advanced (LTE-A) network. In some other implementations, the wireless communications system 100 may be a NR network, such as a 5G network, a 5G- Advanced (5G-A) network, or a 5G ultrawideband (5G-UWB) network. In other implementations, the wireless communications system 100 may be a combination of a 4G network and a 5G network, or other suitable radio access technology including Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20. The wireless communications system 100 may support radio access technologies beyond 5G, for example, 6G. Additionally, the wireless communications system 100 may support technologies, such as time division multiple access (TDMA), frequency division multiple access (FDMA), or code division multiple access (CDMA), etc.

[0037] The one or more NE 102 may be dispersed throughout a geographic region to form the wireless communications system 100. One or more of the NE 102 described herein may be or include or may be referred to as a network node, a base station, a network element, a network function, a network entity, a radio access network (RAN), a NodeB, an eNodeB (eNB), a next-generation NodeB (gNB), or other suitable terminology. An NE 102 and a UE 104 may communicate via a communication link, which may be a wireless or wired connection. For example, an NE 102 and a UE 104 may perform wireless communication (e.g., receive signaling, transmit signaling) over a Uu interface.

[0038] An NE 102 may provide a geographic coverage area for which the NE 102 may support services for one or more UEs 104 within the geographic coverage area. For example, an NE 102 and a UE 104 may support wireless communication of signals related to services (e.g., voice, video, packet data, messaging, broadcast, etc.) according to one or multiple radio access technologies. In some implementations, an NE 102 may be moveable, for example, a satellite associated with a non-terrestrial network (NTN). In someimplementations, different geographic coverage areas associated with the same or different radio access technologies may overlap, but the different geographic coverage areas may be associated with different NE 102.

[0039] The one or more UE 104 may be dispersed throughout a geographic region of the wireless communications system 100. A UE 104 may include or may be referred to as a remote unit, a mobile device, a wireless device, a remote device, a subscriber device, a transmitter device, a receiver device, or some other suitable terminology. In some implementations, the UE 104 may be referred to as a unit, a station, a terminal, or a client, among other examples. Additionally, or alternatively, the UE 104 may be referred to as an Internet-of- Things (loT) device, an Intemet-of-Everything (loE) device, or machine-type communication (MTC) device, among other examples.

[0040] A UE 104 may be able to support wireless communication directly with other UEs 104 over a communication link. For example, a UE 104 may support wireless communication directly with another UE 104 over a device-to-device (D2D) communication link. In some implementations, such as vehicle-to-vehicle (V2V) deployments, vehicle-to-everything (V2X) deployments, or cellular-V2X deployments, the communication link may be referred to as a sidelink. For example, a UE 104 may support wireless communication directly with another UE 104 over a PC5 interface.

[0041] An NE 102 may support communications with the CN 106, or with another NE102, or both. For example, an NE 102 may interface with other NE 102 or the CN 106 through one or more backhaul links (e.g., SI, N2, N2, or network interface). In some implementations, the NE 102 may communicate with each other directly. In some other implementations, the NE 102 may communicate with each other or indirectly (e.g., via the CN 106. In some implementations, one or more NE 102 may include subcomponents, such as an access network entity, which may be an example of an access node controller (ANC). An ANC may communicate with the one or more UEs 104 through one or more other access network transmission entities, which may be referred to as a radio heads, smart radio heads, or transmission-reception points (TRPs).

[0042] The CN 106 may support user authentication, access authorization, tracking, connectivity, and other access, routing, or mobility functions. The CN 106 may be anevolved packet core (EPC), or a 5G core (5GC), which may include a control plane entity that manages access and mobility (e.g., a mobility management entity (MME), an access and mobility management functions (AMF)) and a user plane entity that routes packets or interconnects to external networks (e.g., a serving gateway (S-GW), a Packet Data Network (PDN) gateway (P-GW), or a user plane function (UPF)). In some implementations, the control plane entity may manage non-access stratum (NAS) functions, such as mobility, authentication, and bearer management (e.g., data bearers, signal bearers, etc.) for the one or more UEs 104 served by the one or more NE 102 associated with the CN 106.

[0043] The CN 106 may communicate with a packet data network over one or more backhaul links (e.g., via an SI, N2, N2, or another network interface). The packet data network may include an application server. In some implementations, one or more UEs 104 may communicate with the application server. A UE 104 may establish a session (e.g., a protocol data unit (PDU) session, or the like) with the CN 106 via an NE 102. The CN 106 may route traffic (e.g., control information, data, and the like) between the UE 104 and the application server using the established session (e.g., the established PDU session). The PDU session may be an example of a logical connection between the UE 104 and the CN 106 (e.g., one or more network functions of the CN 106).

[0044] In the wireless communications system 100, the NEs 102 and the UEs 104 may use resources of the wireless communications system 100 (e.g., time resources (e.g., symbols, slots, subframes, frames, or the like) or frequency resources (e.g., subcarriers, carriers)) to perform various operations (e.g., wireless communications). In some implementations, the NEs 102 and the UEs 104 may support different resource structures. For example, the NEs 102 and the UEs 104 may support different frame structures. In some implementations, such as in 4G, the NEs 102 and the UEs 104 may support a single frame structure. In some other implementations, such as in 5 G and among other suitable radio access technologies, the NEs 102 and the UEs 104 may support various frame structures (i.e., multiple frame structures). The NEs 102 and the UEs 104 may support various frame structures based on one or more numerologies.

[0045] One or more numerologies may be supported in the wireless communications system 100, and a numerology may include a subcarrier spacing and a cyclic prefix. A firstnumerology (e.g., / r=0) may be associated with a first subcarrier spacing (e.g., 15 kHz) and a normal cyclic prefix. In some implementations, the first numerology (e.g., / r=0) associated with the first subcarrier spacing (e.g., 15 kHz) may utilize one slot per subframe. A second numerology (e.g., / r=l) may be associated with a second subcarrier spacing (e.g., 30 kHz) and a normal cyclic prefix. A third numerology (e.g., / r=2) may be associated with a third subcarrier spacing (e.g., 60 kHz) and a normal cyclic prefix or an extended cyclic prefix. A fourth numerology (e.g., / r=3) may be associated with a fourth subcarrier spacing (e.g., 120 kHz) and a normal cyclic prefix. A fifth numerology (e.g., / r=4) may be associated with a fifth subcarrier spacing (e.g., 240 kHz) and a normal cyclic prefix.

[0046] A time interval of a resource (e.g., a communication resource) may be organized according to frames (also referred to as radio frames). Each frame may have a duration, for example, a 10 millisecond (ms) duration. In some implementations, each frame may include multiple subframes. For example, each frame may include 10 subframes, and each subframe may have a duration, for example, a 1 ms duration. In some implementations, each frame may have the same duration. In some implementations, each subframe of a frame may have the same duration.

[0047] Additionally, or alternatively, a time interval of a resource (e.g., a communication resource) may be organized according to slots. For example, a subframe may include a number (e.g., quantity) of slots. The number of slots in each subframe may also depend on the one or more numerologies supported in the wireless communications system 100. For instance, the first, second, third, fourth, and fifth numerologies (i.e., / r=0, jU=l , / r=2, jU=3, / r=4) associated with respective subcarrier spacings of 15 kHz, 30 kHz, 60 kHz, 120 kHz, and 240 kHz may utilize a single slot per subframe, two slots per subframe, four slots per subframe, eight slots per subframe, and 16 slots per subframe, respectively. Each slot may include a number (e.g., quantity) of symbols (e.g., OFDM symbols). In some implementations, the number (e.g., quantity) of slots for a subframe may depend on a numerology. For a normal cyclic prefix, a slot may include 14 symbols. For an extended cyclic prefix (e.g., applicable for 60 kHz subcarrier spacing), a slot may include 12 symbols. The relationship between the number of symbols per slot, the number of slots per subframe, and the number of slots per frame for a normal cyclic prefix and an extendedcyclic prefix may depend on a numerology. It should be understood that reference to a first numerology (e.g., fi=O) associated with a first subcarrier spacing (e.g., 15 kHz) may be used interchangeably between subframes and slots.

[0048] In the wireless communications system 100, an electromagnetic (EM) spectrum may be split, based on frequency or wavelength, into various classes, frequency bands, frequency channels, etc. By way of example, the wireless communications system 100 may support one or multiple operating frequency bands, such as frequency range designations FR1 (410 MHz - 7.125 GHz), FR2 (24.25 GHz - 52.6 GHz), FR3 (7.125 GHz - 24.25 GHz), FR4 (52.6 GHz - 114.25 GHz), FR4a or FR4-1 (52.6 GHz - 71 GHz), and FR5 (114.25 GHz - 300 GHz). In some implementations, the NEs 102 and the UEs 104 may perform wireless communications over one or more of the operating frequency bands. In some implementations, FR1 may be used by the NEs 102 and the UEs 104, among other equipment or devices for cellular communications traffic (e.g., control information, data). In some implementations, FR2 may be used by the NEs 102 and the UEs 104, among other equipment or devices for short-range, high data rate capabilities.

[0049] FR1 may be associated with one or multiple numerologies (e.g., at least three numerologies). For example, FR1 may be associated with a first numerology (e.g., / r=0), which includes 15 kHz subcarrier spacing; a second numerology (e.g., / z=l), which includes 30 kHz subcarrier spacing; and a third numerology (e.g., / r=2), which includes 60 kHz subcarrier spacing. FR2 may be associated with one or multiple numerologies (e.g., at least 2 numerologies). For example, FR2 may be associated with a third numerology (e.g., / r=2), which includes 60 kHz subcarrier spacing; and a fourth numerology (e.g., / z=3 ), which includes 120 kHz subcarrier spacing.

[0050] Network analytics and artificial intelligence / machine learning AI / ML may be deployed in the 5G core network via the introducing of one or more network data analytics functions (NWDAF) to support various analytics types that can be distinguished using different Analytics IDs, e.g., “UE Mobility”, “NF Load”, etc. as elaborated in 3GPP TS 23.288. Each NWDAF may support one or more Analytics IDs and may have the role of: (i) AI / ML inference called NWDAF AnLF, or (ii) AI / ML training called NWDAF MTLFor (iii) both. An AnLF that supports a specific Analytics ID inference may subscribe to a corresponding MTLF that is responsible for training.

[0051] Figure 2 illustrates an example of various NWDAF deployment flavours and their respective input data sources and the potential consumer for analytics output results. Specifically, as noted above NWDAFs 205 may include a NWDAF AnLF 205a, a NWDAF MTLF 205b, and / or a combination NWDAF 205c. NWDAFs 205 may further include a MTLF FL Server 205d and / or MTLF FL Client 205e. The NWDAF(s) 205 may receive input data from various sources 210 including one or more of 5G core NFs 210a, application functions (AFs) 210b (which may for example be untrusted AFs whose data is received via a network exposure function (NEF) 210bb, data repositories (such as 5G core repositories) 210c, e.g., network repository function (NRF), user data manager (UDM), etc., and operations, administration and maintenance (0AM) data 210d, including performance measurements (PMs), KPIs, configuration management data and alarms. An NWDAF may provide in turn analytics output results to 5G core NF 210a, AFs 210b, data repositories (such as 5G core repositories) 210c and 0AM 210d. Optionally, data collection coordination functionality (DCCF) and messaging framework adaptor function (MFAF) 215 may be involved to distribute and collect repeated data towards or from various data sources.

[0052] Comparative AI / ML mechanisms, which do not implement the present disclosure, adopt an isolated training approach. In such an isolated approach, each AI / ML model is independently trained for a specific usage without considering prior training knowledge or experience. In other words, the instantiation of an AI / ML model in such a comparative example requires training from scratch, assuming that the corresponding NWDAF MTLF is capable to collect enough input data. In cases where the collected input data is not sufficient, this can be reflected in a confidence degree parameter that is provided to the analytics consumer together with the requested output data. The confidence degree indicates the “quality” of the data output (i.e., mainly for a predictive data output) that the NWDAF MTLF has produced considering the amount of collected input data that is ideally needed.

[0053] Collecting enough input data can be challenging, especially when dealing with a newly installed analytics service, since being capable to collect the required input data is not always feasible or can be too expensive in other cases considering the required network and computing resources. Training an ML model from scratch may take time and can impact the analytics service performance, especially when the requested time for AI / ML model training is limited.

[0054] In the present disclosure it is understood that knowledge is a valuable asset and can be preserved and transferred when applicable. This is the objective of TL. TL is a technique that aims to resolve these issues, by reusing an AI / ML model and / or AI / ML model information, i.e., using a pre-trained AI / ML model with respect to a certain task for a new task, exploiting the knowledge gained from a previous task.

[0055] Figure 3 presents a schematic overview of the TL concept. This figure shows a knowledge transfer, e.g., transfer of AI / ML Model X among two different analytics tasks that use different input data sets to produce a different output.

[0056] Figure 3 depicts two analytics tasks: analytics task A 305a and analytics task B 305b. Analytics task A 305a utilizes AI / ML model X 310 to analyse data set A 315a and thereby produce analytics A 320a. Knowledge gained from training the AI / ML model 310 for analytics task A 305a is transferred to analytics task B 305b. Analytics task B 305b accordingly uses the AI / ML model X 310, including the knowledge gained from training for task A 305a, to analyse data set B 315b and thereby produce analytics B 320b.

[0057] In the present disclosure, TL can reuse a pre-trained AI / ML model: (i) as trained or partially trained, i.e., some of its parameters, (ii) as a starting point for further training with respect to a new task, e.g., using new data, or (iii) modified with respect to a new task or environment. A consideration with TL is to decide which part of the knowledge can be transferred to assist a target task, and this can be based on: (i) the relation between the source and targets tasks considering, e.g., if they are the same or similar tasks, if they share the same features or not, (ii) the relation of the environment to which the source and target tasks belong, e.g., whether the domain of the source and target is the same, shares similar characteristics or is different, and (iii) the similarity and characteristics of data used for training in the source and target environments.

[0058] The task and environment relations among the source and target tasks can reveal whether some knowledge may be common such that it may improve the performance of a target domain or task, discover which knowledge can be transferred and under which circumstances. In other words, TL aims to extract the knowledge from one or more source tasks and to apply the knowledge to a target task. Knowledge transfer may be contained in a: (i) context transfer, which describes how to apply the knowledge gained, (ii) shared AI / ML model characteristic, e.g., a feature or parameter, (iii) transformed AI / ML model characteristic, (iv) relation mapping with respect to the applied network environment or usage.

[0059] The notion of knowledge reflects a developed “skill” or ability that can be adopted to a target task. Hence, knowledge is a quality that can be derived initially before being applied considering commonality and the type of skills that can be re-used. Knowledge can also be assessed after usage, i.e., after applying knowledge transfer, building a history record by allowing the consumer, i.e., the target task, to provide feedback related to the TL model performance. This can develop the ability to recognize when and under which conditions to apply TL to be profitable and minimize negative effects.

[0060] TL is currently may be adopted into 5G systems in several scenarios. Registering an AI / ML model in a repository for further use by employing an AI / ML model profile can assist model discovery and can accelerate the adoption of TL. The AI / ML model profile may carry information related to the AI / ML model identifier, the preparation phase, i.e., considering which data sources were used, the time schedule and duration of training, the geographical area and objects used as well as some post training information including how to validate and test the AI / ML model. With respect to TL, the AI / ML model profile may carry information reflecting how a trained AI / ML model and / or AI / ML model parameters or features for a specific task can be transformed to be re-used for another.

[0061] In the application layer, TL has been considered in 3GPP TR 23.700-82 where an AI / ML model can be exchanged directly among the source and target analytics entities or stored and retrieved from a model registry. Registering an AI / ML model for further use may include the following information: (i) an analytics or application service or task identifier, (ii) user(s), i.e., UE, or group of UEs, (iii) service filter information including thegeographical area, time, environment, usage / intent and charging information, (iv) model context information, e.g., set of features, training data requirements, required computing power and (v) permissions, i.e., access, exposure and vendor specifics.

[0062] In addition, the notion of a TL enabler can be implemented, to detect the need for using TL for a certain AI / ML service or task. A subscriber may provide the necessary model context information including model type, features, dataset requirements, required confidence level, UE(s), service profile, location, and time of interest. The TL enabler can discover, fetch a model from the repository and determine whether such model shall be used as a pre-trained AI / ML model considering similarity in features, input data, service, or task, whether a pre-trained model was used in a similar task or based on model rating or reputation of the pre-trained model.

[0063] Although comparative examples may consider model rating and usage reputation as criteria to discover and select an ML model for the purpose of TL, such comparative examples do not consider how this rating is provided and by which entity. In addition, since TL can be used for different situations, a single rating may not be sufficient to describe all potential knowledge transfer cases. Hence, the present disclosure provides for ratings to depend on the task and / or with respect to the network environment or network conditions.

[0064] Such methods, according to the present disclosure, differ significantly from the rating and feedback in conventional AI / ML models. In conventional AI / ML models feedback and rating is related to accuracy for a specific task, which is the same task for which an AI / ML model is trained for. The notion of rating and feedback may be in the context of analytics and / or AI / ML model accuracy monitoring for example as set out in clause 5C.1 of 3GPP TS 23.288. Analytics and / or AI / ML model accuracy monitoring can be performed from the:• AnLF by (i) comparing predictions and its corresponding ground truth data, per Analytic ID, (ii) observed changes of data collection parameters or data distribution from a data source, (iii) comparing existing records or multiple AI / ML models and (iv) obtaining feedback information from the consumer related to the performance impact of the obtained analytics; or the• MTLF, which can determine the AI / ML model degradation by comparing or evaluating input data, analytics output, and the ground truth data.

[0065] The feedback for TL focuses more on how an AI / ML model performs when adapted and used for a different task and / or in a different network environment. In other words, the rating may not only concentrate on the AI / ML model alone, but it may also contain a rating of the knowledge transfer method used or the specific adaptation strategy and provide a rating which shall reflect how well an AI / ML model was adopted for the target task.

[0066] An AI / ML model rating or reputation in the context of TL may utilize consumer feedback. This may be supported by interface enhancements such as introducing new metadata parameters to describe how TL performed considering also the respective knowledge transfer strategy, the characteristic of network environment and the respective task. Such a rating or reputation can serve as a future decision criterion to identify when TL can be beneficial, avoiding negative effects.

[0067] The present disclosure provides for introduction of consumer feedback into a TL service. A consumer of TL or the target task of TL can provide a rating related to the adoption of an AI / ML model and its related knowledge transfer method or adaptation strategy used.

[0068] Such a rating can be a single rating that reflects the combination of AI / ML model and knowledge transfer or adaptation strategy with respect to the usage and network environment. The rating may consider, take into account, or be specific to, at least one of the following parameters:• a target service, e.g., Analytic ID (UE mobility analytics) as per 3GPP TS 23.288, or target service type in where the transfer AI / ML model was used.• a use case, e.g., energy saving optimization, that was assisted by the target service.• a purpose or intent that was requested by the consumer of the target service.• a network slice, i.e., Single-Network Slice Selection Assistance Information (S- NSSAI) where the target service was used.• an edge data network, e.g., Data Network Name (DDN), where the target service was used.• an area or location where the target service was used.• a target object type, i.e., a UE or set of UE, e.g., a pedestrian, a vehicle, or a drone, etc., or an NF type.• a target object quantity, i.e., total amount, or ratio, i.e., percentage of target objects, that were considered when using the transferred AI / ML model.• a network context including the network environment state, e.g., network load, network performance, energy state, maintenance state, network faults, etc., when the target service was used.• a single or a set of input data source types or identifiers related to the target service.• an input data including statistics, e.g., sample range, sample distribution, minimum or maximum sample time distance, standard deviation of samples, etc.,• a time schedule, including start time, time window or duration, periodicity.• an event or set of events associated with the execution of the target service, e.g., surpassing a load threshold or upon a UE movement out of an area of interest.

[0069] In some examples, an alternative way for rating an AI / ML model and the respective knowledge transfer or adaptation strategy can be implemented, e.g., in the case in which case the AI / ML model is the same and the strategy is different or vice versa.

[0070] The knowledge transfer or adaptation strategy may for example involve at least one of the following:• AI / ML model transfer as it was trained for the source task.• AI / ML model transfer and re-training considering context transfer by re-weighting labeled data to reduce negative effect when used for a target task.• AI / ML model transfer and re-trained to be used as a part of a multi-model arrangement, e.g., for ensemble learning.• AI / ML model deep learning layer transfer, i.e., considering common layers.• AI / ML model feature transfer, i.e., considering common features.• AI / ML model feature representation or transformation for a target task.• AI / ML model parameter transfer, i.e., considering common model parameters.• network domain relation mapping, i.e., network domain with similar characteristics or with a certain relation pattern, e.g., AI / ML models used urban environment domains or AI / ML models used in neighboring areas with one impacting the other.• use case relation mapping, e.g., knowledge of user mobility to be applied for network load prediction.• intent relation mapping, i.e., knowledge addressing services with similar requirements.

[0071] One skilled in the art will appreciate that the knowledge transfer or adaptation strategy examples mentioned above provide a non-exhaustive list of representative examples.

[0072] In the context of the present disclosure, the source task may be an NWDAF AnLF that provides an analytic service utilizes a ML model contained in: (i) an NWDAF MTLF that is the model owner, or (ii) a repository, e.g., an analytical data repository function (ADRF), in where the model owner stored an AI / ML model. Alternatively, the source task may contain both AnLF and MTLF functionalities. The target task may provide an analytic service that is based on re-training from an NWDAF MTLF in the same or different domain and may perform the same, partially the same or a different task. Alternatively, the target task may contain both AnLF and MTLF functionalities.

[0073] In an example, in the process of TL, the ML model is initially transferred towards the NWDAF MTLF related to the target task, i.e., the NWDAF MTLF that the target task containing an NWDAF AnLF functionality subscribed or requested AI / ML model provision services. In another example, the ML model is initially transferred towards a target task that contains both AnLF and MTLF functionality.

[0074] In either example described above, the source tasks are related to entities involved in TL, i.e., NWDAF MTLF of the model owner or a functionality related to the repository in where the model owner stored an AI / ML model can request feedback from the target task, i.e., from the NWDAF MTLF related to the target task or the NWDAF that contains both task AnLF and MTLF functionality of the target task. The feedback requested may contain at least one of the following feedback or rating parameters:• performance accuracy indication when the AI / ML model applied on the target task in terms of: o an accuracy score percentage or a grouping / correlation score considering true positive / negative or false positive / negative. o the deviation with respect to the desired given performance accuracy. o the triggering number for re-training.• performance consistency, i.e., can the re-trained AI / ML model perform as expected for: o a desired give time-period. o which time-period, i.e., a time limit. o a desired precision limit. o a desired Fl score and / or F-measure (described as the harmonic mean of the precision and recall of a classification model)• an indication of how well an AI / ML model can generalize to new input data considering also recall or sensitivity.• adaptability speed, i.e., how fast a transferred AI / ML model can be re-trained for a target task and successfully pass the respective validation and testing.• an adaptability effort for re-training in terms of the required o amount of input data needed for the AI / ML model to successfully pass the validation and testing.o input data statistics (e.g., range of samples, maximum time between samples, standard deviation, etc.) needed for the AI / ML model to successfully pass the validation and testing.• model complexity when the AI / ML model is applied to the target task, for example in terms of computing power / resources and memory.

[0075] In order to be able to request feedback for rating a transferred AI / ML model in the context of TL, the NWDAF MTLF related to the target task is configured to be able to support such a feedback rating capability service. This may be indicated before requesting feedback. The entity that requests TL rating feedback may accordingly be authorized to receive feedback with respect to the rating parameters, i.e., be allowed to request feedback related to specific parameters, since providing feedback can be a service with various levels.

[0076] The request for feedback to the target task may be on-demand or a subscription and may contain at least one of the following:• a time schedule and duration of interest related to the performance of the TL AI / ML model.• an indication of interest to report a single parameter or a list of parameters with respect to the usage and network environment related to the TL AI / ML model; these may assist in indicating: o a specific parameter out of a multiple potential parameters of the same kind, e.g., a specific target service out of multiple potential target services, o a plurality of selected parameters out of many different parameters, e.g., a specific target service, a network slice, and an area of interest. an indication of interest to report the knowledge transfer or adaptation strategy used in the transfer AI / ML model (depending on the case more than one may be applied).• a notification address for reporting (where the feedback is requested by an NWDAF), and / or the ADRF ID and transaction storage ID (where the feedback is request by a functionality related to ADRF).• an event indication for triggering a report, e.g., a measurement or condition in relation to a given threshold.• a reporting style, i.e., how the report shall be organized in terms of the information order that it shall contain and / or the formatting of the information.• a correlation ID related to the communication transaction that applies only for the case of the subscription.

[0077] The feedback response from the target task may contain at least one of the following:• a rating parameter or a set of rating parameters of any combination.• a time schedule and duration related to the rating parameter(s).• a parameter or a list of parameters with respect to the usage and network environment related to the TL AI / ML model when the rating took place.• a knowledge transfer or adaptation strategy used in the transfer AI / ML model when the rating took place.• a transaction storage ID in case the rating feedback was request by a functionality related to ADRF.• a correlation ID related to the communication transaction that applies only for the case of the subscription.

[0078] As described herein, embodiments of this disclosure include requesting and providing feedback rating related to a transfer AI / ML model and / or transfer learning knowledge from a target task. The entity requesting feedback rating is the NWDAF MTLF related to the source task that was involved in the TL process. Two different variations are now described, with reference to Figures 4 and 5:(i) the transferred AI / ML model and the respective TL knowledge is available locally at the NWDAF MTLF related to the source task and hence the feedback is requested and provided to the said NWDAF MTLF.(ii) the transferred AI / ML model and the respective TL knowledge is obtained from an ADRF via the NWDAF MTLF related to the source task and hence the feedback information is handled by the said NWDAF MTLF and stored if needed to the specific ADRF that was obtained, introducing TL updates to the AI / ML model ID that was transferred.

[0079] In either example the request and response for TL feedback• can be piggybacked in existing messages, i.e., Nnwdaf MLModelTraininglnfo Request / Response or• can form new separate message, e.g., Nnwdaf MLModelTraininglnfo RequestTLFeedback Nnwdaf MLModelTraininglnfo Request ResponseTLmodelRating

[0080] Variation (i) will now be described with reference to Figure 4. This figure depicts a communication flow between a MTLF TF Model Consumer 405 (implementing a target task) and a MTLF TF Model Producer 410 (implementing a source task). The source task corresponds to analytics task A 305a of Figure 3, and the target task corresponds to analytics task B 305b of Figure 3. Some of the communications depicted may be omitted and / or performed in a different order to that shown. The following description follows the communication numbering of Figure 4.

[0081] In this example, TL is provided by the NWDAF TL Model Producer 410 related to the source task as illustrated in Figure 4. It may be assumed that the desired TL required from the MTLF Model Consumer 405 related to the target task is contained locally into the NWDAF TL Model Producer 410.

[0082] At 0, the MTLF Model Consumer 405 related to the target task selects the MTLF TL Model Producer 410 related to the source task, which can perform TL with the optimal or desired knowledge transfer or knowledge transfer strategy. Discovery of theoptimal MTLF TL Model Producer 410, can be performed via the NRF, assuming that the MTLF TL Model Producer 410 relating to the source task has registered its TL capabilities.

[0083] At 1 , the MTLF Model Consumer 405 may additionally perform a preparation step, after discovering a list of candidate MTLF TL Model Producers. In the preparation step the MTLF Model Consumer 405 can check selected TL capabilities against specific criteria, e.g., how the selected AI / ML model to be transfer is trained at each candidate MTLF TL Model Producer. These TL capabilities can be of a dynamic nature, e.g., training input data, input source types and data statistics, or a TL strategy that rely on the network environment.

[0084] During this preparation step, or in a separate step, e.g., a negotiation step, the MTLF TL Model Producer 410 may check the TL feedback capabilities of the MTLF Model Consumer 405, i.e., if it is capable to provide TL feedback. Based on the indicated TL feedback capabilities, the MTLF TL model producer 410 may allow or restrict knowledge transfer, e.g., in situations in which the provision of TL feedback and model rating is critical for the MTLF TL model producer 410.

[0085] At 2, the MTLF TL Model Consumer 405, following the candidate MTLF Model Producer discovery, preparation and / or negotiation steps can then select the appropriate MTLF Model Producer 410 to obtain knowledge transfer.

[0086] At 3, the MTLF TL Model Consumer 405 requests the desired knowledge transfer indicating a desire for TL and respective parameters, e.g., Analytics ID usage related to the target task, or the type of TL needed, by invoking the Nnwdaf MLModelTraininglnfo Request service operation.

[0087] The MTLF TL Model Consumer may alternatively subscribe to the MTLF Model Producer for regular updates related to knowledge transfer by involving the Nnwdaf_MLModelTraining_Subscribe.

[0088] At 4, the MTLF TL Model Producer 410 authorizes and authenticates the request received with respect to specific TL parameters included in the request, e.g. for a particular AI / ML model, model parameters or other related filters.

[0089] Two different options may then be performed, depending on whether the TL feedback request is to be piggybacked on existing signalling / services or provided as a separate one.

[0090] For the first option, in which the TL feedback request is piggybacked on existing signalling / services, step 5a is performed.

[0091] At 5a, the MTLF TL Model Producer 410 in the response provides TL AI / ML model and other knowledge transfer information, and it also includes a request for receiving TL feedback indicating: (i) the desired time schedule, (ii) the transfer learning parameters of interest, (iii) an indication of the knowledge transfer or adaptation strategy of interest, (iv) filter information, e.g., an event identification, and (v) reporting information, e.g., notification address, reporting style, etc.

[0092] For the second option, in where the TL feedback introduces a new service and respective signaling, steps 5b and 5c are performed.

[0093] At 5b, the MTLF TL Model Producer 410 provides, in the response, TL AI / ML model and other knowledge transfer information.

[0094] At 5c, the MTLF TL Model Producer 410 introduces, in a separate service, a request for receiving TL feedback indicating: (i) the desired time schedule, (ii) the transfer learning parameters of interest, (iii) an indication of the knowledge transfer or adaptation strategy of interest, (iv) filter information, e.g., an event identification, and (v) reporting information, e.g., notification address, reporting style, etc.

[0095] The response in both cases (i.e., steps 5a and 5b) may include: (i) an on-demand request by invoking Nnwdaf MLModelTraininglnfo Request Response service, or (ii) a notification to a subscription by invoking Nnwdaf_MLModelTraining_Notify.

[0096] The new service, i.e., for step 5c, may include: (i) an on-demand request, which can use a service such as, e.g., Nnwdaf MLModelTraininglnfo RequestTLFeedback, or (ii) a subscription, which can use a service such as, e.g., Nnwdaf MLModelTraining TLFeedbackSubscribe.

[0097] At 6, the MTLF TL Model Consumer 405 prepares the TL AI / ML Model obtained, e.g., provides re-training, and the provisions it using the processes described in clause 6.2A of 3GPP TS 23.288 to the respective AnLF responsible for the target task.

[0098] At 7, the MTLF TL Model Consumer 405 performs accuracy checking for the TL AI / ML model as per clause 5.1C of 3GPP TS 23.288. Such an accuracy check can be based on a comparison or deviation of the collected data or based on feedback information received from the AnLF responsible for the target task. The AnLF in turn may compare the predictions and the ground truth data, observe changes in collected data or receive accuracy information from the AnLF consumer.

[0099] According to the performance monitoring of the TL AI / ML Model, the MTLF TL Model Consumer 405 can provide a TL model rating, which can then be shared with the MTLF TL Model Producer 410. The TL feedback can be provided based on different criteria for example: (i) regularly considering a fixed or variable time schedule, (ii) upon a certain event, e.g., user mobility out of an area of interest, or an input data measurement changed beyond a given indication, or (iii) if the AI / ML performance accuracy is beyond or below a certain given threshold.

[0100] The MTLF TL Model Consumer 405 may provide the TL rating feedback as a response to a piggybacked service or as a response to a new service according to the issued TL feedback request. In either case, the feedback provides TL rating that may include at least one of the following: (i) a rating parameter, (ii) a time schedule, (iii) a parameter or a list of parameters with respect to the usage and network environment, (iv) a knowledge transfer or adaptation strategy, and / or (v) the identification of the reporting transaction.

[0101] For the aforementioned first option, wherein the TL feedback request is piggybacked on existing signalling / services, step 8a is performed.

[0102] At 8a, the response can be included, i.e., piggybacked, in the next round of TL request and may address: (i) an on -demand TL feedback request, contained in Nnwdaf MLModelTraininglnfo Request message or (ii) a subscription contained in Nnwdaf_MLModelTraining_Subscribe.

[0103] For the aforementioned second option, wherein the TL feedback request introduces a new service and respective signaling, step 8b is performed.

[0104] At 8b, the response shall also be a new service, which can be, e.g., a response to: (i) an on-demand TL feedback request, contained inNnwdaf MLModelTraininglnfo RequestTLFeedback Response or (ii) a subscription using Nnwdaf_MLModelTraining_TLFeedbackNotify.

[0105] Variation (ii), in which a TL rating is stored in a repository (such as an ADRF) controlled by the NWDAF MTLF model owner 410 related to the source task will now be described with reference to Figure 5. This figure depicts a communication flow between the MTLF TF Model Consumer 405, the MTLF TF Model Producer 410, and a model repository 505 (which may for example be implemented by an ADRF). Some of the communications depicted may be omitted and / or performed in a different order to that shown. The following description follows the communication numbering of Figure 5.

[0106] In this example, TL is provided by the ADRF 505 via the NWDAF TL Model Producer 410 related to the source task. It may be assumed that the desired TL required from the MTLF Model Consumer 405 related to the target task is contained into the ADRF 505, but the access is authenticated by the respective NWDAF TL Model Producer 410, which owns the AI / ML model and the respective parameters eligible for TL.

[0107] At 0 to 3, the MTLF Model Consumer 405 related to the target task discovers and selects the MTLF TL Model Producer 410 via the NRF and may perform additionally a preparation step to check a potential list of candidate MTLF TL Model Producers against specific TL criteria of a dynamic nature, e.g., training input data or the characteristics of the network environment.

[0108] In the preparation step, or in a separate negotiation step, the MTLF TL Model Producer 410 may check if the MTLF Model Consumer 405 is capable to provide TL feedback and based on that it may allow or restrict knowledge transfer. The MTLF TL Model Consumer 405, following the preparation and / or negotiation steps can then select the appropriate MTLF Model Producer 410 to obtain knowledge transfer and can issue an on-demand request, i.e., Nnwdaf MLModelTraininglnfo Request service operation or a subscription to receive regular updates related to the desired knowledge transfer.

[0109] These steps may for example be substantially similar to the corresponding steps described above in relation to the method of Figure 4.

[0110] At 4, the MTLF TL Model Producer 410 authorizes and authenticates the request received with respect to specific TL parameters included in the request, e.g. for a particular AI / ML model, model parameters or other related filters, and determines if the received request concerns an AI / ML model stored in the ADRF 505.

[0111] At 5, if the request gets authorized and authenticated, the MTLF TL Model Producer 410 (or from the ADRF 505 perspective, the ADRF Service Consumer) requests information relate to the TL AI / ML model stored in ADRF 505 by invoking the Nadrf_MLModelManagement_Retrieval request service operation, which includes a Storage Transaction Identifier or at least a unique AI / ML model Identifier(s) or a unique TL indicator or a combination.

[0112] At 6, the ADRF 505 sends an Nadrf_MLModelManagement_Retrieval response service operation, including the address(es) e.g., URL or FQDN, of the TL AI / ML Model file(s) or TL capabilities and other usage and network environment information stored in ADRF 505.

[0113] Once the MTLF TL Model Producer 410 receives the information back from the ADRF 505, it has two different options to reply to the MTLF TL Model Consumer 405 depending on whether the TL feedback request is piggybacked on existing signalling / services or provided as a separate one. It shall be noted that if the ADRF 505 related address(es) e.g., URL or FQDN, of the TL AI / ML Model file(s) is known to the MTLF TL Model Producer 410, then steps 5 and 6 can be omitted.

[0114] For the first option, wherein the TL feedback request is piggybacked on existing signalling / services, step 7a is performed.

[0115] At 7a, the MTLF TL Model Producer 410 in the response provides the ARDF 505 storage information, and it also includes a request for receiving TL feedbackindicating: (i) the desired time schedule, (ii) the transfer learning parameters of interest, (iii) an indication of the knowledge transfer or adaptation strategy of interest, (iv) filter information, e.g., an event identification, and (v) reporting information, e.g., notification address, reporting style, etc.

[0116] For the second option, in which the TL feedback introduces a new service and respective signaling, steps 7b and 7c are performed.

[0117] At 7b, the MTLF TL Model Producer 410 provides, in the response, the ARDF 505 storage information.

[0118] At 7c the MTLF TL Model Producer 410 introduces, in a separate service, a request for receiving TL feedback indicating: (i) the desired time schedule, (ii) the transfer learning parameters of interest, (iii) an indication of the knowledge transfer or adaptation strategy of interest, (iv) filter information, e.g., an event identification, and (v) reporting information, e.g., notification address, reporting style, etc.

[0119] The response in both cases (i.e., steps 7a and 7b) may include: (i) an on-demand request by invoking Nnwdaf MLModelTraininglnfo Request Response service or (ii) a notification to a subscription by invoking Nnwdaf_MLModelTraining_Notify.

[0120] The new service, i.e., for step 7c, may include: (i) an on-demand request, which can use a service such as, e.g., Nnwdaf MLModelTraininglnfo RequestTLFeedback, or (ii) a subscription, which can use a service such as, e.g., Nnwdaf MLModelTraining TLFeedbackSubscribe.

[0121] At 8, once the MTLF TL Model Consumer 405 receives the ADRF (Set) ID where the TL AI / ML model(s) or transfer knowledge requested are stored, it invokes the Nadrf MLModelManagement Retrieval Subscribe or Request including the Storage Transaction Identifier or at least a unique AI / ML model Identifier(s) or a unique TL indicator or a combination.

[0122] At 9, the ADRF 505 then sends Nadrf MLModelManagement Retrieval notify or response, including the desired TL AI / ML Model or the address of at least a TL AI / MLModel file stored in ADRF or the desired knowledge transfer information to the MTLF TL Model Consumer 405.

[0123] At 10-11 , the MTLF TL Model Consumer 405 prepares the TL AI / ML Model obtained, e.g., provides re-training, and the provisions it using the processes described in clause 6.2A of 3GPP TS 23.288 to the respective AnLF responsible for the target task. It then checks for AI / ML model accuracy as per clause 5.1C of 3 GPP TS 23.288 considering both MTLF and AnLF options.

[0124] As with the previous embodiment, the MTLF TL Model Consumer 405 can provide a TL model rating to the MTLF TL Model Producer 410. The TL feedback can be provided on a fixed or variable time schedule, on an event basis or on AI / ML performance accuracy criteria.

[0125] The MTLF TL Model Consumer 405 may provide the TL rating feedback as a response to a piggybacked service or as a response to a new service according to the issued TL feedback request. In either case, the feedback provides TL rating that may include at least one of the following: (i) a rating parameter, (ii) a time schedule, (iii) a parameter or a list of parameters with respect to the usage and network environment, (iv) a knowledge transfer or adaptation strategy, and / or (v) the identification of the reporting transaction.

[0126] For the first option, wherein the TL feedback request is piggybacked on existing signalling / services, step 12a is performed.

[0127] At 12a, the response can be included, i.e., piggybacked, in the next round of TL request and may address: (i) an on -demand TL feedback request, contained in Nnwdaf MLModelTraininglnfo Request message or (ii) a subscription contained in Nnwdaf_MLModelTraining_Subscribe.

[0128] For the second option, in which the TL feedback request introduces a new service and respective signaling, step 12b is performed.

[0129] At 12b, the response shall also be new service, which can be, e.g., a response to: (ii) an on-demand TL feedback request, contained inNnwdaf MLModelTraininglnfo RequestTLFeedback Response or (ii) a subscription using Nnwdaf_MLModelTraining_TLFeedbackNotify.

[0130] At 13, the MTLF TL Model Producer 410, once it receives the TL feedback related to model rating or to rating of the transfer learning knowledge or transfer learning strategy, can verify that is it legitimate by checking: (i) if the list of parameters with respect to the usage and network environment in the target task were indeed not considered during the original training phase of the source task and / or (ii) by validating the AI / ML model with similar data as the one used in the target task, which can be obtained from specified data sources or via the assistance of a digital twin.

[0131] At 14, the MTLF TL Model Producer 410, once it ensures that the TL rating is legitimate, can store it in the ADRF 505 together with the respective TL AI / ML model using the ADRF identifier together with the TL AI / ML model Identifier(s) and invoking the Nadrf_MLModelManagement_Storage Request.

[0132] The present disclosure thus provides effective ways of implementing TL feedback.

[0133] Figure 6 illustrates an example of a NE 600 in accordance with aspects of the present disclosure. The NE 600 may include a processor 602, a memory 604, a controller 606, and a transceiver 608. The processor 602, the memory 604, the controller 606, or the transceiver 608, or various combinations thereof or various components thereof may be examples of means for performing various aspects of the present disclosure as described herein. These components may be coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces.

[0134] The processor 602, the memory 604, the controller 606, or the transceiver 608, or various combinations or components thereof may be implemented in hardware (e.g., circuitry). The hardware may include a processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or other programmable logic device, or any combination thereof configured as or otherwise supporting a means for performing the functions described in the present disclosure.

[0135] The processor 602 may include an intelligent hardware device (e.g., a general- purpose processor, a DSP, a CPU, an ASIC, an FPGA, or any combination thereof). In some implementations, the processor 602 may be configured to operate the memory 604. In some other implementations, the memory 604 may be integrated into the processor 602. The processor 602 may be configured to execute computer-readable instructions stored in the memory 604 to cause the NE 600 to perform various functions of the present disclosure.

[0136] The memory 604 may include volatile or non-volatile memory. The memory 604 may store computer-readable, computer-executable code including instructions when executed by the processor 602 cause the NE 600 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium such the memory 604 or another type of memory. Computer-readable media includes both non- transitory computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A non-transitory storage medium may be any available medium that may be accessed by a general-purpose or special-purpose computer.

[0137] In some implementations, the processor 602 and the memory 604 coupled with the processor 602 may be configured to cause the NE 600 to perform one or more of the functions described herein (e.g., executing, by the processor 602, instructions stored in the memory 604). For example, the processor 602 may support wireless communication at the NE 600 in accordance with examples as disclosed herein. The NE 600 may be configured to support a means for, subsequent to a first analytics function providing transfer learning to a second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics function corresponds to a transfer learning target task: requesting transfer learning feedback in relation to the second analytics function; and obtaining, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task. In other examples, the NE 600 may be configured to support a means for, subsequent to a first analytics function providing transfer learning to a second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics function corresponds to a transfer learning target task: receiving a request for transfer learningfeedback in relation to the second analytics function; and providing, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task.

[0138] The controller 606 may manage input and output signals for the NE 600. The controller 606 may also manage peripherals not integrated into the NE 600. In some implementations, the controller 606 may utilize an operating system such as iOS®, ANDROID®, WINDOWS®, or other operating systems. In some implementations, the controller 606 may be implemented as part of the processor 602.

[0139] In some implementations, the NE 600 may include at least one transceiver 608. In some other implementations, the NE 600 may have more than one transceiver 608. The transceiver 608 may represent a wireless transceiver. The transceiver 608 may include one or more receiver chains 610, one or more transmitter chains 612, or a combination thereof.

[0140] A receiver chain 610 may be configured to receive signals (e.g., control information, data, packets) over a wireless medium. For example, the receiver chain 610 may include one or more antennas for receive the signal over the air or wireless medium. The receiver chain 610 may include at least one amplifier (e.g., a low-noise amplifier (LNA)) configured to amplify the received signal. The receiver chain 610 may include at least one demodulator configured to demodulate the receive signal and obtain the transmitted data by reversing the modulation technique applied during transmission of the signal. The receiver chain 610 may include at least one decoder for decoding the processing the demodulated signal to receive the transmitted data.

[0141] A transmitter chain 612 may be configured to generate and transmit signals (e.g., control information, data, packets). The transmitter chain 612 may include at least one modulator for modulating data onto a carrier signal, preparing the signal for transmission over a wireless medium. The at least one modulator may be configured to support one or more techniques such as amplitude modulation (AM), frequency modulation (FM), or digital modulation schemes like phase-shift keying (PSK) or quadrature amplitude modulation (QAM). The transmitter chain 612 may also include at least one power amplifier configured to amplify the modulated signal to an appropriate power level suitablefor transmission over the wireless medium. The transmitter chain 612 may also include one or more antennas for transmitting the amplified signal into the air or wireless medium.

[0142] Figure 7 illustrates a flowchart of a method in accordance with aspects of the present disclosure. The operations of the method may be implemented by a NE as described herein. In some implementations, the NE may execute a set of instructions to control the function elements of the NE to perform the described functions. The operations of Figure 7 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of Figure 7 may be performed by a NE as described with reference to Figure 6.

[0143] At 702, the method may include, subsequent to a first analytics function providing transfer learning to a second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics function corresponds to a transfer learning target task, requesting transfer learning feedback in relation to the second analytics function.

[0144] At 704, the method may include obtaining, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task.

[0145] It should be noted that the method described herein describes a possible implementation, and that the operations and the steps may be rearranged or otherwise modified and that other implementations are possible.

[0146] The description herein is provided to enable a person having ordinary skill in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to a person having ordinary skill in the art, and the generic principles defined herein may be applied to other variations without departing from the scope of the disclosure. Thus, the disclosure is not limited to the examples and designs described herein but is to be accorded the broadest scope consistent with the principles and novel features disclosed herein.

Claims

CLAIMSWhat is claimed is:

1. A network entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the network entity to: subsequent to a first analytics function providing transfer learning to a second analytics function, request transfer learning feedback in relation to the second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics function corresponds to a transfer learning target task; and obtain, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task.

2. The network entity of claim 1, wherein the first analytics function is configured to request the transfer learning feedback by transmitting at least one of: a request for a single transfer learning feedback instance; and a subscription request in respect of one or more future transfer learning feedback instances.

3. The network entity of claim 2, wherein the subscription request is a request for the transfer learning feedback based on at least one of: a transfer learning feedback provision schedule; and one or more trigger events.

4. The network entity of any preceding claim, wherein the performance feedback comprises a feedback indication indicative of the adopted transfer learning capability.

5. The network entity of claim 4, wherein the feedback indication comprises at least one feedback value, wherein the feedback value optionally comprises at least one of a rating parameter, a range, a quantified value, and a score.

6. The network entity of claim 4 or claim 5, wherein said feedback indication is specific to at least one of: one or more of a given usage and a given target task, expressed by an Analytics ID corresponding to the target task; one or more of a given network condition and / or a parameter indicative of a given network environment; and one or of a given knowledge transfer capability and / or a given adaptation strategy.

7. The network entity of any preceding claim, wherein the performance feedback comprises at least one of: a performance accuracy indication with respect to the target task; a performance accuracy consistency with respect to time; an indication of model generalization success with respect to the target task; an indication of knowledge transfer adaptability of the model, said indication optionally corresponding to at least one of speed and input data samples; and a complexity indicator indicative of required computational and / or communication resources for knowledge transfer exploitation with respect to the given target task.

8. The network entity of any preceding claim, wherein the at least one processor is configured to cause the first analytics function to request the transfer learning feedback by transmitting a transfer learning feedback request comprising at least one of: a time schedule of interest for the transfer learning feedback; a request to provide the performance feedback comprising at least one parameter, which may optionally be a performance parameter, with respect to usage; a request to provide the performance feedback comprising at least one of a knowledge transfer and an adaptation strategy corresponding to the performance feedback; a request to provide the performance feedback with respect to a given event; a request to provide the performance feedback according to a requested reporting style; anda request to provide the performance feedback comprising at least one of an address and a communication and / or storage transaction identity associated with the transfer learning.

9. The network entity of any preceding claim, wherein the first analytics function is configured to authenticate and / or validate the performance feedback provided by the second analytics function.

10. The network entity of any preceding claim, wherein the at least one processor is configured to cause the first network analytics function to provide the transfer learning to the second analytics function.

11. The network entity of any preceding claim, wherein the at least one processor is configured to cause the network entity to determine a subsequent usage of the transfer learning based on the performance feedback.

12. The network entity of any preceding claim, wherein the network entity comprises at least one of the first analytics function and the second analytics function.

13. The network entity of any preceding claim, wherein the first analytics function is a Model Training entity, producer and the second analytics function is a Model Training entity consumer.

14. The network entity of any preceding claim, wherein the at least one processor is configured to cause the network entity to determine that the second analytics function supports the transfer learning feedback.

15. The network entity of claim 14, wherein the determining that the second analytics function supports the transfer learning feedback comprises transmitting a feedback learning support inquiry message.

16. The network entity of any preceding claim, wherein: a model corresponding to the source task is stored at the first analytics function; and the at least one processor is configured to cause the network entity to request the transfer learning feedback from the second analytics function.

17. The network entity of any of claims 1 to 15, wherein: a model corresponding to the source task is stored at a model repository; and the at least one processor is configured to cause the network entity to: retrieve data indicative of the model from the model repository; and request the transfer learning feedback from the second analytics function.

18. The network entity of any preceding claim, wherein the request for the transfer learning feedback is transmitted in one of: a transfer learning model information request message; and a transfer learning feedback request separate from said transfer learning model information request message, and wherein the performance feedback is transmitted in one of: a transfer learning feedback message; and a performance feedback message separate from said transfer learning feedback message.

19. A method, performed by a network entity, comprising: subsequent to a first analytics function providing transfer learning to a second analytics function, requesting transfer learning feedback in relation to the second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics function corresponds to a transfer learning target task; and obtaining, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task.

20. A network entity for wireless communication, comprising at least one memory; andat least one processor coupled with the at least one memory and configured to cause the network entity to: subsequent to a first analytics function providing transfer learning to a second analytics function, receive a request for transfer learning feedback in relation to the second analytics function, wherein the first analytics function corresponds to a transfer learning source task and the second analytics function corresponds to a transfer learning target task; and provide, responsive to the request, performance feedback for an adopted transfer learning capability, of the source task, for the target task.

Images